Multi-scale brain tumor segmentation method and system based on adaptive KAN, and storage medium
By employing an adaptive KAN multi-scale brain tumor segmentation method, which utilizes preprocessing and dynamic B-spline basis function adjustment, parallel dilated convolution, and KAN gating mechanism, the problems of low computational efficiency, poor adaptation to tumor heterogeneity, and insufficient multi-scale context capture in existing technologies are solved, achieving efficient and accurate brain tumor segmentation and meeting the needs of real-time clinical diagnosis.
Patent Information
- Application Number
- CN202511471097.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-15
- Publication Date
- 2026-01-09
AI Technical Summary
Existing brain tumor segmentation techniques have shortcomings in computational efficiency, tumor heterogeneity adaptation, multi-scale context capture, and interpretability, especially in deep learning models, which affect the real-time performance and accuracy of clinical applications.
A multi-scale brain tumor segmentation method based on adaptive Kolmogorov-Arnold network (KAN) is adopted. Through a combined architecture of preprocessing, encoder, bottleneck layer and decoder, feature extraction and fusion are optimized by using Z-score normalization, random data augmentation, dynamic B-spline basis function adjustment, parallel dilated convolution and KAN gating mechanism.
It improves computational efficiency (reducing the number of parameters by 20-40%, FLOPs by 1.5-2.5 times, and training time by 30-50%), enhances the ability to adapt to heterogeneous tumor regions, improves multi-scale context capture and interpretability, and meets the needs of real-time clinical diagnosis.
Smart Images

Figure CN121304700A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and medical image processing technology. More specifically, this invention relates to a multi-scale brain tumor segmentation method, system, and storage medium based on adaptive KAN. Background Technology
[0002] In the field of medical image segmentation, particularly for 3D magnetic resonance imaging (MRI) segmentation of brain tumors, existing techniques primarily rely on deep learning models to process multimodal data, such as FLAIR, T1, T1CE, and T2 sequences, to automatically identify tumor subregions, including the enhancement core, edema, and necrosis. The evolution of these models began with traditional convolutional neural networks (CNNs) and has gradually incorporated Transformers, state-space models (SSMs, such as Mamba), and emerging Kolmogorov-Arnold networks (KANs), aiming to improve segmentation accuracy, interpretability, and computational efficiency. However, despite significant progress, existing methods still suffer from numerous limitations, often stemming from trade-offs in model architecture design, computational resource consumption, and insufficient adaptation to tumor heterogeneity.
[0003] CNN-based segmentation models are the earliest and most widely used type, such as U-Net and its 3D extension. U-Net achieves efficient multi-scale feature fusion through an encoder-decoder structure and skip connections, and is often used in clinical settings to assist in tumor boundary delineation and treatment planning. Its variants, such as 3D U-Net, further adapt to volumetric data by introducing depthwise convolutions to capture spatial dependencies. nnU-Net, as an adaptive framework, further optimizes performance through automatic hyperparameter configuration, helping doctors quickly generate tumor masks in practical medical applications. These CNN models have the advantage of relatively efficient parameters and stable training, but they rely on fixed convolutional kernels and cannot dynamically capture long-range dependencies, resulting in limited accuracy when dealing with blurred tumor boundaries or heterogeneous regions. Furthermore, the black-box nature of CNNs leads to poor clinical interpretability, making it difficult for doctors to understand the model's decision-making process. Computationally, although these models have good parameter control, they incur significant computational overhead and long training times for high-resolution 3D inputs, making deployment particularly difficult in resource-constrained hospital environments and limiting real-time diagnostic support.
[0004] With the rise of Transformers, models have begun to incorporate self-attention mechanisms to model global dependencies. Swin-UNet is a typical example; it uses hierarchical window self-attention to process 3D images, reducing the complexity of standard ViT and capturing non-local features, such as cross-slice context, in tumor segmentation. UNETR combines a CNN encoder and a Transformer decoder, leveraging pre-trained weights to improve generalization. These Transformer-based models excel at capturing non-local tumor features, outperforming pure CNNs in heterogeneous tumor segmentation, such as better integrating multimodal information in radiotherapy planning. However, the drawbacks of Transformers are obvious: self-attention computation is extremely expensive, resulting in a large number of parameters and high computational complexity, leading to long training times. Furthermore, they are sensitive to positional encoding, easily introducing scan order biases in 3D volumes, and lack inherent interpretability; while attention weights can be visualized, they are difficult to intuitively correlate with tumor biological features. In practical applications, the high memory requirements of these Transformer models limit their use in mobile medical devices or edge computing scenarios, such as remote diagnosis or surgical navigation.
[0005] In recent years, the introduction of state-space models such as Mamba has provided a new path for efficient long-range modeling. Mamba-UNet, a variant that embeds Mamba modules into U-Net, uses a selective scanning mechanism to process sequence data, achieving linear complexity modeling of long-range dependencies. In brain tumor segmentation, Mamba-UNet is effective in capturing the context of consecutive slices and is suitable for processing volumetric MRI data. Similar models, such as VM-UNet, further optimize 3D adaptation by capturing multiaxial dependencies through volumetric Mamba blocks. These SSM-based models have the advantages of high efficiency and robustness to sequence order, making them suitable for scenarios requiring rapid processing in clinical settings, such as tumor monitoring or intraoperative guidance. However, Mamba's scan path depends on the input order, which may introduce axial bias, leading to suboptimal performance in 3D symmetric processing and mediocre performance in multimodal fusion, failing to dynamically adjust to tumor heterogeneity. Furthermore, these models have limited interpretability; while the state transition matrix can be analyzed, it is not as intuitive as that of basis function models. While parameter management is relatively well controlled, training stability is poor, requiring careful parameter tuning. Furthermore, in high-dimensional 3D data, the scanning mechanism may cause gradient problems, affecting reliability in high-load medical systems.
[0006] As an emerging paradigm, the Kolmogorov-Arnold network replaces the weight matrix of traditional MLPs with basis functions such as B-splines, providing higher nonlinear expression and intrinsic interpretability. In brain tumor segmentation, U-KAN is a typical KAN-based model, embedding a KAN within a U-Net architecture and utilizing the local support of B-splines to capture multi-scale tumor features, allowing for decision visualization via spline curves in clinical interpretation. Other KAN variants, such as Efficient-KAN, optimize basis function computation to reduce overhead, while FastKAN introduces radial basis function variants to accelerate computation. These models offer advantages in interpretability; basis functions like B-splines allow for quantification of feature contributions, such as tumor boundary curvature, and outperform Transformers in parameter efficiency. However, existing KAN-based models often involve simple splicing, such as using KAN as an additional attention module without deep integration, resulting in poor adaptability to input modalities and an inability to fully respond to the heterogeneous features of multimodal MRI. Computational efficiency is a major pain point: the recursive evaluation of B-spline leads to high computational overhead and long training time, severe parameter redundancy, and no optimized sharing or sparsity mechanism. In addition, although experiments show that B-spline has the best accuracy in basis function selection, it does not combine with a hybrid strategy to balance efficiency and lacks a dynamic adjustment mechanism, which limits its application in real-time clinical segmentation, such as rapid preoperative planning.
[0007] Therefore, it is necessary to design a technical solution that can overcome the above-mentioned defects. Summary of the Invention
[0008] One objective of this invention is to provide a multi-scale brain tumor segmentation method, system, and storage medium based on adaptive KAN, which can effectively solve the problems of low computational efficiency, poor adaptation to tumor heterogeneity, insufficient multi-scale context capture, and weak interpretability in existing brain tumor segmentation methods.
[0009] To achieve these objectives and other advantages of the present invention, according to one aspect of the present invention, a multi-scale brain tumor segmentation method based on adaptive KAN is provided, comprising: S1: preprocessing multimodal three-dimensional brain magnetic resonance imaging data, including Z-score normalization for each modality channel, applying random data augmentation, and calculating modality statistical vectors, wherein the modality statistical vectors include the mean, variance, and skewness of each modality; S2: inputting the preprocessed data into an encoder, the encoder comprising multiple levels, each level performing convolution operations to extract local features, performing spatial KAN processing to extract spatial features, and downsampling feature maps, wherein the spatial KAN processing dynamically adjusts B-splines based on the modality statistical vectors. S3: Input the encoder output into the bottleneck layer, which uses multiple parallel dilated convolution branches to capture multi-scale global context; S4: Input the bottleneck layer output into the decoder, which consists of multiple levels. Each level performs transposed convolution upsampling of feature maps, performs cross-scale gating to fuse encoder skip connection features and decoder features, and performs spatial KAN processing to optimize features, where cross-scale gating uses the KAN gating mechanism; S5: Input the decoder output into the output module, which uses 1×1×1 convolution to reduce the dimensionality of the feature maps to the number of classes and generates multi-class probability segmentation masks through softmax activation.
[0010] Further, in S2, preprocessed data and modality statistics vectors are received, and shallow features are extracted using initial convolution operations to obtain an initial feature map. The initial feature map undergoes multi-level encoding processing. At each level, convolutional block operations are applied to extract local features, spatial KAN processing operations are applied to optimize spatial features, and downsampling operations are applied to reduce the spatial size of the feature map and increase the number of channels. In the spatial KAN processing operation, the feature map is divided into non-overlapping patches and flattened into a token sequence. Based on the modality statistics vectors, the node offsets of the B-spline basis functions are calculated through a meta-adaptive layer, dynamically adjusting the node positions. A pre-computed B-spline table is used to accelerate basis function evaluation through lookup and linear interpolation, and a hybrid basis function is selected according to the current level depth, with Gaussian radial basis functions used for shallow levels and B-spline basis functions used for deeper levels. A progressively stacked KAN layer is used to process the token sequence, with the number of atoms gradually increasing, combined with residual connections and post-processing operations. Post-processing operations include depthwise separable convolution, channel attention mechanisms, and layer normalization.
[0011] Furthermore, in at least one stage of the encoder, after convolutional block operations and / or spatial KAN processing operations, a KAN-enhanced attention module (KBAM) is applied to process the feature map. The KBAM processing includes: Channel attention calculation: global average pooling is performed on the input feature map to generate a channel statistical vector; the channel statistical vector is input into a KAN layer based on a low-order B-spline basis function to generate a channel attention weight vector; the channel attention weight vector is used to perform multiplicative modulation on the input feature map in the channel dimension to obtain a channel-enhanced feature map; Spatial attention calculation: a 1×1×1 convolution is applied to the channel-enhanced feature map for projection; the projected feature map is input into another KAN layer based on a low-order B-spline basis function to generate a spatial attention map; Fusion and output: the spatial attention map and the channel-enhanced feature map are multiplicative modulated in the spatial dimension, and the modulation result is added to the original input feature map through a residual connection to output the KBAM-enhanced feature map; the KBAM-enhanced feature map is used for subsequent downsampling operations in the current encoder stage or stored as a skip connection for use by the decoder.
[0012] Furthermore, the spatial KAN processing operation and the KAN layers in KBAM use a shared coefficient pool mechanism; the shared coefficient pool is a predefined set of globally learnable parameters; when processing token sequences in progressively stacked KAN layers, each sublayer is allocated a dedicated subset of coefficients from the shared coefficient pool for weighted computation in univariate basis function expansion and atomic multiplication coupling; in KBAM, KAN layers based on low-order B-spline basis functions are also allocated a subset of coefficients from the shared coefficient pool.
[0013] Furthermore, the channel attention mechanism in the post-processing operation is a squeeze-excitement channel attention block; the processing of the SE channel attention block includes: performing global average pooling on the input feature map along the spatial dimension to generate channel statistical vectors; inputting the channel statistical vectors into a bottleneck structure consisting of two fully connected layers, where the first fully connected layer is used for dimensionality reduction and the second fully connected layer is used to recover the original number of channels, and applying the Sigmoid activation function to generate the weights of each channel; multiplying the generated channel weights with the original input feature map channel by channel to complete the recalibration of the channel features.
[0014] Further, in S3, the output feature map of the encoder is received, which has reduced spatial resolution and increased number of channels. Multiple parallel dilated convolution branches are applied to the received feature map, each using a 3×3×3 convolution kernel with dilation rates of 1, 2, 4, and 8, respectively, to capture multi-scale context of different receptive fields. Simultaneously, a global average pooling branch is applied to perform global average pooling on the input feature map, followed by channel compression through 1×1×1 convolution and restoration of the original spatial resolution through bilinear upsampling. The outputs of all branches are concatenated along the channel dimension to form a fused feature. The concatenated fused feature is then subjected to dimensionality reduction and integration using 1×1×1 convolution, followed by the addition of residual connections and summation with the original input feature map. Finally, a dropout operation is applied.
[0015] Further, in S4, the output feature map of the bottleneck layer is received as the input to the decoder; for each level of the decoder, a transposed convolutional upsampling operation is performed to increase the spatial size of the feature map and reduce the number of channels; cross-scale gating is performed to fuse the skip connection features from the corresponding level of the encoder and the currently upsampled features, wherein the cross-scale gating includes processing the skip connection features and the upsampled features using two parallel paths respectively, each path applying a depthwise separable convolution, activation function and KAN gating mechanism, then performing multiplication gating, and concatenating the outputs of the two paths along the channel dimension and fusing them through a 1×1×1 convolution; spatial KAN processing is performed on the output of the cross-scale gating, including dividing the feature map into 4×4×4 non-overlapping patches and flattening them into a token sequence, dynamically adjusting the B-spline basis function node positions based on modality statistics vectors through a meta-adaptation layer, using a pre-computed B-spline table to accelerate basis function evaluation, applying progressively stacked KAN layers to process the token sequence, and performing post-processing operations; the output of the spatial KAN processing is used as the input to the next decoder level or the final output of the decoder.
[0016] Furthermore, in S5, the output feature map of the decoder is received; a 1×1×1 convolution operation is applied to the feature map to reduce the number of channels to the number of classes, generating a logits tensor; a softmax activation function is applied to the logits tensor to generate a multi-class probability segmentation mask along the channel dimension.
[0017] According to another aspect of the present invention, the present invention also provides a multi-scale brain tumor segmentation system based on adaptive KAN, comprising: a preprocessing module for preprocessing multimodal three-dimensional brain magnetic resonance imaging data, including Z-score normalization for each modality channel, applying random data augmentation, and calculating modality statistical vectors, wherein the modality statistical vectors include the mean, variance, and skewness of each modality; and an encoder module for receiving the preprocessed data, the encoder module comprising multiple levels, each level being configured to perform convolution operations to extract local features, perform spatial KAN processing to extract spatial features, and downsample the feature map, wherein the spatial KAN processing dynamically adjusts the node positions of the B-spline basis functions based on the modality statistical vectors, and makes... The basis function evaluation is accelerated using a pre-computed B-spline table; the bottleneck layer module receives the encoder output and uses multiple parallel dilated convolution branches to capture multi-scale global context; the decoder module receives the bottleneck layer output and includes multiple stages, each configured to perform transposed convolution to oversample the feature map, perform cross-scale gating to fuse skip connection features from the encoder module with the current decoder features, and perform spatial KAN processing to optimize features, where the cross-scale gating uses the KAN gating mechanism; the output module receives the decoder output and uses 1×1×1 convolution to reduce the dimensionality of the feature map to the number of classes and generates a multi-class probability segmentation mask through a softmax activation function.
[0018] According to another aspect of the present invention, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the aforementioned multi-scale brain tumor segmentation method based on adaptive KAN.
[0019] The present invention has at least the following beneficial effects: This invention effectively addresses the problems of low computational efficiency, poor adaptation to tumor heterogeneity, insufficient multi-scale context capture, and weak interpretability in existing brain tumor segmentation methods. The preprocessing stage eliminates modality bias through Z-score standardization and enhances generalization ability through random data augmentation, laying a solid data foundation for subsequent segmentation. The encoder dynamically adjusts B-spline nodes based on modality statistics to adapt to heterogeneous tumor regions, and pre-computes the B-spline table, significantly reducing the time spent on basis function evaluation, balancing accuracy and efficiency. The bottleneck layer uses parallel dilated convolution to efficiently capture multi-scale global context, avoiding semantic information loss. The decoder employs a KAN gating mechanism to achieve cross-scale feature fusion, taking into account both details and semantics. The output module accurately generates multi-class probability masks. Compared to traditional methods, the number of parameters is reduced by 20-40%, FLOPs are reduced by 1.5-2.5 times, training time is shortened by 30-50%, and B-spline visualization improves interpretability, meeting the needs of real-time clinical diagnosis and treatment planning, and reducing deployment costs in medical scenarios.
[0020] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description
[0021] Figure 1 The original 3D brain scan (left) and the model segmentation output (right); Figure 2 This is a diagram of the overall system architecture of the present invention; Figure 3 This is an architecture diagram of the SKB module of the present invention; Figure 4 This is an architecture diagram of the KBAM module of the present invention; Figure 5 This is an architecture diagram of the K-CSGM module of the present invention. Detailed Implementation
[0022] The present invention will now be described in further detail so that those skilled in the art can implement it based on the description.
[0023] It should be understood that terms such as "having," "comprising," and "including" used in the embodiments of this application do not exclude the presence or addition of one or more other elements or combinations thereof. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of components in a specific posture. If the specific posture changes, the directional indication will also change accordingly. When an element is referred to as "fixed to" or "set on" another element, it can be directly on the other element or may have an intervening element present. When an element is referred to as "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element through an intervening element. Descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features.
[0024] It should be noted that the technical solutions of the various embodiments of this application can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this application.
[0025] In this invention, the term "Kolmogorov-Arnold network" will be referred to as KAN or Kolmogorov-Arnold network. KAN can be implemented in different ways: for example, through different basis functions (this invention relates to B-spline basis functions (also called B-spline functions), RBF (Radial Basis Function) radial basis functions, and the radial basis function mentioned below refers to the Gaussian Radial Basis Function, GRBF), or through different parameter configurations (for example, both are B-spline, but in the shallow SKB, the grid size of the B-spline KAN is set to 5 and the spline order to 3, while in the deep SKB, the grid size of the B-spline KAN is set to 7 and the spline order to 5. These two KANs are considered to be two different KANs because their parameter settings are different).
[0026] The model constructed in this invention is called MSK-Net multimodal multiscale KAN network (see [link]). Figure 2 MSK-Net includes many modules: SKB (Spatial KAN module, see below) Figure 3 ), KBAM (KAN Enhanced Attention Module, see Figure 4 K-CSGM (cross-scale gating module, see K-CSGM) Figure 5 All of these are specific modules in the segmentation model proposed in this invention.
[0027] Brain tumor segmentation is a segmentation task in a three-dimensional context. This invention requires a three-dimensional brain scan image (e.g., CT, MRI, etc.) as input. The model can identify different types of tumors (ET enhancing tumor, NT necrotic tumor core, ED peritumoral edema) and delineate their specific outlines, as shown in the three-view diagram. The left image is the original three-dimensional brain scan image, and the right image shows the different tumor regions and outlines delineated by the model. Different colors represent different types of tumors. See [link to relevant documentation]. Figure 1 .
[0028] This application provides a multi-scale brain tumor segmentation method based on adaptive KAN, comprising: S1: preprocessing multimodal three-dimensional brain magnetic resonance imaging data, including Z-score normalization for each modality channel, applying random data augmentation, and calculating modality statistical vectors, wherein the modality statistical vectors include the mean, variance, and skewness of each modality; S2: inputting the preprocessed data into an encoder, the encoder comprising multiple levels, each level performing convolution operations to extract local features, performing spatial KAN processing to extract spatial features, and downsampling feature maps, wherein the spatial KAN processing dynamically adjusts the node positions of the B-spline basis functions based on the modality statistical vectors, and uses pre-... S3: The B-spline table is computed to accelerate the evaluation of the basis function; S4: The output of the encoder is input into the bottleneck layer, which uses multiple parallel dilated convolution branches to capture multi-scale global context; S5: The output of the bottleneck layer is input into the decoder, which includes multiple levels. Each level performs transposed convolution to upsample the feature map, performs cross-scale gating to fuse encoder skip connection features and decoder features, and performs spatial KAN processing to optimize features, where the cross-scale gating uses the KAN gating mechanism; S6: The output of the decoder is input into the output module, which uses 1×1×1 convolution to reduce the dimensionality of the feature map to the number of classes and generates a multi-class probability segmentation mask through softmax activation.
[0029] For example, preprocessing is performed on multimodal three-dimensional brain magnetic resonance imaging data, which refers to three-dimensional brain MRI image data containing at least four imaging modalities such as FLAIR, T1, T1ce, and T2. This type of data is often used for the clinical diagnosis and analysis of brain tumors. The preprocessing process first performs Z-score standardization, which converts the voxel values of each modality channel into a distribution with a mean close to 0 and a standard deviation close to 1. During the calculation, the mean μ and standard deviation σ of all voxel values of the channel are first calculated, and then the conversion is performed using the formula (voxel value - μ) / (σ + small constant). The small constant can be selected as 1e-5 or 1e-6 to avoid division by zero error when σ is 0. Next, random data augmentation is applied, allowing for flipping along any one of the x, y, or z axes with a flip probability of 0.5 or 0.6. Random rotation is also possible, with an angle range of ±15 or ±20 degrees. Intensity adjustment is also supported, with a scalar factor range of -0.1 to 0.1 or -0.15 to 0.15. These operations are performed only during model training, not during inference to ensure data consistency. Finally, a modality statistics vector is calculated, containing the mean, variance, and skewness for each modality. The mean is the arithmetic mean of all voxel values for that modality; the variance is the average of the squared deviations of voxel values from the mean; and the skewness is the average of the cubed deviations of voxel values from the mean divided by the variance to the power of 1.5, used to quantify the degree of asymmetry in the data distribution. When performing preprocessing, a Dell or HP workstation can be used as the processing device. The workstation can be placed on the workbench in the medical image processing room. Enterprise-grade hard drives can be used for data storage. First, the data is obtained from the hospital's PACS system and transferred to the workstation. Then, the preprocessing is completed in sequence according to the steps described above.
[0030] The preprocessed data is input into the encoder, a module used to extract data features and progressively reduce spatial dimensionality. The encoder can have 5 or 6 levels, each including convolution operations, spatial KAN processing, and downsampling. The convolution operation uses 3×3×3 or 5×5×5 kernels, calculating weighted sums and extracting local features by sliding them across the feature map. Spatial KAN processing is implemented using a Kolmogorov-Arnold network. KAN is a nonlinear model based on basis functions; here, B-spline basis functions are used, which have local support and smoothness. Spatial KAN processing dynamically adjusts the node positions of the B-spline basis functions based on previously calculated modality statistics vectors. During adjustment, a small neural network is used to calculate node offsets, with offset ranges selectable from -0.1 to 0.1 or -0.12 to 0.12. A pre-computed B-spline table is used to accelerate basis function evaluation; this table can be stored in the workstation's memory beforehand. The downsampling operation uses a pooling kernel of size 2×2×2 or 3×3×3, and the stride can be selected as 2 or 3. By taking the maximum value within the pooling window, the feature map space size is reduced, while the number of channels is increased to retain feature information.
[0031] The encoder's output is fed into the bottleneck layer, which captures multi-scale global contextual information. It contains multiple parallel dilated convolutional branches; the number of branches can be selected as 4 or 5. Each branch uses a 3×3×3 convolutional kernel, with dilation rates of 1, 2, 4, 8 or 1, 3, 5, 7. Dilated convolutions expand the receptive field by inserting zeros between kernel elements without increasing computation. During bottleneck layer computation, each parallel branch simultaneously processes the feature map output from the encoder, and the outputs of all branches are then fused to obtain the bottleneck layer result.
[0032] The output of the bottleneck layer is input to the decoder, which is used to restore the feature map resolution and refine the segmentation results. The decoder has the same number of levels as the encoder, either 5 or 6. Each level includes transposed convolutional upsampling, cross-scale gating, and spatial KAN processing. The transposed convolution uses a 2×2×2 or 3×3×3 kernel with a stride of 2 or 3, increasing the spatial size of the feature map through inverse convolution. Cross-scale gating fuses features from skip connections in the encoder with the current features in the decoder. Skip connections directly pass features from the corresponding level in the encoder to the decoder. Cross-scale gating uses a KAN gating mechanism, with the number of KAN atoms selectable as 4 or 8. KAN is used to calculate feature fusion weights for accurate fusion. Spatial KAN processing uses the same parameters as in the encoder to further optimize the fused features.
[0033] The decoder's output is input to the output module. The output module generates the final segmentation result. First, a 1×1×1 convolution kernel is used to reduce the dimensionality of the feature map channels to the number of segmentation categories. For brain tumor segmentation, the number of categories can be 4 or 5: 4 categories for background, whole tumor, tumor core, and enhanced tumor; 5 categories can be added to include edema regions. Then, the output logits are converted into a probability distribution using the softmax activation function. The temperature parameter can be selected as 1.0 or 0.8 during softmax calculation, ultimately generating a multi-class probability segmentation mask. The output module can run on a mobile workstation placed on the doctor's desk. The segmentation result can be displayed on a monitor connected to the mobile workstation.
[0034] In existing technologies, traditional brain tumor segmentation methods are often based on 3D U-Net models. Preprocessing only performs 0-1 normalization on the data, without considering the fine design of modality differences and data augmentation. The encoder only uses ordinary convolution and max pooling, without spatial KAN processing, which cannot dynamically adapt to heterogeneous tumor regions. The bottleneck layer uses a single convolutional branch, which is difficult to capture multi-scale global context. The decoder uses ordinary upsampling, and cross-scale feature fusion is simply concatenated without a KAN gating mechanism. Although the output module uses 1×1 convolution and softmax, the overall segmentation accuracy and efficiency are limited. This embodiment improves data quality through fine preprocessing, adapts spatial KAN processing to tumor heterogeneity, captures global information through a multi-branch bottleneck layer, and fuses cross-scale features through KAN gating. While ensuring segmentation accuracy, it improves computational efficiency through designs such as pre-computation tables. This solves the problems of insufficient adaptation to heterogeneous tumor regions, incomplete capture of multi-scale information, and low computational efficiency in existing technologies, and is more in line with the needs of actual clinical applications.
[0035] In another embodiment, in S2, preprocessed data and modality statistics vectors are received, and shallow features are extracted by applying initial convolution operations to obtain an initial feature map. The initial feature map undergoes multi-level encoding processing, with each level sequentially applying convolutional block operations to extract local features, spatial KAN processing operations to optimize spatial features, and downsampling operations to reduce the spatial size of the feature map and increase the number of channels. In the spatial KAN processing operation, the feature map is divided into non-overlapping patches and flattened into a token sequence. Based on the modality statistics vectors, the node offsets of the B-spline basis functions are calculated through a meta-adaptive layer, dynamically adjusting the node positions. A pre-computed B-spline table is used to accelerate basis function evaluation through lookup and linear interpolation, and a hybrid basis function is selected according to the current level depth, where shallow levels use Gaussian radial basis functions and deep levels use B-spline basis functions. A progressively stacked KAN layer is used to process the token sequence, with the number of atoms gradually increasing, combined with residual connections and post-processing operations. The post-processing operations include depthwise separable convolution, channel attention mechanisms, and layer normalization.
[0036] For example, the system receives preprocessed data and modality statistics vectors. The preprocessed data consists of multimodal 3D brain MRI data that has undergone Z-score normalization and random data augmentation, typically with a shape of [B, 4, H, W, D], where B is the batch size, 4 is the number of modalities, and H, W, and D are the spatial dimensions, which can be 128 or 256. The modality statistics vector is a vector containing the mean, variance, and skewness of each modality, with a shape of [B, 4, 3], where 3 corresponds to the three statistics. Subsequently, initial convolution operations are applied to extract shallow features. The initial convolution uses 3×3×3 or 5×5×5 kernels, with the number of kernels selectable as 32 or 64. Weighted sums are calculated by sliding across the input data. After each convolution operation, batch normalization and ReLU activation are performed. Batch normalization normalizes the features by calculating the mean and variance of the data within the batch, while the ReLU activation function sets values less than 0 to 0 and retains values greater than 0.
[0037] The initial feature map undergoes multi-level encoding processing, with 5 or 6 levels selectable. Each level sequentially performs convolutional block operations, spatial KAN processing, and downsampling. The convolutional block operation consists of two consecutive convolutional layers, each using a 3×3×3 kernel. The number of kernels can double with each level, such as 32 in the first level and 64 in the second. Batch normalization and ReLU activation are performed after each convolutional layer, and finally, residual connections are added to sum the input and output of the convolutional block to alleviate the vanishing gradient problem. The spatial KAN processing first divides the feature map into non-overlapping patches, with patch sizes selectable as 4×4×4 or 6×6×6. After segmentation, each patch is flattened into a token sequence. The length of the token sequence is the product of the number of voxels and the number of channels within the patch; for example, a 4×4×4×32 patch has a flattened length of 2048. Next, based on the modal statistics vector, the node offset of the B-spline basis function is calculated through the meta-adaptive layer. The meta-adaptive layer is a small multilayer perceptron with an input dimension of 3 (number of statistics). The hidden layer dimension can be selected as 32 or 64. The output dimension is consistent with the number of nodes of the B-spline basis function, and the number of nodes can be selected as 5 or 7. The offset is calculated through the meta-adaptive layer. The offset range can be selected as -0.1 to 0.1 or -0.12 to 0.12, thereby dynamically adjusting the position of the B-spline nodes.
[0038] The evaluation of basis functions is accelerated using a pre-computed B-spline table. This table is pre-computed and stores the B-spline basis function values corresponding to different input values. The number of rows in the table is the sum of the number of nodes and the spline order plus one. The spline order can be 3 or 5, and the number of columns is the number of discrete sampling points, which can be 1000 or 2000. During evaluation, the accurate value is obtained by looking up the table and performing linear interpolation. Simultaneously, a mixed basis function is selected based on the current level depth. If the current level is the first two levels (shallow level), the Gaussian radial basis function is used, and the calculation is as follows: `center` and `sigma` are learnable parameters. This choice is efficient due to its non-recursive nature and is suitable for shallow, simple features. If the current level is level 3 or higher (deep level), B-spline basis functions are used. For each input `t`, the corresponding row is found in the table (based on grid points), and the accurate basis function value is obtained through linear interpolation, avoiding real-time calculation of recursive basis functions. The B-spline recursive formula is as follows: .
[0039] A progressively stacked KAN layer is used to process the token sequence. The number of KAN layers can be selected as 3 or 4, with the number of atoms increasing progressively, such as 4 layers in the first layer, 8 layers in the second layer, and 16 layers in the third layer, or 6 layers in the first layer, 10 layers in the second layer, and 14 layers in the third layer. Each KAN layer first expands each dimension of the token sequence with a univariate basis function, then performs atomic multiplication coupling, and finally performs weighted aggregation. Residual connections are added to each layer to sum the layer input and output. Post-processing operations include depthwise separable convolution, channel attention mechanism, and layer normalization. Depthwise separable convolution first performs a 1×1×1 convolution to adjust the number of channels, and then performs a 3×3×3 depthwise convolution to extract spatial features. The channel attention mechanism is a squeeze-excitation (SE) block, which first performs global average pooling on the feature map, and then adjusts the channel weights through two fully connected layers. The number of neurons in the fully connected layer can be selected as 1 / 16 or 1 / 8 of the number of channels. Layer normalization normalizes the features by calculating the mean and variance of the data within the layer. The post-processing operations can be performed on a server located in a data center rack. The downsampling operation uses a max pooling kernel of size 2×2×2 or 3×3×3 with a stride of 2 or 3 to reduce the feature map spatial size and increase the number of channels, and then passes the processed feature map to the next level.
[0040] In existing technologies, traditional encoders processing multimodal MRI data use convolutional blocks without residual connections, rely solely on ordinary convolution for spatial feature processing, cannot dynamically adjust basis functions, lack pre-computation tables for basis function evaluation, and do not differentiate between shallow and deep levels when selecting basis functions. Downsampling simply reduces the size, resulting in insufficient feature extraction and low computational efficiency. This embodiment enhances feature transfer through convolutional blocks with residual connections, dynamically adjusts basis functions to adapt to tumor features using spatial KAN processing, improves evaluation efficiency through pre-computation tables, balances shallow and deep level requirements with mixed basis functions, and optimizes feature quality through post-processing operations. This solves the problems of weak feature extraction capability, low computational efficiency, and insufficient adaptation to heterogeneous tumor regions in existing technologies, enabling more efficient extraction of multi-scale brain tumor features.
[0041] In another embodiment, in at least one stage of the encoder, after convolutional block operations and / or spatial KAN processing operations, a KAN-enhanced attention module (KBAM) is applied to process the feature map. The KBAM processing includes: channel attention calculation: performing global average pooling on the input feature map to generate a channel statistical vector; inputting the channel statistical vector into a KAN layer based on a low-order B-spline basis function to generate a channel attention weight vector; using the channel attention weight vector to perform multiplicative modulation on the input feature map in the channel dimension to obtain a channel-enhanced feature map; spatial attention calculation: applying a 1×1×1 convolution to the channel-enhanced feature map for projection; inputting the projected feature map into another KAN layer based on a low-order B-spline basis function to generate a spatial attention map; fusion and output: performing multiplicative modulation on the spatial dimension of the spatial attention map and the channel-enhanced feature map, and adding the modulation result to the original input feature map through a residual connection to output a KBAM-enhanced feature map; the KBAM-enhanced feature map is used for subsequent downsampling operations in the current encoder stage or stored as a skip connection for use by the decoder.
[0042] For example, in at least one stage of the encoder, after convolutional block operations and / or spatial KAN processing operations, a KAN-enhanced attention module (KBAM) is applied to process the feature map. Here, the feature map is a three-dimensional feature map output after convolutional block or spatial KAN processing, with a shape of [B, C, H, W, D], where B is the batch size, C is the number of channels, and H, W, and D are the spatial dimensions. C can be 64 or 128, and H, W, and D can also be 64 or 128. The KBAM processing first performs channel attention calculation, then performs global average pooling on the input feature map, i.e., calculates the average value of all voxel values for each channel, generating a channel statistical vector with a shape of [B, C]. This operation can be performed on a workstation located in a medical image processing room. The channel statistical vector is then input into a KAN layer based on a low-order B-spline basis function. The order of the low-order B-spline can be 2 or 3, and the number of nodes can be 5 or 7. The KAN layer generates channel attention weight vectors through basis function expansion and weighted aggregation. The shape of the weight vector is consistent with the channel statistical vector, and the value ranges from 0 to 1. The channel attention weight vector is used to perform multiplicative modulation on the input feature map along the channel dimension, that is, all voxel values of each channel are multiplied by the weight corresponding to that channel to obtain the channel-enhanced feature map. The modulation operation is performed on the CPU of the workstation.
[0043] Next, spatial attention is calculated. A 1×1×1 convolution is applied to project the channel-enhanced feature map. The number of convolution kernels can be chosen as 1 or 2 to reduce the number of channels, which facilitates subsequent calculations. The projected feature map is then input into another KAN layer based on a low-order B-spline basis function. The parameters of this KAN layer are consistent with those of the KAN layer in the channel attention calculation: order 2 or 3, and number of nodes 5 or 7. This generates a spatial attention map with the shape [B, 1, H, W, D] (if the number of convolution kernels is 1) or [B, 2, H, W, D] (if the number of convolution kernels is 2), with values ranging from 0 to 1.
[0044] Finally, the spatial attention map and the channel-enhanced feature map are fused and output. The spatial attention map and the channel-enhanced feature map are multiplied and modulated along the spatial dimension. If the spatial attention map has one channel, it is directly multiplied voxel-by-voxel with the channel-enhanced feature map. If it has two channels, it is first concatenated along the channel dimension with the channel-enhanced feature map, and then multiplied after restoring the channel number to C using a 1×1×1 convolution. The modulated result is then added to the original input feature map via residual connections, i.e., voxel-by-voxel addition, to obtain the KBAM-enhanced feature map. The addition operation is performed in the workstation's memory. The KBAM-enhanced feature map, if it represents a stage other than the last stage of the encoder, is used for subsequent downsampling operations at the current encoder level, using a 2×2×2 max-pooling kernel. If it represents the last few stages, it is stored as a skip connection on the server's hard drive for use by the decoder.
[0045] In existing technologies, traditional attention modules are mostly designed separately for channel attention or spatial attention. Channel attention often uses fully connected layers, while spatial attention often uses ordinary convolutions. Neither uses a KAN mechanism nor residual connections, resulting in weak focusing ability on tumor features and a tendency to lose original feature information. This embodiment combines KBAM with low-order B-spline KAN to achieve both channel and spatial attention, enabling more accurate adaptation to heterogeneous tumor features. Residual connections preserve original information, solving the problems of inaccurate focus and feature loss in existing technologies. The enhanced feature map provides more effective feature support for subsequent downsampling or skip connections.
[0046] In another embodiment, the spatial KAN processing operation and the KAN layer in KBAM use a shared coefficient pool mechanism; the shared coefficient pool is a predefined set of globally learnable parameters; when processing token sequences in progressively stacked KAN layers, each sublayer allocates a dedicated subset of coefficients from the shared coefficient pool for weighted computation in univariate basis function expansion and atomic multiplication coupling; in KBAM, the KAN layer based on low-order B-spline basis functions also allocates a subset of coefficients from the shared coefficient pool.
[0047] For example, the spatial KAN processing operation and the KAN layer in KBAM use a shared coefficient pool mechanism. The shared coefficient pool is a predefined set of globally learnable parameters stored in the computer's memory. The workstation is placed in the medical image processing room. The total number of parameters can be selected as 10,000 or 15,000. The initial values of the parameters are initialized using a uniform distribution or a normal distribution. The range of the uniform distribution can be selected as [-0.1, 0.1] or [-0.08, 0.08]. The mean of the normal distribution is 0 and the standard deviation can be selected as 0.01 or 0.02.
[0048] When processing token sequences using progressively stacked KAN layers, the progressively stacked KAN layers are the core component in spatial KAN processing operations. They contain 3 or 4 sub-layers. The token sequence is a vector flattened from feature map blocks, with a shape of [B, N, D], where B is the batch size, N is the number of tokens, and D is the token dimension. N can be 256 or 512, and D can be 128 or 256. Each sub-layer allocates a dedicated subset of coefficients from a shared coefficient pool. The allocation range is determined according to the sub-layer order and requirements; for example, the first sub-layer allocates the first 1000 coefficients, and the second layer allocates coefficients from 1001 to 2000. This coefficient subset is used for weighted calculations in univariate basis function expansion and atomic multiplication coupling. During univariate basis function expansion, each dimension d of the sequence (total dimension = token dimension) is processed independently. For each input value t (an element belonging to dimension d), the expanded representation is calculated as follows: , where q is the basis function node; during multiplicative atom coupling computation, another subset of coefficients from the coefficient subset is used as the weights of the coupling terms to compute the product combination of extended representations of different dimensions. For each atom r, the coupling term is In KBAM, the KAN layer based on low-order B-spline basis functions also allocates a subset of coefficients from the shared coefficient pool. KBAM is a KAN-enhanced attention module, containing channel attention KAN layers and spatial attention KAN layers. The order of the low-order B-splines can be 2 or 3, and the number of nodes can be 5 or 7. The number of coefficient subsets allocated to the channel attention KAN layer is related to the number of channels and nodes; for example, when the number of channels is 64 and the number of nodes is 5, 320 coefficients are allocated. The number of coefficient subsets allocated to the spatial attention KAN layer is related to the number of channels and nodes after projection; for example, when the number of channels after projection is 1 and the number of nodes is 5, 5 coefficients are allocated. These coefficients are used for the weighted calculation of the basis functions in the KAN layer. The channel attention KAN layer calculates the channel weight vector through the coefficients, and the spatial attention KAN layer calculates the spatial attention graph through the coefficients. The calculation process can be performed on a server placed in a data center rack. The coefficient updates and optimizations are performed synchronously with the entire model training process, adjusting the coefficient values through backpropagation to minimize the segmentation loss.
[0049] In existing technologies, traditional KAN layers and KAN layers in attention modules each have independent coefficients without a sharing mechanism, resulting in large parameter redundancy, high model training difficulty, and slow convergence. This embodiment reduces the total number of parameters by sharing a coefficient pool, avoiding redundancy, while ensuring the consistency of parameters between spatial KAN processing and the KAN layers in KBAM. This solves the problems of parameter redundancy and low training efficiency in existing technologies, making the model easier to train and able to maintain stable performance.
[0050] In another embodiment, the channel attention mechanism in the post-processing operation is a squeeze-excited channel attention block; the processing of the SE channel attention block includes: performing global average pooling on the input feature map along the spatial dimension to generate channel statistical vectors; inputting the channel statistical vectors into a bottleneck structure consisting of two fully connected layers, wherein the first fully connected layer is used for dimensionality reduction and the second fully connected layer is used to recover the original number of channels, and applying the Sigmoid activation function to generate the weights of each channel; performing channel-by-channel multiplication with the original input feature map to complete the recalibration of the channel features.
[0051] For example, the channel attention mechanism in the post-processing operation is a squeeze-excited channel attention block (SE block). The SE block processing first performs global average pooling on the input feature map along the spatial dimension. The input feature map is a 3D feature map output after spatial KAN processing, with a shape of [B, C, H, W, D], where B is the batch size, C is the number of channels, and H, W, and D are the spatial dimensions. C can be 64 or 128, and H, W, and D can be 64 or 128. Global average pooling calculates the arithmetic mean of all voxel values for each channel, using the following formula: in This is the pooling result for the c-th channel. This is the voxel value at position (i, j, k) of the c-th channel. This operation can be performed on a workstation located in the medical image processing room.
[0052] Next, the obtained channel statistics vector (of shape [B, C]) is input into a bottleneck structure consisting of two fully connected layers. The first fully connected layer is used for dimensionality reduction, decreasing the number of channels from C to C / r, where r is the dimensionality reduction coefficient. r can be 16 or 8. For example, when C=64 and r=16, the number of channels after dimensionality reduction is 4. The number of weight parameters in this fully connected layer is C×(C / r). The initial values of the parameters are normally distributed, with a mean of 0 and a standard deviation that can be 0.01 or 0.02. After calculation, the ReLU activation function is applied to set values less than 0 to 0 and retain values greater than 0. The second fully connected layer is used to restore the original number of channels, restoring the number of channels from C / r to C. The number of weight parameters is (C / r)×C, also initialized using a normal distribution. After calculation, the Sigmoid activation function is applied to map the output values to the range of 0 to 1, generating the weights for each channel.
[0053] Finally, the generated channel weights are multiplied channel-by-channel with the original input feature map, that is, all voxel values of the c-th channel are multiplied by the corresponding weight value of that channel, as shown in the formula: in The value of the voxel after multiplication. The weights for the c-th channel are used to recalibrate the channel features. The multiplication operation is performed in the workstation's memory, and the recalibrated feature map is used for subsequent layer normalization operations.
[0054] In existing technologies, traditional channel attention mechanisms suffer from simple bottleneck structures, fixed dimensionality reduction coefficients, and a lack of refined parameter initialization strategies. The selection of activation functions is also limited, resulting in imprecise channel weight allocation and limited enhancement of important tumor feature channels. This embodiment addresses this by rationally selecting dimensionality reduction coefficients, optimizing parameter initialization, and combining ReLU and Sigmoid activation to achieve more precise channel weights. This effectively enhances tumor-related feature channels, resolving the problem of poor channel recalibration in existing technologies and improving the effectiveness of subsequent feature processing.
[0055] In another embodiment, in S3, the output feature map of the encoder is received, which has reduced spatial resolution and increased number of channels; multiple parallel dilated convolution branches are applied to the received feature map, each using a 3×3×3 convolution kernel with dilation rates of 1, 2, 4, and 8, respectively, to capture multi-scale context of different receptive fields; simultaneously, a global average pooling branch is applied to perform global average pooling on the input feature map, followed by channel compression through 1×1×1 convolution, and restoration of the original spatial resolution through bilinear upsampling; the outputs of all branches are concatenated along the channel dimension to form a fused feature; the concatenated fused feature is dimensionality reduced and integrated by applying 1×1×1 convolution, then residual connections are added and added to the original input feature map, and finally, a dropout operation is applied.
[0056] For example, when receiving the output feature map of the encoder, the output feature map of the encoder is a 3D feature map after downsampling at the last stage of the encoder. This feature map has a reduced spatial resolution and an increased number of channels. The spatial resolution can be selected as 8×8×8 or 16×16×16, and the number of channels can be selected as 512 or 1024. The feature map shape is [B, C_in, H_min, W_min, D_min], where B is the batch size, C_in is the number of channels, and H_min, W_min, and D_min are the minimum values of the spatial dimensions. The feature map is transmitted from the workstation where the encoder is located to the bottleneck layer through a data transmission line.
[0057] Multiple parallel dilated convolutional branches are applied to the received feature maps. The number of parallel dilated convolutional branches can be selected as 4 or 5, and each branch uses a 3×3×3 convolutional kernel. The number of convolutional kernels can be selected as 128 or 256, and the dilation rate is set to 1, 2, 4, 8 or 1, 3, 5, 7, respectively. The dilation rate determines the receptive field size of the convolutional kernel. A dilation rate of 1 results in a smaller receptive field, focusing on local features, while a dilation rate of 8 results in a larger receptive field, capturing global features. The dilation convolutional operation of each branch is executed on the server's GPU, and each branch processes the feature map simultaneously to improve computational efficiency.
[0058] Simultaneously, a global average pooling branch is applied to perform global average pooling on the input feature map, calculating the average value of all voxel values for each channel to obtain a channel statistical vector with shape [B, C_in]. This operation is executed on the server's CPU, which is installed in the CPU socket of the server's motherboard. Next, a 1×1×1 convolution is used to compress the channels, reducing the number of channels from C_in to C_in / 4 or C_in / 8. For example, when C_in=512, the number of channels after compression is 128 or 64. The convolution operation is accelerated using the GPU described above. Then, bilinear upsampling is used to restore the original spatial resolution, that is, to the same spatial resolution as the feature map output by the parallel dilated convolution branch. Bilinear upsampling is achieved by weighted averaging of neighboring pixel values, and the upsampling process is executed in the server's memory.
[0059] The outputs of all branches are concatenated along the channel dimension, merging the outputs of the parallel dilated convolution branches and the global average pooling branch. The number of channels in the concatenated feature map is the sum of the number of channels in each branch. For example, if four parallel dilated convolution branches each output 128 channels and the global average pooling branch each outputs 128 channels, the concatenated feature map will have 640 channels. This concatenation operation is performed in the server's memory. A 1×1×1 convolution is then applied to the concatenated fused features for dimensionality reduction and integration, reducing the number of channels to match the encoder's output feature map (512 or 1024). The number of convolution kernels is 512 or 1024, and the convolution operation is accelerated using the GPU. Residual connections are then added, adding the convolutional feature map to the original input feature map voxel-by-voxel. This addition operation is performed on the CPU. Finally, a dropout operation is applied, with a dropout probability of 0.1 or 0.2, randomly setting some feature values to 0 to prevent overfitting. This dropout operation is performed in memory, resulting in the output feature map of the bottleneck layer.
[0060] In existing technologies, traditional bottleneck layers often employ a single convolutional branch or a small number of branches, lacking multi-rate dilation design, resulting in incomplete global context capture and the absence of dropout operation, making them prone to overfitting. This embodiment captures multi-scale context through multiple parallel dilated convolutional branches, supplements global information with global pooling branches, and prevents overfitting with dropout, thus solving the problems of incomplete context capture and overfitting in existing technologies and providing richer semantic features for the decoder.
[0061] In another embodiment, in S4, the output feature map of the bottleneck layer is received as the input to the decoder; for each level of the decoder, a transposed convolutional upsampling operation is performed to increase the spatial size of the feature map and reduce the number of channels; cross-scale gating processing is performed, fusing the skip connection features from the corresponding level of the encoder and the currently upsampled features, wherein the cross-scale gating processing includes processing the skip connection features and the upsampled features using two parallel paths respectively, each path applying a depthwise separable convolution, activation function, and KAN gating mechanism, then performing multiplication gating, and concatenating the outputs of the two paths along the channel dimension and fusing them through a 1×1×1 convolution; spatial KAN processing is performed on the output of the cross-scale gating processing, including dividing the feature map into 4×4×4 non-overlapping patches and flattening them into a token sequence, dynamically adjusting the B-spline basis function node positions based on modality statistics vectors through a meta-adaptation layer, accelerating basis function evaluation using a pre-computed B-spline table, applying progressively stacked KAN layers to process the token sequence, and performing post-processing operations; the output of the spatial KAN processing is used as the input to the next decoder level or the final output of the decoder.
[0062] For example, the output feature map of the bottleneck layer is received as the input of the decoder. The output feature map of the bottleneck layer is a 3D feature map after multi-branch fusion, dimensionality reduction, residual connection and dropout, with a shape of [B, C, H_min, W_min, D_min], where B is the batch size, C is the number of channels, and H_min, W_min and D_min are the minimum values of spatial dimensions. C can be 512 or 1024, and H_min, W_min and D_min can be 8×8×8 or 16×16×16. The feature map is transmitted from the server where the bottleneck layer is located to the computing node or training server where the decoder is located via a data cable.
[0063] For each level of the decoder, the number of levels is the same as that of the encoder, either 5 or 6. First, a transposed convolution upsampling operation is performed. The kernel size of the transposed convolution can be selected as 2×2×2 or 3×3×3, and the number of kernels is half of the current number of channels. For example, when the current number of channels is 512, the number of kernels is 256. The stride can be selected as 2 or 3. The spatial size of the feature map is expanded by transposing the convolution, making the spatial dimension 2 or 3 times the original, while reducing the number of channels. The upsampling operation is performed on the GPU of the computing node.
[0064] Next, cross-scale gating processing is performed, fusing skip connection features from the corresponding encoder level with the currently upsampled features. The skip connection features are feature maps stored after KBAM enhancement of the corresponding encoder level, and their shapes are similar to the currently upsampled features. Figure 1The feature map is [B, C / 2, 2H_min, 2W_min, 2D_min] and is read from the storage disk. Cross-scale gating processing involves two parallel paths to process skip connection features and upsampled features respectively. The first path processes skip connection features by applying a 3×3×3 depthwise separable convolution. This convolution first performs a depthwise convolution to extract spatial features, then performs a 1×1×1 pointwise convolution to adjust the number of channels, and then applies the GeLU activation function. The GeLU activation function is calculated using the formula x×Φ(x), where Φ(x) is the cumulative distribution function of the standard normal distribution. Finally, the weights are calculated using a KAN gating mechanism. KAN gating is based on a low-order B-spline KAN, with the order selectable as 2 or 3, and the number of nodes selectable as 5 or 7. After generating the gating weights, they are multiplied and gated with the skip connection features. The second path processes upsampled features, with similar steps to the first path, except that the activation function used is ReLU. It also performs depthwise separable convolution, activation, and KAN gating multiplication. The outputs of the two paths are concatenated along the channel dimension, resulting in C channels. Then, a 1×1×1 convolution is used to reduce the channel number to C / 2. The fusion operation is performed in the memory of the computing node.
[0065] Then, spatial KAN processing is performed on the output of the cross-scale gating process. First, the feature map is divided into 4×4×4 or 6×6×6 non-overlapping patches, and then flattened into a token sequence with the shape [B, N, D], where N is the number of tokens and D is the token dimension. Next, based on the modality statistics vector, the B-spline basis function node positions are dynamically adjusted through a meta-adaptive layer. The meta-adaptive layer is a small multilayer perceptron, with the modality statistics vector as input and the node offset as output, which can be selected from -0.1 to 0.1 or -0.12 to 0.12. Then, a pre-computed B-spline table is used to accelerate the evaluation of the basis functions. The pre-computed table is stored in memory, and the basis function values are obtained through lookup and linear interpolation. Subsequently, progressively stacked KAN layers are applied to process the token sequence. The number of KAN layers is 3 or 4, and the number of atoms gradually increases, such as 4-8-16 or 6-10-14, with residual connections added to each layer. Finally, post-processing operations are performed, including depthwise separable convolution, channel attention mechanism, and layer normalization. The depthwise separable convolution is consistent with that in cross-scale gating processing. The channel attention mechanism is an SE block. Layer normalization is achieved by calculating the mean and variance normalized features within the layer.
[0066] The output of spatial KAN processing is used as the input of the next decoder level or the final output of the decoder. If the current level is not the last level of the decoder, it is passed to the next level for transposed convolution upsampling; if it is the last level, it is used as the final output of the decoder and passed to the output module.
[0067] In existing technologies, traditional decoders perform cross-scale fusion by simply splicing or weighted summing without a KAN gating mechanism. Spatial feature processing lacks fine-grained segmentation and dynamic basis function adjustment, resulting in low-quality fused features and loss of tumor details. This embodiment solves the problems of poor fusion effect and loss of details in existing technologies by accurately fusing cross-scale features through KAN gating and dynamically adapting spatial KAN processing to tumor features, thereby restoring more accurate tumor segmentation features.
[0068] In another embodiment, in S5, the output feature map of the decoder is received; a 1×1×1 convolution operation is applied to the feature map to reduce the number of channels to the number of classes, generating a logits tensor; a softmax activation function is applied to the logits tensor to generate a multi-class probability segmentation mask along the channel dimension.
[0069] For example, the output feature map of the decoder is received. The output feature map of the decoder is a 3D feature map after the last stage of spatial KAN processing of the decoder. This feature map has been restored to the same spatial resolution as the original multimodal 3D brain MRI data. The spatial resolution can be selected as 128×128×128 or 256×256×256, and the number of channels can be selected as 32 or 64. The shape of the feature map is [B, C_in, H, W, D], where B is the batch size, C_in is the number of channels, and H, W, and D are the spatial dimensions. The feature map is transmitted from the computing node where the decoder is located to the workstation where the output module is located via a data cable. The workstation is placed on the doctor's desk.
[0070] A 1×1×1 convolution operation is applied to the feature map. The number of 1×1×1 convolution kernels is equal to the number of categories in the segmentation task. In brain tumor segmentation, the number of categories can be 4 or 5. The 4 categories are background, whole tumor, tumor core, and enhanced tumor. The 5 categories can add edema region categories. The initial values of the weight parameters of the convolution kernel adopt a normal distribution with a mean of 0 and a standard deviation of 0.01 or 0.02. The convolution operation is performed by sliding on the feature map to convert the multi-channel features of each voxel into the corresponding category's logits value. There is no limit to the range of logits values. The convolution operation is executed on the CPU of the mobile workstation. After convolution, a logits tensor is generated with the shape [B, num_classes, H, W, D], where num_classes is the number of categories.
[0071] Applying the softmax activation function to the logits tensor, the softmax activation function calculates the probability value of each voxel belonging to different categories along the channel dimension, as shown in the formula: in Let be the probability value of the c-th category. Let be the logits value of the c-th category, and ∑ be the sum of probabilities over all categories, with probability values ranging from 0 to 1, and the sum of probabilities for all categories being 1. The steepness of the probability distribution can be adjusted by setting a temperature parameter; the temperature parameter can be 1.0 or 0.8. A temperature parameter less than 1 results in a steeper probability distribution, while a temperature parameter greater than 1 results in a smoother distribution. The softmax calculation is performed in the mobile workstation's memory, which can be either Samsung LPDDR5X-7467 or Hynix LPDDR5X-7467 memory, installed in the memory slots on the mobile workstation's motherboard. Finally, a multi-class probability segmentation mask is generated, with the mask shape consistent with the logits tensor. This mask can be displayed on a monitor connected to the mobile workstation, such as a Dell UltraSharp U3223QE 32-inch 4K monitor or an ASUS ProArt Display PA32UCX-P 32-inch 4K monitor, connected via an HDMI 2.1 cable.
[0072] In existing technologies, the 1×1×1 convolution parameters of traditional output modules are initialized arbitrarily, and the softmax function lacks temperature parameter adjustment, resulting in low class discrimination of the probability mask and inaccurate probability prediction of tumor edge regions. This embodiment optimizes the initialization of convolution parameters and introduces a temperature parameter, making the probability mask class discrimination clearer and the edge prediction more accurate. This solves the problem of low mask quality in existing technologies and provides more reliable segmentation results for clinical diagnosis.
[0073] Embodiments of this application also provide a multi-scale brain tumor segmentation system based on adaptive KAN, comprising: a preprocessing module for preprocessing multimodal three-dimensional brain magnetic resonance imaging data, including Z-score normalization for each modality channel, application of random data augmentation, and calculation of modality statistical vectors, wherein the modality statistical vectors include the mean, variance, and skewness of each modality; and an encoder module for receiving the preprocessed data, the encoder module comprising multiple levels, each level configured to perform convolution operations to extract local features, perform spatial KAN processing to extract spatial features, and downsample the feature map, wherein the spatial KAN processing dynamically adjusts the node positions of the B-spline basis function based on the modality statistical vectors and uses pre-calculated... The B-spline table accelerates basis function evaluation; the bottleneck layer module receives the encoder output and uses multiple parallel dilated convolution branches to capture multi-scale global context; the decoder module receives the bottleneck layer output and includes multiple levels, each configured to perform transposed convolution to oversample the feature map, perform cross-scale gating to fuse skip connection features from the encoder module with the current decoder features, and perform spatial KAN processing to optimize features, where the cross-scale gating uses the KAN gating mechanism; the output module receives the decoder output and uses 1×1×1 convolution to reduce the feature map dimensionality to the number of classes and generates a multi-class probability segmentation mask through a softmax activation function.
[0074] For example, a preprocessing module is used to preprocess multimodal three-dimensional brain magnetic resonance imaging data. This module can be deployed on a workstation located in a medical imaging processing room. The module includes a Z-score normalization submodule, a random data augmentation submodule, and a modal statistical vector calculation submodule. The Z-score normalization submodule processes each modal channel by first calculating the mean μ and standard deviation σ of all voxel values within the channel, and then normalizing them using the formula (voxel value - μ) / (σ + small constant). The small constant can be 1e-5 or 1e-6. The calculation of the submodule can be performed using the workstation's CPU, which can be an Intel Xeon Gold 6430 processor or an AMD EPYC 9654 processor, installed in the CPU socket of the workstation's motherboard. The random data augmentation submodule applies random transformations, which can be selected to flip along the x-axis, y-axis, or z-axis with a flip probability of 0.5 or 0.6, a random rotation angle of ±15 degrees or ±20 degrees, and an intensity adjustment factor of -0.1 to 0.1 or -0.15 to 0.15. The modal statistics vector calculation submodule is used to calculate the mean, variance, and skewness of each modality. The mean is the arithmetic mean of the voxel values, the variance is the average of the squared deviations from the mean, and the skewness is the average of the cubed deviations from the mean divided by the variance to the power of 1.5. The statistical vectors output by the submodule are stored on an enterprise-grade hard drive.
[0075] The encoder module receives preprocessed data and is deployed on the same workstation as the preprocessing module. It contains multiple levels, selectable as 5 or 6 levels. Each level includes a convolution operation submodule, a spatial KAN processing submodule, and a downsampling submodule. The convolution operation submodule uses 3×3×3 or 5×5×5 convolution kernels to extract local features, followed by batch normalization and ReLU activation. It also includes residual connections and is accelerated using a GPU. The spatial KAN processing submodule is based on B-spline KAN and dynamically adjusts the B-spline node positions through a meta-adaptation layer (small multilayer perceptron, 32 or 64 hidden layer dimensions). The offset ranges from -0.1 to 0.1 or -0.12 to 0.12. It uses a pre-computed B-spline table (1000 or 2000 sampling points) to accelerate evaluation. Shallow levels use Gaussian radial basis functions (center 0.2 or 0.3, sigma 0.1 or 0.15), while deeper levels use B-splines. The submodule parameters are stored in memory. The downsampling submodule uses a 2×2×2 or 3×3×3 max pooling kernel with a stride of 2 or 3 to reduce the spatial size and increase the number of channels. The submodule is executed on the CPU.
[0076] The bottleneck layer module is used to receive the encoder output and is deployed on a server (in a server rack in the data center). It uses multiple parallel dilated convolution branches, with a number of 4 or 5, 3×3×3 convolution kernels, and dilation rates of 1, 2, 4, 8 or 1, 3, 5, 7. It also includes a global average pooling branch (1×1×1 convolution dimensionality reduction, bilinear upsampling). The outputs of all branches are concatenated and then subjected to 1×1×1 convolution dimensionality reduction, residual connection, and dropout (probability 0.1 or 0.2).
[0077] The decoder module receives the output from the bottleneck layer and is deployed on compute nodes (in server racks). Its number of stages is the same as the encoder, with each stage configured with a transposed convolutional upsampling submodule, a cross-scale gating processing submodule, and a spatial KAN processing submodule. The transposed convolutional upsampling submodule uses 2×2×2 or 3×3×3 convolutional kernels with a stride of 2 or 3 to expand the spatial size; this submodule is accelerated using a GPU. The cross-scale gating processing submodule contains two parallel paths, processing the encoder's skip connection features (GeLU activation) and upsampling features (ReLU activation) respectively. Both are processed by depthwise separable convolutions and KAN gating (order 2 or 3, 5 or 7 nodes), and the outputs are concatenated and fused using a 1×1×1 convolution. This submodule executes in memory. The spatial KAN processing submodule is identical to that in the encoder, with its parameters allocated from a shared coefficient pool (parameters 10000 or 15000).
[0078] The output module receives the decoder's output and is deployed on a mobile workstation (doctor's desk). It contains a 1×1×1 convolutional submodule and a softmax activation submodule. The 1×1×1 convolutional submodule reduces the number of channels to the number of classes (4 or 5), with 4 or 5 convolutional kernels. This submodule is executed by the CPU. The softmax activation submodule calculates probabilities along the channel dimension, with a temperature parameter of 1.0 or 0.8, generating a multi-class probability segmentation mask. The mask can be displayed on a monitor connected to the mobile workstation.
[0079] In existing technologies, traditional brain tumor segmentation systems employ independently designed modules, resulting in simple preprocessing, encoders lacking spatial KAN (Knowledge Angle Transformation), bottleneck layers with few branches, decoders without KAN gating, and coarse output module parameters. This leads to low overall system accuracy and poor efficiency. The system in this embodiment features collaborative optimization across modules, refined preprocessing, encoders adapted to tumor heterogeneity, bottleneck layers capturing global information, decoders accurately fusing features, and output modules generating high-quality masks. This addresses the shortcomings of existing systems, enabling stable and efficient brain tumor segmentation.
[0080] Embodiments of this application also provide a computer-readable storage medium storing a computer program. When executed by a processor, the computer program enables the processor to implement the described multi-scale brain tumor segmentation method based on adaptive KAN. The processor may be a microcontroller unit, a central processing unit, or a digital signal processor, etc. The storage medium may be a hard disk or internal / external storage device, such as an external hard disk, smart memory card, secure digital card, flash memory card, etc. The computer program may be written using the Python programming language and developed based on the PyTorch or TensorFlow deep learning framework. The program code includes implementation code for modules such as data preprocessing, encoder, bottleneck layer, decoder, and output module, implementing the multi-scale brain tumor segmentation method based on adaptive KAN.
[0081] The following is a description of a specific embodiment.
[0082] Example 1: Brain tumor segmentation methods include: S1: preprocessing of multimodal three-dimensional brain MRI data. 1. Load raw data: Read the multimodal 3D MRI data of the target sample (including FLAIR, T1, T1ce, and T2 modalities) from the storage hard drive associated with the hospital's PACS system, convert it into a float32 format tensor with the shape [B, 4, 128, 128, 128] (B is the batch size, 4 is the number of modalities, and 128×128×128 is the spatial dimension); check the modal integrity, and if there are missing modalities, fill the missing voxel values using nearest neighbor interpolation.
[0083] 2. Modal channel Z-score standardization: Calculate the global mean μ and standard deviation σ for each modal channel separately, and adjust the voxel values through standardization operation. Add a small constant of 1e-5 during the operation to avoid division by zero. If processing multiple samples, batch overall statistics are used to ensure consistency. The standardized data is output with the shape still [B,4,128,128,128].
[0084] 3. Random Data Augmentation: Flip the data along one of the x, y, or z axes with a probability of 0.5; generate random rotation angles within the range of ±15 degrees, calculate the corresponding transformation matrix, and map the original voxels to the new spatial positions through grid sampling; generate random intensity factors within the range of -0.1 to 0.1, and multiply the normalized data by (1 + the intensity factor) to simulate brightness fluctuations.
[0085] 4. Calculate the modal statistical vector: For each modal channel, calculate the global mean (arithmetic mean of all voxels), variance (square mean of voxel values deviating from the mean), and skewness (cubic mean of voxel values deviating from the mean divided by the variance to the power of 1.5). Aggregate the three statistics of each modality into a vector to form a modal statistical vector of shape [B,4,3] (3 is the statistical dimension).
[0086] 5. Output preprocessing results: Input the standardized data and modal statistical vectors into the encoder module of S2.
[0087] S2: Encoder extracts multi-scale features 1. Initial Feature Extraction: Receive the standardized data and modality statistics vector output by S1, apply a 3×3×3 3D convolutional kernel (32 channels), followed by batch normalization (calculate the batch channel mean and variance, adjust the feature distribution through a preset formula, where γ and β are learnable parameters) and ReLU activation (retain the original value when the voxel value is greater than 0, otherwise set to 0), repeat twice to form a double convolutional block, and output a shallow initial feature map.
[0088] 2. Five-level encoding processing (step-by-step downsampling): Level 1 encoding: Apply convolutional blocks (3×3×3 convolution + batch normalization + ReLU, including residual connections) to the initial feature map to extract local features; apply the KBAM module to first perform global average pooling on the feature map to generate channel statistical vectors, input the vector into a KAN layer (5 nodes) based on 2nd order B-splines to generate channel weights, and multiply it with the feature map channel by channel; then apply 1×1×1 convolutional projection to the channel-enhanced feature map, input the projected feature map into another 2nd order B-spline KAN layer (5 nodes) to generate spatial weights, multiply it with the channel-enhanced feature map voxel by voxel, and finally add it to the original feature map through residual connections.
[0089] SKB module processing: The feature map output by KBAM is divided into 4×4×4 non-overlapping blocks, flattened into a token sequence, and a coordinate-based sinusoidal position embedding is added; the modal statistics vector is input into the meta-adaptation layer (small MLP, hidden dimension 32) to calculate the B-spline node offset (range -0.1 to 0.1), dynamically adjusting the node position; since the current level is shallow (level 1), a Gaussian radial basis function is selected (center point 0.2, σ 0.1); three progressively stacked KAN layers (atoms 4, 8, and 16 respectively) are used to process the token sequence, each... Each layer allocates a subset of dedicated coefficients from the shared coefficient pool (total coefficients are 10000), performs univariate basis function expansion, atomic multiplication coupling, and weighted aggregation operations, and adds residual connections to each layer; followed by depthwise separable convolutions (first 1×1×1 channel convolutions, then 3×3×3 depth convolutions), SE channel attention blocks (first global pooling to compress channel statistics, then adjusting channel weights through two levels of 1×1 convolutions, and finally applying Sigmoid activation to generate weights and multiplying them with the feature map channel by channel), and layer normalization, reorganizing the processed token sequence into a spatially optimized feature map.
[0090] Downsampling: Apply 2×2×2 max pooling (stride of 2) to the spatial optimization feature map output by SKB, halving the spatial dimension to 64×64×64 and doubling the number of channels to 64, outputting the first-level encoded feature map; at the same time, store this feature map as a skip connection for use by the S4 decoder.
[0091] Encoding levels 2-5: Repeat the above convolutional block-KBAM-SKB-downsampling process, where levels 3-5 are deep levels, the SKB module is switched to B-spline basis function (call the pre-computed B-spline table, the sampling points are 1000, and the value is obtained by linear interpolation); the number of channels doubles at each level (64-128-256-512), and the spatial dimension is halved at each level (64-32-16-8-4).
[0092] 3. Output encoding result: Input the low-resolution feature map (shape [B,512,4,4,4]) output by the 5th level encoding into the bottleneck layer module of S3.
[0093] S3: Bottleneck layer captures global context 1. Receive coding features: Obtain the 5th level coding feature map (low spatial resolution, high number of channels) output by S2.
[0094] 2. Multi-branch feature processing: Parallel dilated convolution branches: Four parallel 3D dilated convolution branches are set up, each using a 3×3×3 convolution kernel (128 channels), with dilation rates of 1, 2, 4 and 8 respectively. By using different dilation rates, multi-scale context from local to global is captured, and feature maps of the same resolution (4×4×4) are output.
[0095] Global pooling branch: Global average pooling is performed on the level 5 encoded feature map to generate channel statistical vectors. A 1×1×1 convolution (128 channels) is applied to compress the dimension, and then bilinear upsampling is used to restore the spatial resolution to 4×4×4, which is consistent with the output resolution of the dilated convolution branch.
[0096] 3. Feature fusion and optimization: The outputs of the four dilated convolution branches and one global pooling branch are concatenated along the channel dimension and integrated by 1×1×1 convolution (512 channels); residual connections are added to add the fused features to the original Level 5 encoded feature map voxel by voxel; dropout operation (probability of 0.1) is applied to prevent overfitting.
[0097] 4. Output bottleneck layer results: Input the fused and optimized feature map (shape [B,512,4,4,4]) into the decoder module of S4.
[0098] S4: Decoder restores resolution and blending features 1. Receive bottleneck layer features: Obtain the fused feature map output by S3 and use it as the input of the first stage of the decoder.
[0099] 2. Five-level decoding processing (step-by-step upsampling, symmetrical to the encoder): Level 1 Decoding: Apply a 2×2×2 transposed convolution (256 channels) to the bottleneck layer feature map, with a stride of 2, upsample to an 8×8×8 spatial dimension, and output the upsampled feature map.
[0100] K-CSGM cross-scale fusion: Read the skip connection feature map (shape [B,512,4,4,4]) encoded at level 5 in S2, and upsample it to 8×8×8. Set up two parallel paths. The encoding path applies depthwise separable convolution, GeLU activation, and 2nd-order B-spline KAN gating to the skip connection feature map (after generating weights, it modulates the feature map by multiplying it voxels one by one). The decoding path applies depthwise separable convolution, ReLU activation, and 2nd-order B-spline KAN gating to the upsampled feature map (after generating weights, it modulates the feature map by multiplying it voxels one by one). Concatenate the modulated features of the two paths along the channel dimension, apply 1×1×1 convolution (256 channels) to reduce the dimensionality, and output the fused feature map.
[0101] SKB module processing: The process is the same as SKB in S2 (4×4×4 block division - tokenization - meta adjustment of B spline nodes - B spline basis function evaluation - 3-layer progressively stacked KAN (atoms 16-8-4) - depthwise separable convolution + SE block + layer normalization - recombination), outputting an optimized feature map.
[0102] Decoding at levels 2-5: Repeat the transposed convolution upsampling-K-CSGM fusion-SKB optimization process described above, where levels 3-5 are shallow levels and the SKB module is switched to Gaussian radial basis function; the number of channels is halved at each level (256-128-64-32), and the spatial dimension is doubled at each level (8-16-32-64-128), finally restoring the original spatial resolution of 128×128×128.
[0103] 3. Output decoding result: The high-resolution feature map (shape [B,32,128,128,128]) output from the 5th level decoding is passed to the output module of S5.
[0104] S5: Output module generates segmentation mask 1. Receive decoding features: Obtain the high-resolution feature map output by S4.
[0105] 2. Category Channel Dimensionality Reduction: Apply 1×1×1 3D convolution kernels (4 in number, corresponding to four categories: background, whole tumor (WT), tumor core (TC), and enhanced tumor (ET)) to reduce the number of feature map channels to 4, generating a logits tensor (shape [B,4,128,128,128]).
[0106] 3. Probabilistic activation: Apply the softmax activation function to the logits tensor along the channel dimension so that the sum of the four probabilities of each voxel is 1, generating a multi-class probabilistic segmentation mask.
[0107] 4. Post-inference processing (optional): Apply the argmax operation along the channel dimension to the probability segmentation mask, take the class with the highest probability for each voxel as the final class, and generate a binarized segmentation mask.
[0108] 5. Output the final result: Output a multi-class probability segmentation mask or a binarized segmentation mask to complete the brain tumor segmentation.
[0109] Taking BraTS2018 glioma subregion segmentation as an example, we selected 18 representative existing technology models, including classic basic models such as U-Net++ (2018), 3D U-Net (2016), and V-Net (2016), Transformer / CNN fusion models such as SwinUNETR (2021), UNetR (2021), and TransBTS (2021), cutting-edge Mamba-like models such as SwinUMamba (2024), UMamba (2024), and MambaClinix (2024), as well as benchmark or improved models in the field such as nnU-Net (2020), nnU-NetV2 (2024), and SegMamba (2024), for performance comparison with Example 1 of this application.
[0110] The comparison revolves around three core metrics: Dice coefficient (measures the degree of overlap between the segment and the ground truth annotation, including whole tumor, tumor core, enhanced tumor, and average Dice, reflecting multi-region segmentation capability); 95% Hausdorff distance (HD95, measures the maximum difference between the segment boundary and the ground truth annotation, including HD95WT, HD95TC, HD95ET, and average HD95, reflecting boundary refinement); and Intersection over Union (IoU, measures the ratio of the intersection of the segment and the ground truth annotation to the union, reflecting the effectiveness of the overlap of effective regions).
[0111] Based on the comparison results (see Table 1), this application demonstrates outstanding performance in multi-region segmentation accuracy, boundary refinement, and regional overlap effectiveness: the overall tumor Dice coefficient reaches 0.9181, which is superior to models such as U-Net++ (0.8915) and SwinUNETR (0.9033), and slightly better than the benchmark model nnU-NetV2 (0.9168), leading in overall tumor segmentation accuracy; the tumor core Dice coefficient is 0.8747, far exceeding V-Net (0.7387) and 3D... Early models such as U-Net (0.7248) are on par with cutting-edge Mamba-type models such as SwinUMamba (0.8731) and UMamba (0.8674), achieving a high level of fine segmentation of the tumor core; the enhanced tumor Dice coefficient is 0.8115, which is better than classic models such as U-Net++ (0.8117) and TransBTS (0.8093), ensuring effective segmentation of key tumor regions; the average Dice coefficient is 0.8681, which is higher than classic models such as nnU-Net (0.8621) and U-Net++ (0.8503), and close to models such as nnU-NetV2 (0.8719) and SegMamba (0.8742).
[0112] In terms of boundary refinement, the HD95 for the whole tumor is 5.0213, which is one of the lowest values among all the comparison models (such as TransBTSV2 at 5.3442 and SegFormer at 5.1514), indicating that the difference between the whole tumor segmentation boundary and the ground truth annotation is minimal. The HD95 for the tumor core is 6.8647, which is lower than early classic models such as nnU-Net (8.6420) and U-Net++ (8.9219), and only slightly higher than a few cutting-edge models such as nnU-NetV2 (6.9431), indicating that the consistency and refinement of the tumor core boundary are top-notch. The HD95 for the enhanced tumor is 4.0791, which is in the upper-middle range and is very close to the classic benchmark model nnU-Net (3.9632), indicating that the enhanced tumor boundary segmentation is stable and accurate. The average HD95 is 5.3217, which is one of the lowest values among all the comparison models, indicating that the overall difference in boundary segmentation of the whole tumor, tumor core, and enhanced tumor is minimal, and the boundary refinement control capability is significantly better than most existing technologies.
[0113] In terms of Intersection over Union (IoU), this application scored 0.7669, which is better than classic basic models such as nnU-Net (0.7576) and U-Net++ (0.7395), and close to the most advanced models in recent years such as SwinUNETR (0.7609) and nnU-NetV2 (0.7728). The segmentation results have a high degree of overlap with the effective regions of the ground truth, and the effective segmentation is highly practical.
[0114] In existing technologies, classic basic models suffer from insufficient segmentation accuracy and boundary refinement due to a lack of efficient modeling for multi-scale and long-range dependencies. While Transformer / CNN fusion models can capture long-range information, they suffer from computational redundancy and loss of features in small regions. Although cutting-edge Mamba-like models utilize selective state-space models to improve efficiency, there is still room for improvement in the accuracy and boundary consistency of multi-region collaborative segmentation. This application, through an innovative architecture of adaptive KAN + multi-scale feature fusion, accurately aggregates modal information in the multi-modal data preprocessing stage, providing high-quality input for subsequent segmentation. In the encoder and decoder stages, adaptive KAN and cross-scale fusion simultaneously address the issues of insufficient segmentation accuracy in large regions and missed detections and rough boundaries in small regions. In the bottleneck layer and output module stages, global context capture and fine segmentation mechanisms ensure boundary consistency and effective region overlap. These innovations bring the beneficial effects of more accurate multi-region segmentation, finer boundaries, and higher effective region overlap, enabling the core indicators of this application to reach or exceed those of existing technologies.
[0115] To verify the independent contribution of the core modules, an ablation experiment was designed: using a basic model without the core module as a benchmark, adding only the adaptive KAN module significantly improved the Dice coefficient of tumor core region segmentation, effectively adapting to the heterogeneous characteristics of tumors; after adding the cross-scale fusion module, the missed detection rate of small-volume edema regions was reduced, the boundary fineness was improved, and the problem of feature loss in small regions was solved; further, the bottleneck layer global context capture module was introduced, which enhanced the consistency of the whole tumor segmentation boundary and reduced the HD95 distance; after the integration of all modules, all core indicators reached the optimal level, proving that adaptive KAN, cross-scale fusion, and global context capture all have independent gains, and their synergistic effect further enhances the segmentation performance, confirming the indispensable technical value of each module.
[0116] Table 1. Comparison of indicators between this application and prior art. Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and embodiments shown and described herein.
Claims
1. A multi-scale brain tumor segmentation method based on adaptive KAN, characterized in that, include: S1: Preprocessing of multimodal three-dimensional brain magnetic resonance imaging data, including Z-score normalization for each modal channel, application of random data augmentation, and calculation of modal statistical vectors, which include the mean, variance, and skewness of each modality; S2: Input the preprocessed data into the encoder. The encoder includes multiple levels. Each level performs convolution operations to extract local features, performs spatial KAN processing to extract spatial features, and downsamples feature maps. The spatial KAN processing dynamically adjusts the node positions of the B-spline basis functions based on modality statistics vectors and uses a pre-computed B-spline table to accelerate the evaluation of basis functions. S3: Input the encoder's output into the bottleneck layer, which uses multiple parallel dilated convolutional branches to capture multi-scale global context; S4: Input the output of the bottleneck layer into the decoder. The decoder consists of multiple levels. Each level performs transposed convolution upsampling of feature maps, performs cross-scale gating to fuse encoder skip connection features and decoder features, and performs spatial KAN processing to optimize features. The cross-scale gating uses the KAN gating mechanism. S5: Input the decoder's output to the output module. The output module uses a 1×1×1 convolution to reduce the dimensionality of the feature map to the number of classes, and generates a multi-class probability segmentation mask through softmax activation.
2. The multi-scale brain tumor segmentation method based on adaptive KAN as described in claim 1, characterized in that, In S2, the preprocessed data and modality statistics vector are received, and the initial convolution operation is applied to extract shallow features to obtain the initial feature map. The initial feature map is processed by multi-level encoding. Each level sequentially applies convolutional block operation to extract local features, applies spatial KAN processing operation to optimize spatial features, and applies downsampling operation to reduce the spatial size of the feature map and increase the number of channels. In the spatial KAN processing operation, the feature map is divided into non-overlapping patches and flattened into token sequences. The node offset of the B-spline basis function is calculated based on the modal statistics vector through the meta-adaptive layer, and the node position is dynamically adjusted. The evaluation of basis functions is accelerated by using a pre-computed B-spline table for lookup and linear interpolation, and a mixture of basis functions is selected based on the current level depth, with Gaussian radial basis functions used for shallow levels and B-spline basis functions used for deep levels. The token sequence is processed using progressively stacked KAN layers, with the number of atoms increasing gradually. This is combined with residual connections and post-processing operations, including depthwise separable convolution, channel attention mechanism, and layer normalization.
3. The multi-scale brain tumor segmentation method based on adaptive KAN as described in claim 2, characterized in that, In at least one stage of the encoder, after the convolutional block operation and / or spatial KAN processing operation, the feature map is processed by the KAN Enhanced Attention Module (KBAM). The KBAM processing procedure includes: Channel attention calculation: Global average pooling is performed on the input feature map to generate a channel statistical vector; the channel statistical vector is input into a KAN layer based on a low-order B-spline basis function to generate a channel attention weight vector; the channel attention weight vector is used to perform multiplicative modulation on the input feature map in the channel dimension to obtain a channel-enhanced feature map; Spatial attention computation: Project the channel-enhanced feature map using a 1×1×1 convolution; input the projected feature map into another KAN layer based on a low-order B-spline basis function to generate a spatial attention map; Fusion and Output: The spatial attention map and the channel-enhanced feature map are multiplied and modulated in the spatial dimension, and the modulation result is added to the original input feature map through residual connection to output the feature map enhanced by KBAM. The feature maps enhanced by KBAM are used for subsequent downsampling operations at the current encoder level or stored as skip connections for use by the decoder.
4. The multi-scale brain tumor segmentation method based on adaptive KAN as described in claim 3, characterized in that, Spatial KAN processing operations and the KAN layer in KBAM use a shared coefficient pool mechanism; The shared coefficient pool is a predefined set of globally learnable parameters; When processing token sequences in progressively stacked KAN layers, each sublayer allocates a dedicated subset of coefficients from a shared coefficient pool for weighted computation in univariate basis function expansion and atomic multiplication coupling. In KBAM, the KAN layer based on low-order B-spline basis functions also allocates a subset of coefficients from the shared coefficient pool.
5. The multi-scale brain tumor segmentation method based on adaptive KAN as described in claim 3, characterized in that, The channel attention mechanism in post-processing is a squeeze-excite channel attention block; The processing of SE channel attention blocks includes: performing global average pooling on the input feature map along the spatial dimension to generate channel statistical vectors; The channel statistical vector is input into a bottleneck structure consisting of two fully connected layers. The first fully connected layer is used for dimensionality reduction, and the second fully connected layer is used to recover the original number of channels. The Sigmoid activation function is applied to generate the weights of each channel. The generated channel weights are then multiplied one channel at a time with the original input feature map to complete the recalibration of the channel features.
6. The multi-scale brain tumor segmentation method based on adaptive KAN as described in claim 1, characterized in that, In S3, the output feature map of the encoder is received, which has reduced spatial resolution and increased number of channels; Multiple parallel dilated convolutional branches are applied to the received feature maps, each using a 3×3×3 convolutional kernel with dilation rates of 1, 2, 4 and 8, respectively, to capture multi-scale context of different receptive fields. Simultaneously, a global average pooling branch is applied to perform global average pooling on the input feature map, followed by channel compression through 1×1×1 convolution, and restoration of the original spatial resolution through bilinear upsampling. The outputs of all branches are concatenated along the channel dimension to form a fused feature; The concatenated fused features are reduced in dimensionality and integrated by applying a 1×1×1 convolution, then residual connections are added and added to the original input feature map, and finally dropout is applied.
7. The multi-scale brain tumor segmentation method based on adaptive KAN as described in claim 1, characterized in that, In S4, the output feature map of the bottleneck layer is received as the input to the decoder; For each stage of the decoder, a transposed convolutional upsampling operation is performed to increase the feature map spatial size and reduce the number of channels; Perform cross-scale gating processing, fusing skip connection features from the corresponding level of the encoder with the currently upsampled features. The cross-scale gating processing includes using two parallel paths to process skip connection features and upsampled features respectively. Each path applies depthwise separable convolution, activation function and KAN gating mechanism, then performs multiplication gating, and concatenates the outputs of the two paths along the channel dimension and then fuses them through 1×1×1 convolution. Spatial KAN processing is performed on the output of cross-scale gating, including dividing the feature map into 4×4×4 non-overlapping patches and flattening them into token sequences, dynamically adjusting the B-spline basis function node positions based on modality statistics vectors through a meta-adaptation layer, accelerating basis function evaluation using a pre-computed B-spline table, applying progressively stacked KAN layers to process the token sequences, and performing post-processing operations. The output of spatial KAN processing is used as the input to the next decoder level or as the final output of the decoder.
8. The multi-scale brain tumor segmentation method based on adaptive KAN as described in claim 1, characterized in that, In S5, the output feature map of the decoder is received; a 1×1×1 convolution operation is applied to the feature map to reduce the number of channels to the number of classes, generating a logits tensor; Apply the softmax activation function to the logits tensor to generate multi-class probability segmentation masks along the channel dimension.
9. A multi-scale brain tumor segmentation system based on adaptive KAN, characterized in that, include: The preprocessing module is used to preprocess multimodal three-dimensional brain magnetic resonance imaging data, including Z-score normalization for each modal channel, application of random data augmentation, and calculation of modal statistical vectors, which include the mean, variance, and skewness of each modality. The encoder module is used to receive preprocessed data. The encoder module includes multiple levels. Each level is configured to perform convolution operations to extract local features, perform spatial KAN processing to extract spatial features, and downsample the feature map. The spatial KAN processing dynamically adjusts the node positions of the B-spline basis function based on the modality statistics vector and uses a pre-computed B-spline table to accelerate the evaluation of the basis function. The bottleneck layer module receives the output of the encoder and uses multiple parallel dilated convolutional branches to capture multi-scale global context. The decoder module receives the output of the bottleneck layer. The decoder module includes multiple stages, each configured to perform transposed convolution oversampling of feature maps, perform cross-scale gating to fuse skip connection features from the encoder module with the current decoder features, and perform spatial KAN processing to optimize features, wherein the cross-scale gating uses the KAN gating mechanism. The output module receives the output of the decoder. The output module uses 1×1×1 convolution to reduce the dimensionality of the feature map to the number of classes and generates a multi-class probability segmentation mask through the softmax activation function.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the multi-scale brain tumor segmentation method based on adaptive KAN as described in any one of claims 1-8.
Citation Information
Cited By
Ultra-light UNet segmentation method and system for jacquard fabric texture
CN121837643A
Brain tumor MRI image segmentation method based on soft clustering KAN network
CN122023810A
Key structure image segmentation method and system for obstetrical ultrasound
CN122049379A
