Medical image segmentation method based on KAN double branch and MIFA-HiLo collaborative optimization

Abstract

This invention discloses a medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization, belonging to the field of artificial intelligence and medical image processing technology. It aims to solve the technical problems of insufficient nonlinear expression capability, imbalance between local and global feature modeling, and insufficient multi-scale information fusion in existing methods. The method includes: acquiring a preprocessed image tensor; constructing a dual-branch encoder containing convolutional and KAN-Transformer branches; constructing a MIFA multi-scale integrated feature aggregation module to achieve dynamic modulation and fusion of local and global features through a bidirectional cross-gating mechanism; constructing a multi-scale progressive decoder to progressively restore spatial resolution; constructing a HiLo feature optimization module to perform high-frequency enhancement and low-frequency recalibration of features through channel partitioning; and finally outputting a pixel-level probability prediction map.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical image processing technology. More specifically, this invention relates to a medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization. Background Technology

[0002] Medical image segmentation is a crucial component of computer-aided diagnostic systems. Its goal is to accurately segment lesion regions from two-dimensional medical images (such as dermoscopy, ultrasound, and endoscopic images) to provide quantitative evidence for clinical diagnosis. Because lesion regions typically exhibit large scale variations, irregular shapes, blurred boundaries, and strong background interference, medical image segmentation tasks place high demands on the model's expressive power, local detail capture capabilities, and global semantic modeling abilities.

[0003] In two-dimensional medical image segmentation tasks, deep learning methods have become the mainstream technology. Existing methods mainly include segmentation models based on convolutional neural networks (CNN), segmentation models based on Transformers, and improved models that introduce novel nonlinear representation structures into existing frameworks.

[0004] Early medical image segmentation methods were mostly based on convolutional neural networks to construct encoder-decoder structures. They extracted features through multiple layers of convolution and downsampling, and restored spatial resolution through skip connections. These methods are strong in local texture modeling, effective at extracting edge details and low-level features, and have relatively controllable parameter sizes and stable training processes. However, due to the inherent limitation of local receptive fields in convolutional operations, their ability to model long-distance dependencies is insufficient. They have limitations in handling large-scale structures, complex morphological regions, or cross-regional semantic associations, easily leading to insufficient global semantic representation and thus affecting overall segmentation consistency.

[0005] With the development of visual Transformers in computer vision, segmentation models based on self-attention mechanisms are increasingly being applied to medical image segmentation tasks. Transformers, through global self-attention, can explicitly model dependencies between distant pixels, demonstrating strong global semantic modeling capabilities in segmenting complex backgrounds and irregular lesion regions. However, existing Transformer structures typically rely on linear mappings to construct query, key, value, and feedforward network (MLP) layers. Their feature transformation process is mainly based on matrix multiplication, and their nonlinear expressive power depends on the superposition of activation functions, limiting their function approximation ability. In medical images, lesion regions often exhibit complex shapes, low contrast, and a high proportion of small objects; simply relying on stacked linear mappings may be insufficient to fully characterize fine-grained feature changes.

[0006] Furthermore, the computational complexity of the standard self-attention mechanism increases quadratically with the feature map size. Although some improved methods employ window attention or hierarchical structures to reduce computation, they still suffer from high computational overhead and memory consumption. This problem is particularly pronounced in high-resolution medical image segmentation tasks. Simultaneously, a single Transformer branch is relatively insufficient for modeling low-level detailed features, easily leading to blurred boundaries or missed detections in small regions.

[0007] In recent years, the Kolmogorov-Arnold Network (KAN), as a nonlinear representation model based on basis function expansion, has been introduced into visual models to enhance the approximation ability of nonlinear functions. Some existing studies have attempted to embed KAN modules into traditional network structures, such as replacing or adding them to certain layers of convolutional networks or Transformer structures. However, most of these methods only locally replace feedforward networks (MLPs) or introduce KAN structures as additional modules, failing to achieve a systematic reconstruction of the self-attention structure. This results in limited effectiveness in improving global dependency modeling and nonlinear representation capabilities. Furthermore, improvements to a single structure often neglect multi-scale information interaction and frequency feature decomposition, making it difficult to simultaneously consider both global semantic information and local detail representation.

[0008] In practical two-dimensional medical image segmentation tasks, lesion regions typically exhibit characteristics such as large scale variations, irregular shapes, blurred boundaries, and strong background interference. This places the following requirements on the model: it should possess strong nonlinear expression capabilities to characterize complex grayscale changes and structural morphology; it should have the ability to capture local details and model global dependencies; it should be able to perform efficient information fusion between different scales; and it should control computational complexity and parameter scale while ensuring segmentation accuracy.

[0009] However, existing technologies often involve trade-offs between the above objectives: pure CNN models lack global modeling capabilities; pure Transformer models have high computational complexity and limited detail representation; existing KAN improvement methods are mostly local replacements and have not been deeply integrated with attention mechanisms and multi-scale fusion structures. At the same time, existing methods lack the ability to separate and model high- and low-frequency information, making it difficult to balance overall structural consistency and boundary fineness.

[0010] Therefore, it is necessary to propose a two-dimensional medical image segmentation method that can enhance nonlinear expressive power, strengthen multi-scale information fusion, and take into account both local details and global dependency modeling, while keeping computational complexity under control, in order to overcome the above-mentioned shortcomings of existing technologies. Summary of the Invention

[0011] This invention provides a medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization, which can significantly improve the segmentation accuracy of lesion areas, especially performing well in small-scale lesions and complex boundary areas.

[0012] To achieve these objectives and other advantages of the present invention, a medical image segmentation method based on KAN dual-branch and MIFA-HiLo co-optimization is provided, comprising the following steps: S1. Obtain the two-dimensional medical image and its corresponding mask. After format cleaning, binarization, data augmentation, tensor transformation and standardization, the preprocessed image tensor and mask tensor are obtained. S2. Construct a dual-branch encoder including a convolutional branch and a KAN-Transformer branch, and input the preprocessed image tensor into the two branches respectively; the convolutional branch is constructed based on a residual network, and extracts local texture feature maps at different scales through the residual module; the KAN-Transformer branch segments the image into blocks and maps them into a sequence of feature vectors, which are then input into a multi-layer KAN-Transformer module; each KAN-Transformer module uses a KANLinear layer to replace the linear mapping layer and feedforward network layer generated from the query, key, and value in the attention mechanism, and the module output is spatially reconstructed into a two-dimensional feature map to obtain global semantic feature maps at different scales; S3. Construct a multi-scale integrated feature aggregation module, which receives local texture feature maps and global semantic feature maps with consistent spatial resolution from the dual-branch encoder. Align the global semantic feature map through a 1×1 convolution. Apply depthwise separable convolution to the aligned feature maps and activate them with ReLU and GELU respectively. Then, generate local gated weight maps and global gated weight maps using a Sigmoid function. Multiply the global gated weight map and the local texture feature map pixel-by-pixel to obtain the modulated local features. Multiply the local gated weight map and the aligned global semantic feature map pixel-by-pixel to obtain the modulated global features. Add the two modulated features pixel-by-pixel and reconstruct them using depthwise separable convolution to output a fused feature map. S4. Construct a multi-scale progressive decoder that receives fused feature maps at multiple scales, upsamples them sequentially starting from the smallest scale, concatenates them with the corresponding scale fused feature map channels, and then performs feature reconstruction through a convolutional layer to output the decoded feature map. S5. Construct a HiLo feature optimization module, which receives the decoded feature map and the corresponding scale-encoded feature map. After unifying the number of channels through convolutional layers, it is linearly weighted and fused using learnable weights. In the channel dimension, the fused feature map is proportionally divided into high-frequency sub-branch feature maps and low-frequency sub-branch feature maps. The high-frequency sub-branch is subjected to depthwise separable convolution, group normalization, and ReLU, and then added to the input residual to output the enhanced high-frequency feature map. The low-frequency sub-branch is subjected to global average pooling, convolutional layers, and Sigmoid to generate channel weights, and then multiplied with the input channel by channel to output the enhanced low-frequency feature map. The enhanced high-frequency and low-frequency feature maps are concatenated and then reconstructed through convolutional layers, group normalization, and ReLU to output the optimized feature map. S6. Map the optimized feature map to a segmentation logits map, upsample it to the original image size, and then use the Sigmoid activation function to output a probability prediction map of each pixel belonging to the target region.

[0013] Preferably, in step S2, the specific process by which the convolutional branch extracts local texture feature maps at different scales through the residual module is as follows: Apply 7×7 convolution with stride 2 and 3×3 max pooling with stride 2 to the preprocessed image tensor to obtain a shallow feature map with a spatial size of 1 / 4 of the spatial size of the preprocessed image tensor. The shallow feature map is sequentially input into multiple cascaded residual stages. Each residual stage consists of at least one residual block, and each residual stage is downsampled by a convolution with a stride of 2, so that the spatial size of the feature map is halved step by step and the number of channels is doubled step by step, and local texture feature maps of different spatial scales are output respectively. The residual stage includes a first residual stage, a second residual stage, and a third residual stage. The spatial dimensions of the local texture feature maps output by these stages are 1 / 4, 1 / 8, and 1 / 16 of the tensor spatial dimensions of the preprocessed image, respectively, and the number of channels are 64, 128, and 256, respectively. Each residual block performs the following operations: the first feature map x, which is the input of the current residual block, is sequentially processed through a first 3×3 convolution, batch normalization, and ReLU activation function, and then through a second 3×3 convolution and batch normalization to obtain a residual mapping feature map F(x). The residual mapping feature map F(x) is the feature representation of the input feature map after two convolution and batch normalization processes. The first feature map x is then processed through a 1×1 convolution to adjust the number of channels to match the number of channels in the residual mapping feature map F(x), resulting in a skip connection feature map W. s x, the skip connection feature map W s x is the feature representation of the input feature map after a 1×1 convolution transformation; according to the formula y = F(x, {W i})+W s x combines the residual mapping F(x) with the skip connection feature W sx is added element by element and then activated by the ReLU function to obtain the second feature map y as the output of the residual block. The second feature map y is the output feature map after the current residual block is processed. When the number of channels of the input feature map x is equal to the number of channels of the residual mapping F(x), the skip connection feature is the input feature map x itself, and no 1×1 convolution adjustment is required. The convolutional branch does not contain a global average pooling layer or a fully connected classification layer; it only outputs local texture feature maps at the above three scales for use in the subsequent multi-scale integrated feature aggregation module.

[0014] Preferably, in the KAN-Transformer branch of S2, each KAN-Transformer module internally replaces the linear mapping layer for query, key, and value generation and the feedforward network layer in the self-attention mechanism with a KANLinear layer; the output of the KANLinear layer is... Composition, where w b Based on activation weights, w s For spline weights; Spline(x) takes the form of Gaussian radial basis functions: α k For the basis function weights, μ k Center of basis functions, σ k denoted as the width of the basis functions, and K as the number of basis functions.

[0015] Preferably, in the KAN-Transformer branch, when the spatial size of the input image is inconsistent with the preset training size, bilinear interpolation is used to apply bilinear interpolation to the learnable position encoding tensor E. pos Adjustments are made to obtain a new positional encoding tensor E that matches the length of the current input feature vector sequence. pos new The bilinear interpolation calculation formula is as follows: Where H and W are the height and width of the input image.

[0016] Preferably, in S2, the KAN-Transformer branch divides the preprocessed image tensor into a sequence of image blocks through a 16×16 convolution with a stride of 16 pixels, and linearly maps each image block to a feature vector to obtain an initial feature vector sequence. The dimension of the initial feature vector sequence is B×N×C, where B is the batch size, N is the number of image blocks, and C is the embedding dimension. The learnable position encoding tensor is added element by element to the initial feature vector sequence to obtain a feature vector sequence with position information. The dimension of the learnable position encoding tensor is 1×(N+1)×C, which contains a token for classification. The sequence of feature vectors with location information is sequentially input into multiple consecutive KAN-Transformer modules. Each KAN-Transformer module performs the following operations: The input feature vector sequence is subjected to layer normalization to obtain the first normalized sequence. The layer normalization calculation formula is as follows: Where μ is the mean and σ is the mean. 2 Let ε be the variance, γ be the minimum constant, and β be learnable parameters. The KANLinear layer maps the first normalized sequence to a query vector sequence Q, a key vector sequence K, and a value vector sequence V, each with a dimension of B×h×N×d, where h is the number of attention heads, d is the dimension of each head, and C=h×d. The self-attention output is calculated based on the query vector sequence Q, the key vector sequence K, and the value vector sequence V. The calculation formula is as follows: The first attention output is obtained; The first attention output is added element by element to the input feature vector sequence of the current KAN-Transformer module to obtain the first residual output; The first residual output is layer normalized to obtain the second normalized sequence; The second normalized sequence is transformed by two consecutive KANLinear layers. The first KANLinear layer expands the feature dimension from C to rC, where r is a preset multiple. The second KANLinear layer restores the feature dimension from rC to C, thus obtaining the feedforward output. The feedforward output is added element by element to the first residual output to obtain the output feature vector sequence of the current KAN-Transformer module; Spatial reconstruction is performed on the feature vector sequence output by the last KAN-Transformer module. The tokens used for classification are removed, and the remaining feature vector sequence is reshaped into a two-dimensional feature map to obtain a global semantic feature map aligned with the spatial scale of the convolution branch. Its dimension is B×C×(H / 16)×(W / 16), where H and W are the height and width of the preprocessed image tensor. Specifically, when the spatial size of the preprocessed image tensor is inconsistent with the preset training size, bilinear interpolation is used to adjust the learnable position encoding tensor so that the length of the adjusted position encoding tensor matches the length of the feature vector sequence generated by the current input. The bilinear interpolation calculation formula is as follows: , where (x i ,y j ) represents the coordinates of the four grid points surrounding the interpolation point, w ij These are the corresponding interpolation weights.

[0017] Preferably, in step S3, the multi-scale integrated feature aggregation module receives the first feature map F from the convolutional branch. c ∈R C×H×W And the second feature map F from the KAN-Transformer branch t ∈R Ct×H×W Where C is the number of feature channels in the convolution branch, Ct is the number of feature channels in the KAN-Transformer branch, and H and W are the height and width of the feature map; The second feature map F is processed by 1×1 convolution. t Perform channel alignment, and denote the aligned second feature map as F. t '∈R C ×H×W ; For the first feature map F respectively c and the aligned second feature map F t 'Applying 3×3 depthwise separable convolution, we obtain the first convolution feature and the second convolution feature. The 3×3 depthwise separable convolution includes two steps: depthwise convolution and pointwise convolution. The depthwise convolution performs 3×3 convolution independently on each input channel, and the pointwise convolution fuses the information of each channel through 1×1 convolution.' Apply the ReLU activation function to the first convolutional feature and the GELU activation function to the second convolutional feature to obtain the first activation feature and the second activation feature; The first and second activation features are mapped to the 0-1 interval using the Sigmoid function, respectively, to generate a local gate control weight map M. c and global gating weight graph M t ; The global gating weight graph M t With the first feature map F c Pixel-by-pixel multiplication yields the modulated local features. The calculation formula is: , where ⊙ represents pixel-wise multiplication; The departmental control weight map M c With the aligned second feature map F t 'Multiply pixel by pixel to obtain the modulated global features' The calculation formula is: ; Modulated local features With modulated global features By adding pixels one by one, we obtain the initial fusion feature F. sum The calculation formula is: ; Preliminary fusion feature F sum A 3×3 depthwise separable convolution is applied for feature reconstruction to obtain the final fused feature map F. fusion∈R C×H×W The fused feature map F fusion Simultaneously includes the first feature map F c Local texture information and second feature map F t Global semantic information.

[0018] Preferably, in step S4, the multi-scale progressive decoder receives a set of fused feature maps from the multi-scale integrated feature aggregation module. The set of fused feature maps includes a first fused feature map F1, a second fused feature map F2, and a third fused feature map F3, wherein the spatial size of F1 is 1 / 4 of the tensor space size of the preprocessed image, the spatial size of F2 is 1 / 8 of the tensor space size of the preprocessed image, and the spatial size of F3 is 1 / 16 of the tensor space size of the preprocessed image. Starting with the third fused feature map F3, which has the smallest spatial size, the spatial resolution is restored level by level, and the decoding feature map D of the current level is processed sequentially. k Applying bilinear interpolation for a 2x upsampling, we obtain the upsampled feature map U. k The bilinear interpolation calculation formula is as follows: , where w ij These are weighting coefficients calculated based on the distance between four adjacent pixels; Upsampled feature map U k The fused feature map F{k-1} at the corresponding scale is concatenated along the channel dimension to obtain the concatenated feature map Concat. k The concatenated feature map Concat k The number of channels is U k The sum of the number of channels of F{k-1}; Concat the spliced ​​feature maps k Feature reconstruction is performed sequentially using 3×3 convolution, batch normalization, and ReLU activation function to obtain the output decoded feature map D{k-1} of the current layer. The calculation formula for the 3×3 convolution is as follows: , where W is the convolution kernel weight; Repeat the above upsampling, stitching and reconstruction steps to generate the second decoded feature map D2 and the first decoded feature map D1 in sequence, wherein the spatial size of the second decoded feature map D2 is 1 / 16 of the tensor space size of the preprocessed image, and the spatial size of the first decoded feature map D1 is 1 / 8 of the tensor space size of the preprocessed image. Applying bilinear interpolation to the first decoded feature map D1 again for upsampling by a factor of 2, we obtain the final decoded feature map whose spatial size is restored to 1 / 4 of the tensor spatial size of the preprocessed image. .

[0019] Preferably, in step S5, the HiLo feature optimization module receives a first input feature map, a second input feature map, and a third input feature map, wherein the first input feature map is an encoded feature map from the convolutional branch. The second input feature map is the encoded feature map from the KAN-Transformer branch. The third input feature map is a decoded feature map from a multi-scale progressive decoder. Where B is the batch size, and H and W are the feature map height and width; The first input feature map F is processed by three independent 1×1 convolutions. c Second input feature map F b2 and the third input feature map F c2 Perform channel alignment, and denote the aligned three-way feature maps as follows: as well as Their dimensions are all uniformly B×128×H×W; The learnable fusion weight vector w is normalized using the Softmax function and then weighted and combined with a preset smoothing coefficient to obtain the smoothed fusion weight vector. The smoothed fusion weight vector The calculation formula is =0.5×Softmax(w) + 0.5 / 3, where w is a learnable parameter vector of dimension 3; Based on the smoothed fusion weight vector The three-way aligned feature maps are linearly weighted and fused to obtain the fused feature map F∈R. 128×H×W The calculation formula is: ,in The smoothed fusion weight vector The three components; The fused feature map F is divided along the channel dimension according to a preset scaling factor to obtain the high-frequency sub-branch feature map F. hi and low-frequency sub-branch feature map F lo The high-frequency sub-branch feature map F hi The number of channels is 64, and the low-frequency sub-branch feature map F lo The number of channels is 64, and the value of the scaling factor is 0.5; For the high-frequency sub-branch feature map F hi By sequentially applying 3×3 depthwise separable convolution, group normalization, and ReLU activation function, the enhanced high-frequency feature F is obtained. hi ', will enhance the high-frequency feature F hi 'Feature map F of high-frequency sub-branch hi Pixel-by-pixel addition outputs the enhanced high-frequency feature map F. hiout The calculation formula is F hi out = F hi '+F hi ; For the low-frequency sub-branch feature map F lo Global average pooling, 1×1 convolution, and sigmoid activation function are applied sequentially to generate channel weight vector A. Global average pooling compresses the spatial dimension to 1×1, resulting in an output dimension of B×64×1×1. The sigmoid activation function maps the output of the 1×1 convolution to the 0-1 interval, yielding channel weight vector A. Channel weight vector A is then compared with the low-frequency sub-branch feature map F. lo Channel-by-channel multiplication yields the enhanced low-frequency feature map F. lo out The calculation formula is F lo out =F lo ⊙A, where ⊙ represents channel-by-channel multiplication; The enhanced high-frequency feature map F hi out Compared with the enhanced low-frequency feature map F lo out The feature map is obtained by stitching the data along the channel dimension to restore it to 128 channels. ; splicing feature map F cat The features are reconstructed by sequentially applying 3×3 convolution, group normalization, and ReLU activation function to obtain the final optimized feature map. .

[0020] Preferably, the training phase also includes a deep supervision step: at least one side-path output is extracted from the intermediate layer of the multi-scale progressive decoder, each side-path output is mapped to a side-path segmentation logits map through an independent 1×1 convolutional layer, upsampled to the original image size by bilinear interpolation, and the loss function between the side-path segmentation logits map and the mask tensor is calculated. This loss function is a weighted sum of the binary cross-entropy loss and the Dice loss. The side-path loss and the main output loss are then combined according to... The weighted sum is used to update the network parameters, where λ i is the weighting coefficient for the i-th side path loss.

[0021] Preferably, after outputting the probability prediction map of each pixel belonging to the target region, a post-processing step is further included: obtaining a segmentation threshold τ through adaptive search of the validation set, setting pixels in the probability prediction map greater than the threshold to 1, and the rest to 0, generating a binary segmentation mask M that satisfies... , where M(x,y) is the pixel value of the probability prediction map at position (x,y).

[0022] The present invention offers at least the following advantages: The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization, constructed using a convolutional neural network and a KAN-Transformer dual-branch encoder, introduces a KANLinear layer within the Transformer to replace the traditional linear mapping, significantly enhancing the model's ability to express nonlinearities of complex lesions and more precisely characterizing grayscale variations and structural differences. Through the bidirectional cross-gating mechanism of the multi-scale integrated feature aggregation module, dynamic modulation and deep collaboration between local texture features and global semantic features are achieved, effectively solving the problem of inconsistent feature spaces in heterogeneous branches and sharpening lesion boundaries while suppressing background noise. Through channel partitioning and differential enhancement in the HiLo feature optimization module, high-frequency branches strengthen boundary and texture details, while low-frequency branches recalibrate global semantics through channel attention, further improving the consistency and boundary accuracy of segmented regions. In terms of computational efficiency, the present invention employs depthwise separable convolution and a lightweight normalization structure, achieving low computational overhead while ensuring high performance. The model exhibits strong robustness to changes in input resolution, device differences, and noise interference, with minimal performance degradation when transferring data across datasets, demonstrating good generalization ability and clinical adaptability. Furthermore, this invention is implemented based on standard convolution operators, making it easy to integrate and deploy in existing medical imaging systems.

[0023] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0024] Figure 1 This is a flowchart of the medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization described in this invention.

[0025] Figure 2 This is a flowchart of the multi-scale integrated feature aggregation module (MIFA) in this invention.

[0026] Figure 3 This is a flowchart of the feature optimization module (HiLo) in this invention.

[0027] Figure 4 A visual comparison of the segmentation results of ultrasound images.

[0028] Figure 5 A comparison of the segmentation results of dermoscopy images.

[0029] Figure 6 A visualization comparison of the segmentation results of endoscopic images. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to specific embodiments, so that those skilled in the art can implement it based on the description.

[0031] It should be understood that terms such as “having,” “comprising,” and “including” as used herein do not exclude the presence or addition of one or more other elements or combinations thereof.

[0032] It should be noted that, unless otherwise specified, the experimental methods described in the following implementation plan are all conventional methods, and the reagents and materials described are all commercially available unless otherwise specified.

[0033] Example 1 like Figure 1-3 As shown, the medical image segmentation method based on KAN dual-branch and MIFA-HiLo co-optimization includes the following steps: S1: Data Preprocessing and Input Located at the front end of the entire deep learning system is a data preprocessing and input module, consisting of a data loading and format cleaning sub-component, an Albumentations strong data augmentation sub-component, and a tensor transformation and normalization sub-component. The data loading sub-component is responsible for directly extracting and normalizing the data format from efficient NumPy arrays (.npy); the data augmentation sub-component integrates a set of composite transformation operations designed for medical image features (including contrast-limited adaptive histogram equalization CLAHE, elastic deformation, and noise injection); the tensor normalization sub-component is a linear transformation layer based on ImageNet pre-trained statistics. These sub-components are connected in a serial pipeline: the raw data first undergoes format cleaning and alignment, then high-intensity random augmentation is applied during the training phase, and finally it is converted into PyTorch tensors and globally normalized. The entire module runs on the CPU, using multi-threaded asynchronous loading (num_workers) to ensure efficient data flow to the GPU.

[0034] The working principle of this data preprocessing and input module combines the characteristics of 2D medical images. The module significantly enhances lesion edges through CLAHE, simulates physical tissue deformation caused by probe pressure through elastic deformation, and improves the model's robustness to debris artifacts through various noise injections. This dynamic preprocessing and enhancement mechanism effectively avoids model overfitting and ensures that the data input to the network can fully cover the heterogeneity of tumors.

[0035] The specific processing steps are as follows: Loading raw data and format cleaning: Pre-packaged image and mask NumPy files (.npy format) are read from the specified path. After extracting a single sample based on the index, the image channel count is checked first: if a single-channel grayscale image is detected, it is forcibly copied to a 3-channel RGB image using color space conversion (e.g., cv2.cvtColor) to adapt to the pre-trained backbone network; then, the data type is checked; if the pixel value is normalized to between 0 and 1, it is multiplied by 255 and converted to a uniform uint8 (0-255) format. Simultaneously, strict binarization is performed on the Ground Truth mask (pixels greater than 0.5 are set to 1.0, and the rest to 0.0) to ensure the purity of the supervision signal.

[0036] Strong data augmentation using Albumentations involves applying a series of probabilistic spatial and pixel-level transformations to the cleaned image and mask. First, the CLAHE algorithm is applied with a certain probability (e.g., 0.5) to enhance local contrast and highlight lesion boundaries. Next, geometric transformations are performed, including random scaling and cropping, horizontal / vertical flipping, and translational scaling and rotation, to eliminate positional biases. Subsequently, Elastic Transform or Grid Distortion is applied to simulate the non-rigid deformation of biological tissue. Finally, pixel-level perturbations are applied, including random brightness and contrast adjustments, and the injection of Gaussian blur, Gaussian noise, or multiplicative noise to simulate imaging differences under different devices or lighting conditions. The augmentation operations strictly ensure synchronization of the spatial transformations of the image and mask.

[0037] Tensor Transformation and Global Standardization: The processed NumPy array is converted into a PyTorch tensor. Image data is first transformed from [H, W, C] to [C, H, W] using the ToTensor operation, and pixel values ​​are linearly scaled to the [0.0, 1.0] range. Then, for each of the image's RGB channels, the channel mean of the ImageNet dataset (0.485, 0.456, 0.406) is subtracted and divided by the standard deviation (0.229, 0.224, 0.225) to perform Z-score standardization, accelerating network convergence. Finally, an explicit channel dimension is added to the mask tensor to adapt its format to the computational requirements of the network output.

[0038] Input: A set of NumPy numbers consisting of the original 2D image and its corresponding label mask. The spatial dimension of a single image is not fixed, and the number of channels is usually 3 or 1.

[0039] Output: A preprocessed, normalized image tensor (shape [B, 3, 224, 224], where B is the batch size, 3 is the number of RGB channels, and 224 is the normalized spatial height and width) and the corresponding binarized mask tensor (shape [B, 1, 224, 224], where 1 is the single-channel class label). The output will be directly fed into the starting point of the dual-branch (CNN and Transformer) encoder module.

[0040] S2: CNN and KAN-Transformer dual-branch encoder In medical image segmentation tasks, capturing the fine edges of lesions (local features) is just as important as understanding the relative position of lesions within anatomical structures (global features). This invention employs a dual-branch encoder architecture based on CNN (Convolutional Neural Network) and KAN-Transformer (Kolmogorov-Arnold network-based transformer). This architecture aims to achieve deep decoupling and synergy between local feature capture and global long-range dependency modeling through parallel processing streams.

[0041] Overall workflow: Input preprocessed image Subsequently, the data is simultaneously fed into two independent branches. The CNN branch, acting as a "bottom-up" path, compresses the image into high-dimensional, low-resolution feature maps through layer-by-layer convolution and pooling. The Transformer branch, acting as a "bottom-up and globally coupled" path, transforms the image into serialized vectors (tokens) through patch embedding. The Transformer branch abandons traditional linear MLPs and linear attention mappings, instead employing the KAN (Kolmogorov-Arnold Networks) architecture for non-linear feature reconstruction. The two feature paths remain aligned across three different spatial scales, providing multi-dimensional semantic information for subsequent multi-scale integrated feature aggregation (MIFA).

[0042] CNN branches: The CNN branch is built on a deep residual network (ResNet34) and is specifically responsible for extracting texture, color variations and subtle edge features in medical images through sliding computation of local receptive fields.

[0043] Furthermore, this branch belongs to a typical hierarchical convolutional feature encoding structure. It relies on the spatial locality and parameter sharing characteristics of the convolution kernel to gradually expand the receptive field through layer-by-layer nonlinear mapping, thereby achieving progressive abstraction from low-level texture features to high-level semantic features.

[0044] The basic form of convolution operation is: Where: K is the kernel size, C in is the number of input channels, W is the kernel weight, and b is the bias term.

[0045] Each convolution is followed by Batch Normalization and ReLU activation functions: Where μ and σ 2 This is a batch statistical measure.

[0046] Working principle and detailed steps: Preliminary feature mapping: Input image First, it goes through a large 7×7 convolutional layer with a stride of 2. and the max-pooling layer MaxPool(3×3, stride=2) This reduces the spatial dimension to 1 / 4 of the original size, allowing for the initial extraction of shallow textures. The receptive field growth formula is approximately: RF l =RF l-1 +(K l -1)×S l-1 .

[0047] Residual cascaded downsampling: The model utilizes the first three stages of ResNet (Layer 1, Layer 2, Layer 3) for cascaded processing. Layer 1: It contains multiple residual blocks, with an output size of 56×56 and 64 channels. This stage preserves extremely high spatial resolution to record the minute jagged features of lesion edges. Layer 2: Downsampling is performed using convolutions with a stride of 2, resulting in an output size of 28×28 and 128 channels. Layer 3: Further downsampling to 14×14, with 256 channels, resulted in features possessing strong semantic attributes.

[0048] Customized structural adjustments: To prevent excessive loss of spatial information from deep features, the model explicitly replaces ResNet's Layer 4 with nn.Identity() (identity mapping) and removes the global average pooling layer and classification head, ensuring that the output is a set of 2D feature maps F that preserves spatial structure. cnn ={x u2 ,x u1 ,x u}

[0049] Characteristic formula description: The residual calculation for each layer follows: y=F(x,{W i})+W s x where F represents the residual mapping, and W sThis is a 1×1 convolution performed to match the channel dimension. The residual mapping is internally calculated as F(x) = σ(BN(W2(σ(BN(W1(x)))))), where W1 and W2 are 3×3 convolutions, BN is Batch Normalization, and σ is ReLU. The computational complexity of the convolutional layer is O(K). 2 ·C in ·C out (·H·W), as the spatial size decreases, although the number of channels increases, the overall computational cost remains manageable. This is due to the existence of residual connections: The identity term I guarantees that the lower bound of the gradient is non-zero, thus alleviating the problem of deep degradation.

[0050] KAN-Transformer branch: The Transformer branch reshapes the linear transformation logic within the Transformer by introducing the KAN structure. Structurally, this branch inherits the ideas of the Vision Transformer (ViT) framework (similar to the architectural paradigm proposed by AnImage isWorth 16x16 Words), but replaces all linear mapping layers with KANLinear nonlinear operators, upgrading the feature transformation from "linear projection + fixed activation" to "learnable function family mapping".

[0051] The basic input form of Transformer is Where B is the batch size, N is the number of tokens, and C is the embedding dimension.

[0052] KAN's core substitution principle: Unlike traditional Transformers that use linear layers (y=Wx+b) with fixed activation functions (such as ReLU / GELU), this branch uses the KANLinear module. Traditional linear mappings are represented as: Essentially an affine transformation, KAN's expressive power depends on the stacking depth. The core theoretical foundation of KAN originates from the Kolmogorov-Arnold Representation Theorem: This demonstrates that any multivariate continuous function can be represented as a combination of univariate functions. The basis formula for KAN is: , where w b Based on activation weights, w s For the Spline weights, silu(x) = x·sigmoid(x).

[0053] In this implementation, Spline(x) takes the form of Gaussian radial basis function (RBF): , where μ k Centered on, σ k For width, α k These are learnable weights. Therefore, a single KAN node can be considered as: ; Compared with traditional linear layers, the mapping capability is improved to the "function space" level, the parameter expression capability is enhanced, and higher-order nonlinearity is explicitly modeled.

[0054] Details of core component replacement: KAN-Attention (Non-linear Attention): In traditional Multi-Head Attention, In this model, it is replaced with: Attention weight calculation still follows: However, since Q and K have already undergone mapping by a family of nonlinear functions, the similarity matrix is ​​no longer a simple inner product matching, and is more suitable for capturing complex cross-regional structural dependencies. If we let the number of heads be h, then The complexity remains O(N). 2 d), but the ability to express features is significantly enhanced.

[0055] KAN-MLP (Nonlinear Feedforward Network): The traditional MLP formula is MLP(x) = W2(σ(W1x)), which is replaced in this model with KAN-MLP(x) = KANLinear2(KANLinear1(x)). Dimensional change: , where r = mlp_ratio (usually 4).

[0056] Since each node contains a learnable activation function, the overall function approximation capability of the model is approximately: It can effectively fit complex morphological distributions in medical images.

[0057] Work steps and feature reconstruction: Patch embedding and position enhancement: The image is segmented into 16×16 patches, embedded into a vector sequence via convolution, and then incorporating a learnable positional encoding E. pos .

[0058] Patch Embedding is essentially... The output is Add positional encoding x'=x+E pos ,in .

[0059] KAN-Block stacking calculation: The sequence is then fed into 12 (Base) KAN-Transformer Blocks. A single layer structure is X1 = X + KAN-Attention(LN(X)), X2 = X1 + KAN-Attention(LN(X1)). The LayerNorm formula is... The formula for ensuring gradient stability using residual structure is: .

[0060] Dynamic positional encoding interpolation: For inputs other than 224, the module will dynamically adjust E through bilinear interpolation. pos The shape, the formula is: Bilinear interpolation is calculated as follows: This is to ensure spatial continuity.

[0061] Tokens-to-Map spatial reorganization: To integrate with CNN branches, the model will output the sequence After removing the CLS Token Reshape into a 2D shape ,Right now .

[0062] In this way, the global semantics extracted by the Transformer are mapped back to the spatial coordinate system, achieving a perfect intersection with the local features of the CNN on a 14×14 scale.

[0063] Comparison of expressive abilities The traditional Transformer function family is The KAN-Transformer formula is: Theoretically, it has a stronger function approximation capability.

[0064] Parameter size influence Assuming embed_dim = 768 and mlp_ratio = 4, the number of parameters in a traditional MLP is approximately 768×3072+3072×768. KANLinear adds RBF basis function parameters on this basis, but does not significantly increase the order of magnitude.

[0065] S3: Multi-scale Integrated Feature Aggregation (MIFA) To achieve deep collaboration between the local details captured by CNN branches and the global semantics modeled by KAN-Transformer branches, this paper designs a Multi-scale Integrative Feature Aggregation (MIFA) module. This module, through a bidirectional cross-gating mechanism, achieves dynamic modulation and feature reconstruction of cross-branch information while maintaining an extremely lightweight structure, effectively solving the problem of inconsistency in the feature spaces of heterogeneous branches.

[0066] Module structure and key component characteristics The architecture of the MIFA module follows the principle of balancing efficiency and non-linear representation. Its first step is the channel alignment layer. Since the embedding dimension of the Transformer branch typically does not match the number of channels in the output of each stage of the CNN, the module first uses a 1×1 convolution to adjust the Transformer features F. t Perform a linear transformation to map it to the CNN feature F c The same channel space C is used, thus laying the foundation for subsequent element-level interactions without changing the spatial resolution.

[0067] The module extensively employs 3×3 depthwise convolution (DWConv). Compared to traditional convolution, depthwise convolution performs spatial aggregation independently within each channel, and its parameter count is only 1 / C of that of standard convolution. This design ensures that the module maintains extremely low computational overhead when extracting spatial structural information of lesions.

[0068] For nonlinear modeling, the module introduces an asymmetric activation gating design. When generating the guiding weight map, ReLU activation is used for the CNN branch, while GELU activation is used for the more semantically abstract Transformer branch. The two feature paths are then mapped through depthwise convolutions and passed through the Sigmoid function. Mapping to the [0,1] interval generates a gated graph M that can represent the saliency of features. c With M t This differentiated nonlinear mapping layer enhances the model's ability to capture the heterogeneous features of lesion regions.

[0069] Working principle and information flow mechanism The core mechanism of MIFA is bidirectional cross-modulation. It uses the spatial importance distribution learned by one branch as modulation weights to guide the feature recalibration of the other branch, thereby achieving collaborative optimization of cross-branch features. Its complete information processing flow is as follows: During the feature input phase, the module receives the same-scale feature map F from the two branches. c With F t First, the Transformer features are aligned by channels to obtain F. t Subsequently, the module calculates the two-way gating weights in parallel: using F c Generate local control M c Using F t Generate global gate M t .

[0070] Upon entering the cross-modulation stage, the module breaks free from the limitation of independent processing by each branch, implementing interchangeable guidance. Specifically, the global gating graph M... t The feature F applied to the CNN c Global context information is used to suppress noise responses in local branches caused by hair or imaging artifacts in the image; at the same time, the local departmental map M c The feature F applied to the Transformer t This method utilizes the fine boundary information captured by CNNs to spatially sharpen relatively blurred global semantics. This process is achieved through pixel-by-pixel multiplication. .

[0071] After modulation, the two enhanced features are initially fused by element-wise addition. To further eliminate the distribution bias introduced by cross-branch interactions, the fused feature map is fed into a second deep convolutional reconstruction layer. This layer is responsible for local structural smoothing and semantic reorganization of the fused features, ultimately outputting a high-quality feature map F that retains both the edge sensitivity of the CNN and the global consistency of the Transformer. fusion .

[0072] Input-output characteristic analysis The MIFA module has excellent adaptability. Its input receives two feature maps F with identical spatial dimensions but different channel numbers. c ∈R C×H×W and F t ∈R Ct×H×W Its output produces a fused feature F that is aligned with the dimensions of the CNN branches. fusion ∈R C×H×W .

[0073] The technical advantage of this design lies in the fact that adaptive adjustment of the gating weights enables dynamic focusing on the lesion region without introducing a complex QKV attention mechanism. In the TransFuse architecture, the MIFA module runs in cascade at multiple downsampling scales, ensuring that each level of the multi-scale features completes cross-dimensional semantic alignment and detail complementarity during transmission to the decoder.

[0074] S4: Progressive Multi-scale Decoder After completing dual-branch encoding, MIFA multi-scale fusion, and HiLo structural remodeling, the network has obtained high-quality feature representations that combine local boundary sensitivity with global semantic consistency. However, these features are still in a low-resolution semantic space (such as 14×14 or 28×28), which cannot directly generate fine pixel-level segmentation results. Therefore, a progressive multi-scale decoder is designed to progressively restore spatial resolution and refine lesion boundaries.

[0075] The decoder follows the U-shaped layer-by-layer upsampling idea as a whole, but combines features from the fusion of CNN and Transformer branches at each scale to achieve cross-scale semantic compensation and structural enhancement.

[0076] Decoder overall structure Let F be the fused multi-scale feature set. fusion ={F3,F2,F1}, where , , The decoder starts from the deepest feature F3 and recovers the spatial resolution step by step from the bottom up. The core process is as follows: ,in For 2× upsampling, Concat indicates channel splicing. Represents the convolutional reconstruction module Upsampling mechanism The decoder uses bilinear interpolation for spatial amplification. Compared to deconvolution (Transposed Conv), bilinear interpolation has the advantages of having no additional parameters, avoiding checkerboard artifacts, and being computationally stable, thus making the overall decoding process more lightweight.

[0077] Feature concatenation and convolutional reconstruction unit At each scale, the decoded features are skip-connected to the corresponding encoder features. cat =Concat(F up ,F skip After splicing, the data is fed into the convolutional reconstruction unit F. out =σ(BN(Conv 3×3 (F cat ))), where Conv is a 3×3 convolution and BN is BatchNorm, It is ReLU.

[0078] Convolution calculation form This step serves to eliminate channel distribution differences caused by splicing, smooth local space, and enhance boundary continuity.

[0079] Progressive semantic refinement mechanism The decoder does not simply enlarge the feature map, but rather performs a step-by-step transformation from semantics to structure at different scales: Deep layer (14×14): semantically dominant, locating lesion areas. Middle layer (28×28): Structural compensation, restoring organ contours Shallow layer (56×56): Boundary reinforcement, finely depicting the jagged structure. If the number of decoding layers is L, then the overall gradient propagation is: ; The identity mapping I in skip connections guarantees a non-zero lower bound for the gradient, thereby preventing the decay of deep semantics during backpropagation.

[0080] Integration with HiLo module After the final stage of decoding is completed, the features are obtained. .

[0081] This feature is then fed into the HiLo module for high and low frequency reshaping and optimization. The decoder itself focuses on spatial resolution restoration and multi-scale semantic integration. HiLo, on the other hand, is responsible for boundary high-frequency enhancement and global channel recalibration. Together, they form a cascaded enhancement mechanism of "structural restoration + frequency optimization".

[0082] Input / output characteristics Decoder input: {F3,F2,F1}.

[0083] Output: The spatial resolution is eventually restored to the scale of the input image.

[0084] S5: HiLo Feature Optimization After completing the MIFA multi-scale interactive fusion, the network has obtained composite features containing local texture information and cross-layer semantic information. However, the features after multi-scale superposition may still have semantic redundancy and frequency component mixing problems. To further enhance the ability to express boundary details while maintaining the consistency of the overall region structure, this paper introduces a HiLo feature optimization module before the main prediction head to perform structure reorganization and frequency approximate decomposition modeling on the fusion result.

[0085] This module is not a traditional frequency domain transformation, but rather a "high-frequency-low-frequency" approximate modeling path based on the channel partitioning concept. Through adaptive fusion of three features, channel partitioning-based frequency approximation decoupling, and a differentiated enhancement mechanism, it achieves collaborative optimization of local and global information.

[0086] Structural features The HiLo module first receives three same-scale features from the decoding stage: Transformer high-level semantic features, Transformer decoding features, and CNN decoding features. Let the three inputs be... , , .

[0087] Since the three feature channels have different numbers of channels, the module internally aligns them using three independent 1×1 convolutions, uniformly mapping them to an embedding dimension of embed_dim=128. This process only changes the channel dimension and does not affect the spatial resolution, thus ensuring the structural consistency of subsequent fusion.

[0088] After channel alignment, the modules are not simply spliced ​​together, but a learnable fusion weight vector is introduced. Obtained through Softmax normalization .

[0089] To avoid excessive concentration of weights on a single path and to improve stability in the early stages of training, the code performs smoothing regularization on the weights. Finally, the three features are linearly weighted and fused. This design ensures that features at different semantic levels can adaptively adjust their contribution ratios during training, rather than being fused using a fixed structure.

[0090] After obtaining fusion characteristics Then, the module is divided along the channel dimension according to a scaling factor α=0.5, dividing the features into high-frequency branches and low-frequency branches F. hi ,F lo =Split(F), where , This division is not a strict frequency domain decomposition, but rather based on the assumption that different channels in deep features may carry different structural information, and frequency approximation modeling is achieved through structural constraints.

[0091] Working principle and implementation mechanism In the high-frequency branch, the module uses 3×3 depthwise convolutions for local structure modeling. The depthwise convolution is achieved by setting `groups=C`. hi Channel-wise convolution is implemented, reducing the number of parameters while enhancing the local spatial response. The convolution output is then normalized by GroupNorm and activated by ReLU to enhance nonlinear expressiveness. The computation process can be represented as F... hi = ReLU(GN(DWConv(F hiTo prevent high-frequency details from being weakened during convolution, the module introduces residual connection structures in the high-frequency paths, directly superimposing the original features onto the enhanced feature F. hi out = F hi '+F hi This residual enhancement mechanism helps maintain the integrity of boundary information and texture details while improving gradient propagation stability.

[0092] In the low-frequency branch, the module uses global average pooling to compress the spatial dimension to 1×1, thereby extracting the global semantic description vector. Subsequently, channel mapping is performed through 1×1 convolution, and the Sigmoid function is used to generate channel weight coefficients A=σ(Conv) in the range [0,1]. 1×1 (AvgPool(F lo The weight vector is then multiplied channel-by-channel with the original low-frequency features to achieve channel-level recalibration F. lo out =F lo ⊙A. This design is essentially similar to a lightweight channel attention mechanism, used to suppress noise responses and enhance discriminative global semantic channels.

[0093] After the high-frequency and low-frequency paths have completed their respective enhancements, the module stitches the data together at the channel level to restore the original number of channels F. cat =Concat(F hi out ,F lo out To eliminate differences in feature distribution between branches and promote cross-frequency information interaction, the module finally reconstructs F using a fusion unit containing 3×3 convolutions, GroupNorm, and ReLU. out =ReLU(GN(Conv 3×3 (F cat This achieves further smoothing of features and semantic integration.

[0094] Input and Output The HiLo module input consists of three channels of features of the same scale, which are then aligned and unified into a single channel. After adaptive fusion and high / low frequency enhancement, the output features maintain the same spatial size and channel dimension. This feature is then fed into the final prediction head, where it is mapped to a segmentation vector through 3×3 convolutions and 1×1 convolutions. .

[0095] S6: Output Module and Prediction Generation The output module, located at the end of the entire segmentation network, is a key component that maps the high-level semantic features optimized by multi-scale fusion and HiLo Attention to the final pixel-level segmentation result. This module is designed with a lightweight structure as its core principle, reducing additional parameter overhead while maintaining expressive power. Furthermore, it enhances gradient propagation efficiency and convergence stability during training through a deep supervision mechanism.

[0096] Structural features The output module mainly consists of the following three parts: Final Prediction Head The main prediction head uses a 1×1 convolutional layer to achieve channel dimension compression. This convolutional layer is essentially a pixel-wise linear mapper that performs channel fusion on the high-dimensional feature tensors from the HiLo Attention module or the decoder. The output features are... Where C is the number of feature channels (e.g., 128 or 64), then it is obtained through a 1×1 convolution: For binary classification tasks, the number of output channels is 1; for multi-class classification tasks, the number of output channels equals the number of classes.

[0097] Bilinear Upsampling Because of the downsampling operation in the encoder-decoder architecture, the feature map resolution of the prediction head output is typically lower than the original input image size. To restore the original spatial resolution, the module uses a bilinear interpolation algorithm for scale restoration: P map =Interpolate(Logits,size=(H0,W0)), where (H0,W0) are the original input image dimensions. Bilinear interpolation is more stable than deconvolution, avoids the checkerboard artifact problem, and reduces additional parameters.

[0098] Activation function layer During the inference phase, the Logits are mapped to the [0,1] interval using the Sigmoid function: .in This represents the probability value that each pixel belongs to the lesion area.

[0099] Working principle The core objective of the output module is to transform the semantic features, which have undergone high- and low-frequency decomposition, cross-scale fusion, and attention enhancement, into segmentation results with clear structure and smooth boundaries.

[0100] First, the master prediction head performs channel-linear combination on the feature tensor. This operation is equivalent to learning a class decision weight vector for each pixel location, achieving space-preserving channel compression.

[0101] Subsequently, spatial resolution is restored through bilinear interpolation. This process maintains spatial continuity, makes the prediction boundaries smoother, and avoids unstable gradients introduced by deconvolution.

[0102] Finally, during the inference phase, a probability map is output, and a binary segmentation mask is generated using a thresholding strategy. ; where τ is the optimal threshold obtained through adaptive search on the validation set.

[0103] Deep oversight mechanism To improve the efficiency of gradient propagation during the training phase and alleviate the gradient decay problem in deep networks, this model introduces a multi-level deep supervision mechanism during the training phase.

[0104] Specifically, a side output prediction branch is introduced from the intermediate layer of the decoder (such as the output features of the MIFA module). Each side output branch contains a 1×1 convolutional layer and an upsampling unit to generate prediction maps at different scales. The loss function is calculated for each scale prediction map, and then weighted and fused with the main output loss. The loss function typically employs a weighted combination of BCE Loss and Dice Loss to simultaneously optimize region overlap and class discrimination ability. During the inference phase, only the main prediction branch is retained to ensure model computational efficiency.

[0105] Input and Output enter: Optimized feature tensors from the HiLo Attention module or the end of the decoder .

[0106] Output: Inference phase output: , which represents the probability value that each pixel is a lesion area.

[0107] Training phase output: a set of main prediction maps and multiple side prediction maps, used for multi-scale joint optimization.

[0108] This invention is a medical image segmentation network based on a dual-branch structure of CNN and KAN-Transformer, and introduces a multi-scale interactive fusion mechanism and a high- and low-frequency structure optimization mechanism. By adding KAN to improve modeling ability, construct a cross-layer semantic dynamic fusion structure (MIFA) and a frequency approximate decoupling optimization structure (HiLo), it shows significant advantages in terms of model accuracy, structural expressive ability, computational efficiency, deployment cost and sustainability.

[0109] In terms of segmentation quality and accuracy, this invention breaks through the structural limitations of traditional CNNs and Transformers that are simply spliced ​​together or fused in one direction. It uses a cross-scale feature interaction mechanism as the core driving logic of the network, enabling a bidirectional reinforcement relationship between high-level semantic information and low-level texture information, rather than a one-way complementary relationship. Through an adaptive fusion weight mechanism, the model can dynamically adjust the contribution ratio of different semantic levels, thereby more accurately characterizing the heterogeneous features of lesions, especially showing stronger discrimination ability in regions with blurred boundaries, low-contrast regions, and regions with fine structures.

[0110] Comparative experiments on the publicly available medical image segmentation datasets BUSI, ISIC, and CVC show that the proposed method achieves competitive performance improvements on all three datasets.

[0111] Figure 4-6 This is a visual comparison of the segmentation results of the medical image segmentation method based on KAN dual-branch and MIFA-HiLo co-optimization under three typical medical image modalities, which is used to intuitively verify the generalization ability, segmentation accuracy and boundary capture performance of the method of the present invention.

[0112] In the figure, the grayscale background image represents the original medical image input; the green area represents the predicted segmentation result obtained by the medical image segmentation method proposed in this invention, which is based on the KAN dual-branch and MIFA-HiLo co-optimization; and the red boundary contour represents the real human annotations (Ground Truth) provided in the dataset. By overlaying the prediction results of this invention with the real annotations, the degree of consistency between the two in spatial position and shape can be intuitively observed.

[0113] from Figure 4-6 As can be seen, the method of this invention can accurately locate the lesion area and maintain good boundary continuity against complex texture backgrounds. The green predicted area and the red ground truth annotation highly overlap in most locations, with only slight deviations at a few minor boundaries. This indicates that the KAN dual-branch feature extraction structure proposed in this invention, combined with the MIFA multi-scale interactive fusion mechanism and the HiLo attention optimization strategy, can effectively improve the model's ability to model complex structures and blurred boundaries in medical images, thereby obtaining more accurate segmentation results.

[0114] On the BUSI dataset, our method achieves a Dice of 81.24%, a 2.69% improvement over TransUNet (78.55%) and over 10% improvement over SwinUNet (71.18%); the IoU is improved to 68.34%, representing an improvement of approximately 2%–13% compared to mainstream Transformer architectures. On the ISIC dataset, our method achieves a Dice of 92.81%, a 1.54% improvement over TransUNet (91.27%) and a 2.08% improvement over UNetXt (90.73%); the IoU is improved to 87.04%, representing an improvement of approximately 2%–6% compared to existing models. On the CVC dataset, our method achieves a Dice of 91.51%, a 6.00% improvement over UNetXt (85.51%) and over 8% improvement over MambaUNet (83.08%); the IoU is 85.63%, representing an overall improvement of approximately 2%–14%.

[0115] As shown in Table 1, although the boundary index is not absolutely optimal in this model, it still remains at a low level overall, indicating that this method has a certain degree of stability in suppressing large-scale boundary errors.

[0116] Table 1 To verify the independent contribution of each structural module, this invention conducted systematic ablation experiments on different module combinations under the same training configuration. The results are shown in Table 2.

[0117] Table 2 After introducing KAN-MLP onto the basic TransFuse structure, Dice improved from 0.9084 to 0.9202, and IoU improved from 0.8446 to 0.8575, indicating that the KAN nonlinear mapping enhanced the feature representation capability. Further introducing KAN into the attention structure improved Dice to 0.9239 and IoU to 0.8614, demonstrating that KAN has a more significant enhancement effect in the global semantic modeling path. When both KAN-MLP and KAN-Attention are introduced simultaneously, the performance tends to stabilize, indicating some overlap in the representation space between the two improvements. Introducing MIFA or HiLo modules alone on this basis resulted in a certain degree of performance degradation. Analysis suggests that without a complete structural collaboration mechanism, cross-scale interactions or frequency decomposition may perturb the original feature distribution, thus affecting model stability. However, when MIFA and HiLo are introduced simultaneously to form a complete collaborative optimization structure, the model performance is significantly improved, with Dice reaching 0.9249 and IoU increasing to 0.8796. The significant improvement in IoU indicates that frequency decoupling optimization and cross-scale semantic fusion form a complementary enhancement mechanism, effectively improving the consistency of segmented regions and boundary accuracy. Overall, the results show that the modules are not simply superimposed, but rather form a complementary structure under a collaborative mechanism, demonstrating a clear functional division and system optimization effect.

[0118] In terms of interpretability, this invention explicitly constructs high-frequency enhancement paths and low-frequency channel recalibration paths, enabling the model structure to possess interpretable frequency approximation decomposition characteristics. High-frequency branches focus on boundary and texture enhancement, while low-frequency branches are responsible for global semantic weighting. Clinical researchers can visually observe boundary enhancement regions and semantic focus regions through channel response maps and branch feature maps. Compared to traditional black-box deep network structures, interpretability is significantly enhanced, improving clinical trust.

[0119] In terms of computational efficiency, this invention replaces the highly complex global self-attention structure with depthwise separable convolution, GroupNorm, and a lightweight channel attention mechanism. Compared to improved methods that introduce additional Transformer modules or complex frequency domain transformations, parameter growth is controlled within 5%, FLOPs increase is no more than 8%, but accuracy is significantly improved. Compared with traditional simple stitching structures, it achieves higher segmentation performance while maintaining close computational complexity, demonstrating a superior performance-to-complexity ratio.

[0120] In terms of training efficiency, the adaptive fusion weight mechanism can stabilize the gradient flow between different feature paths, improving the training convergence speed by about 10% to 15%, significantly reducing the fluctuation amplitude of the validation curve, allowing the model to enter the stable convergence range more quickly, reducing the number of invalid training rounds, and thus saving training time and computing power.

[0121] In terms of cost and deployment, the structure of this invention is entirely based on standard convolution operators and lightweight normalization structures, requiring no additional frequency domain transformation modules or complex matrix operations, making it easy to integrate into existing medical imaging systems. Due to its moderate parameter size, the model can achieve near real-time inference in mainstream GPU environments, making it suitable for hospital data centers or edge computing devices, reducing hardware upgrade requirements and maintenance costs.

[0122] In terms of environmental protection and green computing, this invention effectively controls the growth of FLOPs while ensuring improved accuracy. Compared with the improved structure that requires large-scale self-attention stacking, the overall training energy consumption is reduced by about 10% to 20%. Long-term deployment can significantly reduce carbon emissions and energy consumption, which is in line with the development trend of green medical AI.

[0123] In terms of stability and lifespan, this invention enhances model robustness through a residual enhancement mechanism and a frequency branching co-optimization structure, exhibiting stronger adaptability to changes in input resolution, device differences, and noise interference. In cross-dataset transfer tests, performance degradation is reduced by approximately 25%–35% compared to traditional structures, supporting subsequent fine-tuning and continuous learning, reducing the need for frequent retraining, and extending the model's clinical usability.

[0124] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and embodiments shown and described herein.

Claims

1. A medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization, characterized in that, Includes the following steps: S1. Obtain the two-dimensional medical image and its corresponding mask. After format cleaning, binarization, data augmentation, tensor transformation and standardization, the preprocessed image tensor and mask tensor are obtained. S2. Construct a dual-branch encoder including a convolutional branch and a KAN-Transformer branch, and input the preprocessed image tensor into the two branches respectively; the convolutional branch is constructed based on a residual network, and extracts local texture feature maps at different scales through the residual module; the KAN-Transformer branch segments the image into blocks and maps them into a sequence of feature vectors, which are then input into a multi-layer KAN-Transformer module; each KAN-Transformer module uses a KANLinear layer to replace the linear mapping layer and feedforward network layer generated from the query, key, and value in the attention mechanism, and the module output is spatially reconstructed into a two-dimensional feature map to obtain global semantic feature maps at different scales; S3. Construct a multi-scale integrated feature aggregation module, which receives local texture feature maps and global semantic feature maps with consistent spatial resolution from the dual-branch encoder. The global semantic feature map is channel-aligned by a 1×1 convolution. Depthwise separable convolutions are applied to the aligned feature maps, and ReLU and GELU activations are performed. After activation, a local gated weight map and a global gated weight map are generated by Sigmoid. The global gated weight map and the local texture feature map are multiplied pixel by pixel to obtain the modulated local features. The local gated weight map and the aligned global semantic feature map are multiplied pixel by pixel to obtain the modulated global features. After the two modulated features are added pixel by pixel, they are reconstructed by depthwise separable convolution to output a fused feature map. S4. Construct a multi-scale progressive decoder that receives fused feature maps at multiple scales, upsamples them sequentially starting from the smallest scale, concatenates them with the corresponding scale fused feature map channels, and then performs feature reconstruction through a convolutional layer to output the decoded feature map. S5. Construct a HiLo feature optimization module, which receives the decoded feature map and the corresponding scale-encoded feature map. After unifying the number of channels through a convolutional layer, it is linearly weighted and fused using learnable weights. In the channel dimension, the fused feature map is proportionally divided into high-frequency sub-branch feature maps and low-frequency sub-branch feature maps. The high-frequency sub-branch is subjected to depthwise separable convolution, group normalization, and ReLU, and then added to the input residual to output the enhanced high-frequency feature map. The low-frequency sub-branch is subjected to global average pooling, convolutional layers, and Sigmoid to generate channel weights, and then multiplied with the input channel by channel to output the enhanced low-frequency feature map. The enhanced high- and low-frequency feature maps are concatenated and then processed by convolutional layers, group normalization, and ReLU reconstruction to output the optimized feature map. S6. Map the optimized feature map to a segmentation logits map, upsample it to the original image size, and then use the Sigmoid activation function to output a probability prediction map of each pixel belonging to the target region.

2. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 1, characterized in that, In S2, the specific process by which the convolutional branch extracts local texture feature maps at different scales through the residual module is as follows: Apply 7×7 convolution with stride 2 and 3×3 max pooling with stride 2 to the preprocessed image tensor to obtain a shallow feature map with a spatial size of 1 / 4 of the spatial size of the preprocessed image tensor. The shallow feature map is sequentially input into multiple cascaded residual stages. Each residual stage consists of at least one residual block, and each residual stage is downsampled by a convolution with a stride of 2, so that the spatial size of the feature map is halved step by step and the number of channels is doubled step by step, and local texture feature maps of different spatial scales are output respectively. The residual stage includes a first residual stage, a second residual stage, and a third residual stage. The spatial dimensions of the local texture feature maps output by these stages are 1 / 4, 1 / 8, and 1 / 16 of the tensor spatial dimensions of the preprocessed image, respectively, and the number of channels are 64, 128, and 256, respectively. Each residual block performs the following operations: the first feature map x, which is the input of the current residual block, is sequentially processed through a first 3×3 convolution, batch normalization, and ReLU activation function, and then through a second 3×3 convolution and batch normalization to obtain a residual mapping feature map F(x). The residual mapping feature map F(x) is the feature representation of the input feature map after two convolution and batch normalization processes. The first feature map x is then processed through a 1×1 convolution to adjust the number of channels to match the number of channels in the residual mapping feature map F(x), resulting in a skip connection feature map W. s x, the skip connection feature map W s x is the feature representation of the input feature map after a 1×1 convolution transformation; according to the formula y = F(x, {W i })+W s x combines the residual mapping F(x) with the skip connection feature W s x is added element by element and then activated by the ReLU function to obtain the second feature map y as the output of the residual block. The second feature map y is the output feature map after the current residual block is processed. When the number of channels of the input feature map x is equal to the number of channels of the residual mapping F(x), the skip connection feature is the input feature map x itself, and no 1×1 convolution adjustment is required. The convolutional branch does not contain a global average pooling layer or a fully connected classification layer; it only outputs local texture feature maps at the above three scales for use in the subsequent multi-scale integrated feature aggregation module.

3. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 1, characterized in that, The output of the KANLinear layer in S2 is... Composition, where w b Based on activation weights, w s For spline weights; Spline(x) takes the form of Gaussian radial basis functions: α k For the basis function weights, μ k Center of basis functions, σ k denoted as the width of the basis functions, and K as the number of basis functions.

4. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 3, characterized in that, In the KAN-Transformer branch, when the spatial size of the input image is inconsistent with the preset training size, bilinear interpolation is used to apply the learnable position encoding tensor E. pos Adjustments are made to obtain a new positional encoding tensor E that matches the length of the current input feature vector sequence. pos new The bilinear interpolation calculation formula is as follows: Where H and W are the height and width of the input image.

5. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 4, characterized in that, In S2, the KAN-Transformer branch divides the preprocessed image tensor into a sequence of image blocks through a 16×16 convolution with a stride of 16 pixels, and linearly maps each image block to a feature vector to obtain an initial feature vector sequence. The dimension of the initial feature vector sequence is B×N×C, where B is the batch size, N is the number of image blocks, and C is the embedding dimension. The learnable position encoding tensor is added element by element to the initial feature vector sequence to obtain a feature vector sequence with position information. The dimension of the learnable position encoding tensor is 1×(N+1)×C, which contains a token for classification. The sequence of feature vectors with location information is sequentially input into multiple consecutive KAN-Transformer modules. Each KAN-Transformer module performs the following operations: The input feature vector sequence is subjected to layer normalization to obtain the first normalized sequence. The layer normalization calculation formula is as follows: Where μ is the mean and σ is the mean. 2 Let ε be the variance, γ be the minimum constant, and β be learnable parameters. The KANLinear layer maps the first normalized sequence into a query vector sequence Q, a key vector sequence K, and a value vector sequence V, each with a dimension of B×h×N×d, where h is the number of attention heads, ranging from 4 to 16; d is the dimension of each head, and satisfies C=h×d, where C is the embedding dimension of the Transformer branch, ranging from 256 to 768. The self-attention output is calculated based on the query vector sequence Q, the key vector sequence K, and the value vector sequence V. The calculation formula is as follows: The first attention output is obtained; The first attention output is added element by element to the input feature vector sequence of the current KAN-Transformer module to obtain the first residual output; The first residual output is layer normalized to obtain the second normalized sequence; The second normalized sequence is transformed by two consecutive KANLinear layers. The first KANLinear layer expands the feature dimension from C to rC, where r is a preset multiple, ranging from 2 to 8. The second KANLinear layer restores the feature dimension from rC to C, resulting in the feedforward output. The feedforward output is added element by element to the first residual output to obtain the output feature vector sequence of the current KAN-Transformer module; Spatial reconstruction is performed on the feature vector sequence output by the last KAN-Transformer module. The tokens used for classification are removed, and the remaining feature vector sequence is reshaped into a two-dimensional feature map to obtain a global semantic feature map aligned with the spatial scale of the convolution branch. Its dimension is B×C×(H / 16)×(W / 16), where H and W are the height and width of the preprocessed image tensor. Specifically, when the spatial size of the preprocessed image tensor is inconsistent with the preset training size, bilinear interpolation is used to adjust the learnable position encoding tensor so that the length of the adjusted position encoding tensor matches the length of the feature vector sequence generated by the current input. The bilinear interpolation calculation formula is as follows: , where (x i ,y j ) represents the coordinates of the four grid points surrounding the interpolation point, w ij These are the corresponding interpolation weights.

6. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 1, characterized in that, In step S3, the multi-scale integrated feature aggregation module receives the first feature map F from the convolutional branch. c ∈R C×H×W And the second feature map F from the KAN-Transformer branch t ∈R Ct×H×W Where C is the number of feature channels in the convolution branch, Ct is the number of feature channels in the KAN-Transformer branch, and H and W are the height and width of the feature map; The second feature map F is processed by 1×1 convolution. t Perform channel alignment, and denote the aligned second feature map as F. t '∈R C×H×W ; For the first feature map F respectively c and the aligned second feature map F t 'Applying 3×3 depthwise separable convolution, we obtain the first convolution feature and the second convolution feature. The 3×3 depthwise separable convolution includes two steps: depthwise convolution and pointwise convolution. The depthwise convolution performs 3×3 convolution independently on each input channel, and the pointwise convolution fuses the information of each channel through 1×1 convolution.' Apply the ReLU activation function to the first convolutional feature and the GELU activation function to the second convolutional feature to obtain the first activation feature and the second activation feature; The first and second activation features are mapped to the 0-1 interval using the Sigmoid function, respectively, to generate a local gate control weight map M. c and global gating weight graph M t ; The global gating weight graph M t With the first feature map F c Pixel-by-pixel multiplication yields the modulated local features. The calculation formula is: , where ⊙ represents pixel-wise multiplication; The departmental control weight map M c With the aligned second feature map F t 'Multiply pixel by pixel to obtain the modulated global features' The calculation formula is: ; Modulated local features With modulated global features By adding pixels one by one, we obtain the initial fusion feature F. sum The calculation formula is: ; Preliminary fusion feature F sum A 3×3 depthwise separable convolution is applied for feature reconstruction to obtain the final fused feature map F. fusion ∈R C×H×W The fused feature map F fusion Simultaneously includes the first feature map F c Local texture information and second feature map F t Global semantic information.

7. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 1, characterized in that, In step S4, the multi-scale progressive decoder receives a set of fused feature maps from the multi-scale integrated feature aggregation module. The set of fused feature maps includes a first fused feature map F1, a second fused feature map F2, and a third fused feature map F3. The spatial size of F1 is 1 / 4 of the tensor space size of the preprocessed image, the spatial size of F2 is 1 / 8 of the tensor space size of the preprocessed image, and the spatial size of F3 is 1 / 16 of the tensor space size of the preprocessed image. Starting with the third fused feature map F3, which has the smallest spatial size, the spatial resolution is restored level by level, and the decoding feature map D of the current level is processed sequentially. k Applying bilinear interpolation for a 2x upsampling, we obtain the upsampled feature map U. k The bilinear interpolation calculation formula is as follows: , where w ij These are weighting coefficients calculated based on the distance between four adjacent pixels; Upsampled feature map U k The fused feature map F{k-1} at the corresponding scale is concatenated along the channel dimension to obtain the concatenated feature map Concat. k The concatenated feature map Concat k The number of channels is U k The sum of the number of channels of F{k-1}; Concat the spliced ​​feature maps k Feature reconstruction is performed sequentially using 3×3 convolution, batch normalization, and ReLU activation function to obtain the output decoded feature map D{k-1} of the current layer. The calculation formula for the 3×3 convolution is as follows: , where W is the convolution kernel weight; Repeat the above upsampling, stitching and reconstruction steps to generate the second decoded feature map D2 and the first decoded feature map D1 in sequence, wherein the spatial size of the second decoded feature map D2 is 1 / 16 of the tensor space size of the preprocessed image, and the spatial size of the first decoded feature map D1 is 1 / 8 of the tensor space size of the preprocessed image. Applying bilinear interpolation to the first decoded feature map D1 again for upsampling by a factor of 2, we obtain the final decoded feature map whose spatial size is restored to 1 / 4 of the tensor spatial size of the preprocessed image. .

8. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 1, characterized in that, In step S5, the HiLo feature optimization module receives a first input feature map, a second input feature map, and a third input feature map. The first input feature map is an encoded feature map from the convolutional branch. The second input feature map is the encoded feature map from the KAN-Transformer branch. The third input feature map is a decoded feature map from a multi-scale progressive decoder. Where B is the batch size, and H and W are the feature map height and width; The first input feature map F is processed by three independent 1×1 convolutions. c Second input feature map F b2 and the third input feature map F c2 Perform channel alignment, and denote the aligned three-way feature maps as follows: , as well as Their dimensions are all uniformly B×128×H×W; The learnable fusion weight vector w is normalized using the Softmax function and then weighted and combined with a preset smoothing coefficient to obtain the smoothed fusion weight vector. The smoothed fusion weight vector The calculation formula is =0.5×Softmax(w) + 0.5 / 3, where w is a learnable parameter vector of dimension 3; Based on the smoothed fusion weight vector The three-way aligned feature maps are linearly weighted and fused to obtain the fused feature map F∈R. 128×H×W The calculation formula is: ,in , , The smoothed fusion weight vector The three components; The fused feature map F is divided along the channel dimension according to a preset scaling factor to obtain the high-frequency sub-branch feature map F. hi and low-frequency sub-branch feature map F lo The high-frequency sub-branch feature map F hi The number of channels is 64, and the low-frequency sub-branch feature map F lo The number of channels is 64, and the value of the scaling factor is 0.5; For the high-frequency sub-branch feature map F hi By sequentially applying 3×3 depthwise separable convolution, group normalization, and ReLU activation function, the enhanced high-frequency feature F is obtained. hi ', will enhance the high-frequency feature F hi 'Feature map F of high-frequency sub-branch hi Pixel-by-pixel addition outputs the enhanced high-frequency feature map F. hi out The calculation formula is F hi out = F hi '+F hi ; For the low-frequency sub-branch feature map F lo Global average pooling, 1×1 convolution, and sigmoid activation function are applied sequentially to generate channel weight vector A. Global average pooling compresses the spatial dimension to 1×1, resulting in an output dimension of B×64×1×1. The sigmoid activation function maps the output of the 1×1 convolution to the 0-1 interval, yielding channel weight vector A. Channel weight vector A is then compared with the low-frequency sub-branch feature map F. lo Channel-by-channel multiplication yields the enhanced low-frequency feature map F. lo out The calculation formula is F lo out =F lo ⊙A, where ⊙ represents channel-by-channel multiplication; The enhanced high-frequency feature map F hi out Compared with the enhanced low-frequency feature map F lo out The feature map is obtained by stitching the data along the channel dimension to restore it to 128 channels. ; splicing feature map F cat The features are reconstructed by sequentially applying 3×3 convolution, group normalization, and ReLU activation function to obtain the final optimized feature map. .

9. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 1, characterized in that, It also includes a deep supervision step during the training phase: at least one side-path output is extracted from the intermediate layer of the multi-scale progressive decoder. Each side-path output is mapped to a side-path segmentation logits map through an independent 1×1 convolutional layer, upsampled to the original image size by bilinear interpolation, and the loss function between the side-path segmentation logits map and the mask tensor is calculated. The loss function is a weighted sum of binary cross-entropy loss and Dice loss. The side-path loss and the main output loss are then combined according to... The weighted sum is used to update the network parameters, where λ i is the weighting coefficient for the i-th side path loss.

10. The medical image segmentation method based on KAN dual-branch and MIFA-HiLo collaborative optimization as described in claim 1, characterized in that, After outputting the probability prediction map of each pixel belonging to the target region, the post-processing step is included: obtaining a segmentation threshold τ through adaptive search of the validation set, setting pixels in the probability prediction map greater than the threshold to 1, and the rest to 0, generating a binary segmentation mask M that satisfies... , where M(x,y) is the pixel value of the probability prediction map at position (x,y).

Images