A frequency-adaptive based image segmentation interpretation method and system
Patent Information
- Application Number
- CN202610702637.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-21
- Publication Date
- 2026-08-21
AI Technical Summary
本发明针对KAN在图像分割中高频响应不平衡的问题，通过自适应低频选择与动态频带调制双机制协同，实现边界细节增强与内部噪声抑制的智能权衡，从而提升KAN在图像分割任务中的性能与稳定性
[0036] Compared with the prior art, the present invention has the following advantages:
Smart Images
Figure CN122617899A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an image segmentation and interpretation method and system, specifically to a frequency-adaptive image segmentation and interpretation method and system, belonging to the field of computer image processing technology. Background Technology
[0002] Image segmentation is a core task in the field of computer vision, aiming to assign each pixel in an image to a specific semantic category. It is of great significance for practical applications such as autonomous driving, medical image analysis, and remote sensing interpretation.
[0003] The Kolmogorov-Arnold network (KAN) is a novel neural network based on the Kolmogorov-Arnold representation theorem. It replaces the fixed weight matrix in traditional multilayer perceptrons with a learnable activation function, exhibiting stronger functional expressiveness while maintaining parameter efficiency. In image segmentation tasks, integrating KAN into the feedforward network of a visual Transformer has shown superior performance compared to traditional architectures. However, while KAN's activation mechanism based on nonlinear basis functions such as B-splines enhances the model's expressiveness, it also induces a significant spectral regeneration effect, leading to the over-amplification of high-frequency components in the features. While this characteristic helps enhance the detail information of object boundaries, it also amplifies high-frequency noise in originally smooth regions within objects, disrupting intra-class consistency and producing spurious responses and internal artifacts in the segmentation results.
[0004] Frequency domain analysis, as an important tool for understanding and optimizing the representational characteristics of visual models, has been widely applied in convolutional neural networks and Transformer architectures, for example, by optimizing frequency domain features to improve the robustness and segmentation performance of models. However, research on the frequency domain characteristics of KAN networks and their high-frequency response imbalance problem in image segmentation tasks is still lacking. Existing KAN improvements mainly focus on improving computational efficiency and expanding expressive power, but have not systematically addressed the problem of the mixture of high-frequency signals at the boundaries and high-frequency noise inside from a frequency domain perspective. Therefore, how to achieve adaptive differentiation and processing of beneficial high-frequency signals at the boundaries and harmful high-frequency noise inside while retaining KAN's powerful nonlinear modeling capabilities has become a key challenge in improving KAN's performance in image segmentation tasks. Traditional frequency domain methods usually employ global filtering strategies, which cannot adaptively adjust according to image content and structural characteristics, making it difficult to meet the segmentation accuracy requirements in complex scenes.
[0005] In summary, developing an image segmentation method that can intelligently adjust the KAN response at the frequency domain, enhance boundary details, and suppress internal noise is of great value for improving the performance of KAN in practical segmentation applications and is also a key issue that urgently needs to be addressed in the current development of image segmentation technology. Summary of the Invention
[0006] The purpose of this invention is to address the shortcomings and deficiencies of existing technologies by creatively proposing a frequency-adaptive image segmentation and interpretation method. This invention addresses the problem of unbalanced high-frequency response in KAN during image segmentation by employing a dual mechanism of adaptive low-frequency selection and dynamic band modulation to achieve an intelligent trade-off between boundary detail enhancement and internal noise suppression, thereby improving the performance and stability of KAN in image segmentation tasks.
[0007] This invention utilizes an adaptive low-pass selection module to dynamically fuse the KAN output with its low-frequency filtered version through pixel-level weighting, thereby preserving high-frequency information at key boundaries. Simultaneously, a dynamic frequency band modulation module is introduced to perform band-weighted calibration of the feature spectrum in the frequency domain, coarsely suppressing internal high-frequency noise and providing a cleaner high-frequency input for subsequent selection. The two modules work in cascade to jointly optimize the proportion of high-frequency boundaries, enhancing intra-class consistency while maintaining inter-class discriminability. Experimental results show that this method achieves significant performance improvements on multiple image segmentation datasets.
[0008] To achieve the above objectives, the present invention employs the following technical solutions.
[0009] A frequency-adaptive image segmentation and interpretation method includes the following steps:
[0010] Step 1: Perform block embedding on the input image.
[0011] Specifically, the input image is processed using a convolutional kernel with a kernel size of 16×16 and a stride of 16 to obtain a block embedding feature map with a length and width that are both one-sixteenth the size of the original image.
[0012] Step 2: Extract multi-level deep features from the block embedding feature map.
[0013] Specifically, twelve encoder layers are used to process block embedding features, with the output of each layer being the same as the input features. The outputs of the third, sixth, ninth, and twelfth layers are taken as the multi-level deep feature outputs for this stage.
[0014] Specifically, each encoder layer contains an attention module, a frequency-adaptive KAN, and two layer normalization modules.
[0015] Specifically, the attention module receives the feature map output from the previous layer as input. The input features first enter a multi-head self-attention computation unit. In this unit, the feature map undergoes three independent linear transformations to obtain a query feature map, a key feature map, and a value feature map. Attention weights are obtained by calculating the similarity between the query and the key, and these weights are then weighted and fused with the value feature map to obtain the attention-enhanced feature representation. Afterward, feature normalization is performed through a layer normalization module. Subsequently, this module uses a residual connection structure to add the attention output to the original input features element-wise, obtaining the output of the entire attention module.
[0016] Specifically, the frequency-adaptive KAN receives the output of the attention module as input. The input features are processed sequentially through the KAN module, the dynamic band modulation module, and the adaptive low-pass selection module. Finally, layer normalization is applied, and the output is added to the input features to obtain the output of the frequency-adaptive KAN.
[0017] Specifically, the KAN module consists of two basic KAN layers. Each basic KAN layer contains a learnable activation matrix and a random deactivation layer. The learnable activation matrix is an activation function matrix, where each element is a learnable activation function. This matrix operates on features similarly to linear matrix multiplication. After processing by this layer, regularization is performed using the random deactivation layer. The first basic KAN layer increases the dimensionality of the features, and the second basic KAN layer restores the dimensionality. The output of the KAN module is obtained after processing through these two basic KAN layers.
[0018] Specifically, the dynamic band modulation module includes four main operations: frequency domain transformation, radial spectral pooling, gated network generation, and residual connection. The frequency domain transformation operation converts spatial features into a frequency domain representation using a Fast Fourier Transform (FFT). The radial spectral pooling operation divides the spectrum into multiple frequency bands based on radial distance and calculates the average energy of each band. The gated network generation operation uses a one-dimensional convolutional network to analyze the energy distribution of the frequency bands and generate adaptive weights for each band. The residual connection operation transforms the modulated frequency domain features back to the spatial domain and fuses them with the original input features using a learnable scaling factor. After these operations, the module can adaptively adjust the response intensity of different frequency components according to the content characteristics of the input features, thus obtaining the output of the dynamic band modulation module.
[0019] Specifically, the adaptive low-pass selection module includes three main operations: low-pass filtering, high-frequency difference calculation, and selection map generation. The low-pass filtering operation extracts stable low-frequency components from the original input features using average pooling. The high-frequency difference calculation operation obtains a high-frequency difference map as a guiding signal by calculating the absolute difference between the KAN output and the low-frequency components. The selection map generation operation concatenates the original input features with the high-frequency difference map, generates pixel-level selection weights through a convolutional layer, and then performs Softmax normalization. Finally, the module adaptively fuses the KAN output processed by the dynamic frequency band modulation module with the low-frequency components using a weighted fusion mechanism. The output of the adaptive low-pass selection module is obtained after these operations.
[0020] Step 3: Fuse and share multi-level deep features.
[0021] Specifically, after the feature extraction operation in the previous step, four levels of deep features were obtained, all with the same size but containing different semantic information. A multi-level feature pyramid structure was constructed to process the features at different levels separately. Specifically, the deepest features were upsampled by a factor of 4 using two layers of 2×2 transposed convolutions; the next deepest features were upsampled by a factor of 2 using a single layer of 2×2 transposed convolutions; the intermediate features maintained their identity mapping; and the shallowest features were downsampled by a factor of 2 using 2×2 max pooling. Finally, the adjusted multi-level features were merged into a unified multi-scale representation, achieving the fusion and sharing of features at different semantic levels.
[0022] Step 4: Multi-scale feature pyramid fusion.
[0023] Specifically, the multi-scale feature pyramid fusion comprises three parts: a pyramid pooling module, a feature pyramid network structure, and feature refinement processing. The pyramid pooling module takes the deepest layer features as input and outputs the fused features after multi-scale pooling. The feature pyramid network structure takes the original features from four levels and the output of the pyramid pooling module as input, and achieves feature fusion between levels through a top-down path. The feature refinement processing takes the aforementioned fusion result as input and outputs a refined multi-level pyramid feature map.
[0024] Specifically, the pyramid pooling module comprises four parallel pooling branches and a feature fusion bottleneck layer. The four pooling branches perform average pooling operations using 1×1, 2×2, 3×3, and 6×6 pooling kernels, respectively. The output of each branch is uniformly adjusted to 128 channels through an independent 1×1 convolutional layer and upsampled to the size of the original deep features using bilinear interpolation. The feature fusion bottleneck layer is a 3×3 convolutional layer used to compress the concatenated features (original deep features and the outputs of the four pooling branches) to 128 channels, forming the pyramid pooled features.
[0025] Specifically, the feature pyramid network structure includes a lateral connection processing layer and a top-down fusion path. The lateral connection processing layer consists of three independent 1×1 convolutional layers, which process the first three layers of input features, adjusting their channel count to a uniform 128 channels. The top-down fusion path starts from the deepest layer, upsampling the current layer's features to the previous layer's feature size using bilinear interpolation, and then adding it element-wise with the lateral connection output of that layer, sequentially completing the feature fusion from the third layer to the first layer.
[0026] Specifically, the feature refinement process consists of three independent 3×3 convolutional layers. Each convolutional layer corresponds to a fused hierarchical feature, and the feature representation is further extracted and enhanced through convolutional operations to generate the final pyramid feature map.
[0027] Step 5: Feature integration and classification output.
[0028] Specifically, the feature integration and classification output consists of two parts: a multi-level feature integration operation and a segmentation prediction operation. The multi-level feature integration operation takes the multi-level pyramid feature map generated by the feature pyramid network as input and outputs a fused feature with uniform size and channels; the segmentation prediction operation takes this fused feature as input and outputs the final segmentation prediction heatmap.
[0029] Specifically, the multi-level feature integration operation includes a size unification sub-operation, a feature concatenation sub-operation, and a final fusion sub-operation. The size unification sub-operation uses bilinear interpolation to upsample the pyramid features across all levels to the highest resolution (one-quarter of the input image size). The feature concatenation sub-operation connects the 128-channel features from the four levels into a 512-channel feature map. The final fusion sub-operation integrates the 512-channel features into a unified 128-channel representation through a 3×3 convolutional bottleneck layer.
[0030] Specifically, the segmentation prediction operation includes a pixel-level classification process and a probability mapping process. The pixel-level classification process is implemented through a 1×1 convolutional layer, mapping 128-channel features to a channel space corresponding to the number of categories. The probability mapping process transforms the convolutional output into a probability distribution for each category at each spatial location, generating the final segmentation prediction heatmap.
[0031] Furthermore, to achieve the objectives of this invention, based on the above method, this invention also proposes a frequency-adaptive image segmentation and interpretation system, including a frequency-adaptive feature extraction module, a multi-level feature fusion module, and a segmentation head module.
[0032] Among them, the frequency-adaptive feature extraction module is used to extract features with high intra-class consistency and strong inter-class discriminativeness from the image in a frequency-adaptive manner.
[0033] The multi-level feature fusion module is used to achieve information fusion and sharing of multi-level features;
[0034] The segmentation head module is used to form the final segmentation boundaries and region classification.
[0035] Beneficial effects
[0036] Compared with the prior art, the present invention has the following advantages:
[0037] 1. This invention solves the high-frequency imbalance problem of KAN network in image segmentation through a frequency adaptive mechanism, which significantly improves segmentation accuracy while maintaining parameter efficiency.
[0038] 2. The adaptive low-pass selection module of the present invention can dynamically fuse high and low frequency features, specifically suppress internal noise and preserve boundary details, thereby improving the quality of segmentation boundaries.
[0039] 3. The dynamic frequency band modulation module of the present invention can adaptively calibrate the characteristic spectrum in the frequency domain, enhance the useful frequency band and suppress noise, and improve the robustness of the model in complex scenarios. Attached Figure Description
[0040] Figure 1 This is a schematic flowchart of the method of the present invention;
[0041] Figure 2 This is a schematic diagram of the neural network structure used in the method of the present invention;
[0042] Figure 3 This is a schematic diagram of the system composition of the present invention.
[0043] Figure 4 This is a general overview of the process of this invention. Detailed Implementation
[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.
[0045] like Figure 1 As shown, an image segmentation and interpretation method based on frequency-adaptive KAN includes the following steps:
[0046] Step 1: Perform block embedding on the input image.
[0047] Furthermore, the input image is processed using a convolutional kernel with a size of 16×16 and a stride of 16, resulting in a block embedding feature map with dimensions one-sixteenth that of the original image. The input image has 3 channels, and the output block embedding feature map has 768 channels.
[0048] Step 2: Extract multi-level deep features from the block embedding feature map.
[0049] Furthermore, twelve encoder layers are used to process the block embedding features, with the output of each layer having the same width, height, and number of channels as the input features. The outputs of the third, sixth, ninth, and twelfth layers are taken as the multi-level deep feature outputs for this stage.
[0050] Furthermore, each encoder layer contains an attention module, a frequency-adaptive KAN, and two layer normalization modules.
[0051] Furthermore, the attention module receives the feature map output from the previous layer as input. The input features first enter the multi-head self-attention computation unit to obtain the query feature map, key feature map, value feature map, and attention weights. The weights and value feature maps are weighted and fused to obtain the attention-enhanced feature representation. Then, the features are normalized through a layer normalization module. Finally, the attention output is added element-wise to the original input features to obtain the output of the entire attention module.
[0052] Specifically, let the input feature map be... Output feature map for:
[0053]
[0054]
[0055] in To query feature maps, Key feature map, For value feature map, For query weight, For key weights, As value weight, This represents the number of channels in the feature map.
[0056] like Figure 2 As shown in (a), the frequency-adaptive KAN receives the output of the attention module as input. The input features are processed sequentially through the KAN module, the dynamic band modulation module, and the adaptive low-pass selection module. Finally, layer normalization is applied, and the output is added to the input features to obtain the output of the frequency-adaptive KAN.
[0057] like Figure 2 As shown in (b), the KAN module contains two basic KAN layers. Let the input features of the basic KAN layers be... Its output characteristics for:
[0058]
[0059]
[0060]
[0061] in It is the intermediate matrix. for No. Line 1 Column elements; for No. Line 1 Column elements; The first learnable activation matrix Line 1 The elements of the column.
[0062] like Figure 2 As shown in (c), for the dynamic band modulation module, let its input characteristics be... Its output characteristics for:
[0063]
[0064]
[0065]
[0066] in For the first A set of spectral values for each frequency band. and These represent Fourier transform and inverse Fourier transform, respectively; Decom is frequency band decomposition; Average is frequency band mean; and FFN is a feedforward neural network. For the first Modulation weights for each frequency band.
[0067] like Figure 2 As shown in (d), the adaptive low-pass selection module includes three main operations: low-pass filtering, high-frequency difference calculation, and selection map generation. Specifically, let the input features from the attention module be... The source of dynamic frequency band modulation is Then output features for:
[0068]
[0069]
[0070]
[0071]
[0072]
[0073] in Indicates low-frequency input. For average pooling, High-frequency difference characteristics, For connection operations, Features after connection This indicates the selection weight for low-frequency inputs. This represents the selection weight for high-frequency inputs. This indicates element-wise multiplication.
[0074] Step 3: Perform multi-scale expansion on multi-level deep features.
[0075] Specifically, after the feature extraction operation in the previous step, four levels of deep features were obtained, all with the same size but containing different semantic information. A multi-level feature pyramid structure was constructed to process the features at different levels separately. Specifically, the deepest features were upsampled by a factor of 4 using two layers of 2×2 transposed convolutions; the next deepest features were upsampled by a factor of 2 using a single layer of 2×2 transposed convolutions; the intermediate features maintained their identity mapping; and the shallowest features were downsampled by a factor of 2 using 2×2 max pooling. Finally, the adjusted multi-level features were merged into a unified multi-scale representation.
[0076] Step 4: Multi-scale feature pyramid fusion.
[0077] Specifically, the multi-scale feature pyramid fusion comprises three parts: a pyramid pooling module, a feature pyramid network structure, and feature refinement processing. The pyramid pooling module takes the deepest layer features as input and outputs the fused features after multi-scale pooling. The feature pyramid network structure takes the original features from four levels and the output of the pyramid pooling module as input, and achieves feature fusion between levels through a top-down path. The feature refinement processing takes the aforementioned fusion result as input and outputs a refined multi-level pyramid feature map.
[0078] Specifically, the pyramid pooling module comprises four parallel pooling branches and a feature fusion bottleneck layer. The four pooling branches perform average pooling operations using 1×1, 2×2, 3×3, and 6×6 pooling kernels, respectively. The output of each branch is uniformly adjusted to 128 channels through an independent 1×1 convolutional layer and upsampled to the size of the original deep features using bilinear interpolation. The feature fusion bottleneck layer is a 3×3 convolutional layer used to compress the concatenated features (original deep features and the outputs of the four pooling branches) to 128 channels, forming the pyramid pooled features.
[0079] Specifically, the feature pyramid network structure includes a lateral connection processing layer and a top-down fusion path. The lateral connection processing layer consists of three independent 1×1 convolutional layers, which process the first three layers of input features, adjusting their channel count to a uniform 128 channels. The top-down fusion path starts from the deepest layer, upsampling the current layer's features to the previous layer's feature size using bilinear interpolation, and then adding it element-wise with the lateral connection output of that layer, sequentially completing the feature fusion from the third layer to the first layer.
[0080] Specifically, the feature refinement process consists of three independent 3×3 convolutional layers. Each convolutional layer corresponds to a fused hierarchical feature, and the feature representation is further extracted and enhanced through convolutional operations to generate the final pyramid feature map.
[0081] Step 5: Feature integration and classification output.
[0082] Specifically, the feature integration and classification output consists of two parts: a multi-level feature integration operation and a segmentation prediction operation. The multi-level feature integration operation takes the multi-level pyramid feature map generated by the feature pyramid network as input and outputs a fused feature with uniform size and channels; the segmentation prediction operation takes this fused feature as input and outputs the final segmentation prediction heatmap.
[0083] Specifically, the multi-level feature integration operation includes a size unification sub-operation, a feature concatenation sub-operation, and a final fusion sub-operation. The size unification sub-operation uses bilinear interpolation to upsample the pyramid features across all levels to the highest resolution (one-quarter of the input image size). The feature concatenation sub-operation connects the 128-channel features from the four levels into a 512-channel feature map. The final fusion sub-operation integrates the 512-channel features into a unified 128-channel representation through a 3×3 convolutional bottleneck layer.
[0084] Specifically, the segmentation prediction operation includes a pixel-level classification process and a probability mapping process. The pixel-level classification process is implemented through a 1×1 convolutional layer, mapping the 128-channel features to a channel space corresponding to the number of classes. The probability mapping process transforms the convolutional output into a probability distribution for each class at each spatial location, generating the final segmentation prediction heatmap.
[0085] like Figure 3 As shown, an image segmentation and interpretation system based on frequency adaptive KAN includes a frequency adaptive feature extraction module M1, a multi-level feature fusion module M2, and a segmentation head module M3.
[0086] The frequency-adaptive feature extraction module M1 is used to process the input image to obtain multi-level deep features;
[0087] The multi-level feature fusion module M2 is used to fuse and share information from multi-level deep features;
[0088] The segmentation head module M3 uses the fused multi-level features to output the segmentation boundaries and region categories.
[0089] The connection relationships between the above modules are as follows:
[0090] The input is connected to the input terminal of the frequency adaptive feature extraction module M1;
[0091] The output of the frequency adaptive feature extraction module M1 is connected to the input of the multi-level feature fusion module M2;
[0092] The output of the multi-level feature fusion module M2 is connected to the input of the segmentation head module M3;
[0093] The output of the segmentation head module M3 is connected to the output detection result.
[0094] This invention discloses an image segmentation and interpretation method and system based on frequency adaptive KAN. It achieves high-precision semantic segmentation of KAN network in complex scenes through frequency domain adaptive adjustment mechanism. It uses adaptive low-pass selection module and dynamic frequency band modulation module to jointly optimize feature spectrum distribution, which effectively suppresses internal noise of objects while enhancing high-frequency details at the boundary, thereby improving intra-class consistency and inter-class separability of segmentation results and achieving high-precision and robust image segmentation and interpretation effect.
Claims
1. A frequency-adaptive image segmentation and interpretation method, characterized in that, Includes the following steps: Step 1: Perform block embedding processing on the input image to extract the initial feature map; Step 2: Extract multi-level deep features from the block embedding feature map. Each encoder layer contains an attention module and a frequency adaptive KAN module, wherein the frequency adaptive KAN module includes a KAN sub-module, a dynamic frequency band modulation sub-module, and an adaptive low-pass selection sub-module in sequence. Step 3: Perform multi-scale expansion and fusion of multi-level deep features to construct a feature pyramid structure; Step 4: Perform pyramid fusion on the multi-scale features to generate a refined multi-level pyramid feature map; Step 5: Integrate and classify the multi-level pyramid feature maps, and output the final segmentation prediction results.
2. The method as described in claim 1, characterized in that, The block embedding process in step 1 specifically involves using a convolution kernel with a kernel size of 16×16 and a stride of 16 to perform a convolution operation on the input image, outputting a block embedding feature map whose length and width are both one-sixteenth of the original image.
3. The method as described in claim 1, characterized in that, The frequency adaptive KAN module in step 2 specifically includes: KAN submodule: consists of two basic KAN layers, each containing a learnable activation matrix and a random deactivation layer, used to extract non-linear features; Dynamic frequency band modulation submodule: Adaptively adjusts the response intensity of different frequency bands in the characteristic spectrum through frequency domain transformation, radial spectrum pooling, gated network generation and residual connection operations; Adaptive Low-Pass Selection Submodule: Through low-pass filtering, high-frequency difference calculation and selection map generation operations, it dynamically fuses low-frequency components with KAN output, preserves boundary high-frequency details and suppresses internal noise.
4. The method as described in claim 3, characterized in that, The frequency band weight generation process of the dynamic frequency band modulation submodule is as follows: in For the first A set of spectral values for each frequency band. and These represent Fourier transform and inverse Fourier transform, respectively; Decom is frequency band decomposition; Average is frequency band mean; and FFN is a feedforward neural network. For the first Modulation weights for each frequency band.
5. The method as described in claim 3, characterized in that, The output feature calculation method of the adaptive low-pass selection submodule is as follows: in, Low-frequency components, This is a high-frequency difference graph. and Weights are selected for low frequency and high frequency, respectively.
6. The method as described in claim 1, characterized in that, The multi-scale expansion in step 3 specifically includes: upsampling the deepest feature by 4 times, upsampling the second deepest feature by 2 times, maintaining the identity mapping for intermediate features, downsampling the shallowest feature by 2 times, and merging the processed features into a multi-scale representation.
7. The method as described in claim 1, characterized in that, The pyramid fusion in step 4 includes a pyramid pooling module, a feature pyramid network, and feature refinement processing, wherein the pyramid pooling module uses a multi-scale average pooling branch to perform multi-scale fusion of deep features.
8. The method as described in claim 1, characterized in that, The feature integration and classification in step 5 includes: upsampling the multi-level pyramid features to a uniform size and stitching them together, integrating them through a convolutional layer, and then using a 1×1 convolution to perform pixel-level classification to generate a segmentation prediction heatmap.
9. A frequency-adaptive image segmentation and interpretation system, characterized in that, include: A frequency-adaptive feature extraction module is used to perform steps 1 and 2 of the method according to any one of claims 1 to 8; A multi-level feature fusion module is used to execute steps 3 and 4 of the method; The segmentation head module is used to execute step 5 of the method and output the final segmentation result.