Lightweight image segmentation method based on frequency-aware multi-level detail condensation

Abstract

The lightweight image segmentation method based on frequency-aware multi-level detail condensation belongs to the technical field of computer vision and image processing. The method decouples the input image in the frequency domain, decomposes the image features into low-frequency global information and high-frequency local detail information, and models them respectively. In the encoding stage, low-frequency convolution blocks and high-frequency Mamba blocks are designed to compensate for global semantic information and detail feature information respectively. In the decoding stage, a multi-level detail condensation structure is constructed, cross-layer feature interaction is realized through a context detail enhancement module, and the multi-scale semantic expression ability is further enhanced through a multi-scale feature extraction module. The experimental results show that the invention has better segmentation performance than existing methods on multiple public datasets, while maintaining a low model parameter quantity and computational complexity, and has good engineering application value, and is suitable for underwater target detection, automatic driving environment perception and industrial vision detection scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to a lightweight image segmentation method based on frequency-aware multi-level detail condensation. Background Technology

[0002] With the rapid development of computer vision technology, semantic segmentation, as an important fundamental task, plays a crucial role in fields such as autonomous driving, underwater inspection, remote sensing analysis, and industrial inspection. The goal of semantic segmentation is to predict the category of each pixel in an image, thereby achieving a detailed understanding of the scene. In practical applications, especially in underwater robot inspection or edge device deployment scenarios, models not only need high segmentation accuracy but also low computational complexity and fast inference speed. However, existing methods often struggle to achieve a good balance among these multiple metrics. Traditional convolutional neural networks (CNNs) have significant advantages in local feature extraction, but due to their limited receptive field, they struggle to effectively capture long-range dependencies. To address this issue, researchers have introduced the Transformer architecture, which uses a self-attention mechanism to model global information. However, Transformer models typically have high computational complexity, making them difficult to apply directly to resource-constrained devices.

[0003] In recent years, the Mamba architecture based on the state-space model has been able to model long-range dependencies with linear complexity, alleviating the computational overhead problem to some extent. However, existing methods often neglect local high-frequency details when using Mamba for global modeling, thus affecting the accuracy of the segmentation results. On the other hand, from a frequency domain perspective, images can be decomposed into low-frequency and high-frequency components. The low-frequency components mainly contain global semantic information, while the high-frequency components contain details such as edges and textures. In deep networks, low-frequency information tends to gradually decay, while high-frequency information is easily ignored during global modeling, leading to incomplete feature representation. Therefore, how to effectively integrate low-frequency global information and high-frequency local details under the premise of lightweight design has become a key problem that urgently needs to be solved in the field of image segmentation. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight image segmentation method (LFNet) based on frequency-aware multi-level detail condensation, in order to solve the problems of insufficient feature representation ability, loss of detail information, and difficulty in balancing model efficiency and performance in the existing technology.

[0005] To achieve the above objectives, this invention proposes an image segmentation method that combines a frequency domain decoupling mechanism with a multi-level feature fusion strategy. This method decomposes the image into frequencies, modeling low-frequency and high-frequency information separately, thereby achieving coordinated optimization of global semantics and local details. In the encoding stage, low-frequency convolutional blocks and high-frequency Mamba blocks are designed to effectively utilize frequency domain information. In the decoding stage, a multi-level detail condensation mechanism is introduced to enhance the interaction between features at different levels, and multi-scale feature fusion further improves the model's expressive power. Specifically, the method includes the following steps: (1) Input image preprocessing: The RGB input image to be segmented is obtained, and initial feature extraction is performed through the Stem module. The Stem module consists of convolutional layers, normalization layers, and activation functions, which are used to perform shallow feature encoding on the input image, extract basic texture and edge information, and obtain an initial feature map for subsequent encoding processing.

[0006] (2) Constructing a frequency-aware encoder: An encoder consisting of multiple encoding stages is constructed. Each encoding stage includes a downsampling module, several low-frequency convolutional blocks (LFCBs) and at least one high-frequency Mamba block (HFMB) for frequency decoupling modeling and extraction of multi-scale features.

[0007] The encoder adopts a step-by-step downsampling structure, which reduces spatial resolution and increases the number of channels to achieve feature compression and semantic enhancement. In each encoding stage, low-frequency convolutional blocks (LFCB) and high-frequency Mamba blocks (HFMB) are introduced to decouple and model the low-frequency and high-frequency information in the input features, thereby improving the feature representation capability with lower computational cost and realizing the collaborative modeling of local and global features.

[0008] (3) Frequency-aware coding: In each coding stage, downsampling, low-frequency information modeling, and high-frequency information modeling are performed sequentially, wherein: (3.1) Downsampling: The downsampling module compresses its input features by reducing spatial resolution and increasing the number of channels; (3.2) Low-frequency information modeling: The low-frequency convolutional block LFCB maps its input features to the frequency domain through fast Fourier transform, extracts low-frequency components using a low-frequency filter, restores them to the spatial domain through inverse transform, and fuses them with its input features; the fusion result is further enhanced by convolution. (3.3) High-frequency information modeling: The high-frequency Mamba block divides its input features in the channel dimension into two branches. One branch extracts high-frequency details through Fourier transform and high-frequency filtering, and then enhances them through convolution. The other branch models long-range dependencies by linearly mapping the input state space model. The output results of the two branches are merged in the channel dimension to obtain multi-scale features, thereby realizing the collaborative modeling of high-frequency details and global information.

[0009] In the low-frequency information modeling process, low-frequency components are extracted through low-frequency filters to preserve the overall structural and semantic information in the image. Then, the image is restored to the spatial domain through inverse transformation and fused with the original features to enhance the global expressive power of the features. The fusion result is further used for feature extraction through convolution operations, thereby compensating for the shortcomings of traditional convolutional networks in global information modeling while ensuring computational efficiency.

[0010] In the process of high-frequency information modeling, one branch extracts high-frequency detail information through Fourier transform and high-frequency filtering, and enhances edge and texture features through convolution operation; the other branch performs sequence modeling by inputting features into the state space model through linear mapping, so as to capture long-range dependencies and enhance the expression of global contextual information; finally, feature fusion is used to achieve collaborative modeling of high-frequency detail information and global semantic information, thereby improving the model's ability to express complex scenes.

[0011] (4) Constructing a multi-level detail condensation decoder: The decoder upsamples the multi-scale features output from the higher-order encoding stage and then performs cross-layer fusion through the Contextual Detail Enhancement (CDRB) module to restore spatial resolution and enhance feature representation, thereby enabling information interaction between features at different levels.

[0012] In the above process, through step-by-step upsampling and cross-layer connection structure, the multi-scale features output in the encoding stage are fused and reconstructed layer by layer, thereby gradually restoring the spatial resolution and realizing information interaction between features at different levels, thus improving the integrity and consistency of feature representation.

[0013] (5) Multi-layered detailed decoding: (5.1) Feature extraction and upsampling: Extract content information, background information and edge information from the output of the higher-order encoding stage to enhance feature discrimination ability and improve segmentation boundary accuracy, and then upsample to the size of the output of the previous encoding stage; (5.2) Contextual detail enhancement: The Contextual Detail Enhancement (CDRB) module is used to fuse the upsampled result with the output of the previous encoding stage; (5.3) Multi-scale feature extraction module: After fusion, the multi-scale feature extraction module MSFEB is introduced. It models different receptive field features through multi-branch parallel convolution and fuses each branch to obtain multi-scale semantic representation, so as to improve the adaptability to targets of different sizes.

[0014] In the above process, the decoder extracts content, background, and edge information from the output of the high-level encoder and uses this information as guiding information to fuse with adjacent low-level features, thereby achieving effective correlation between features at different levels. This approach enhances the discriminative power of features and significantly improves the fine-grained representation of segmentation results in boundary regions. After feature fusion, a multi-scale feature extraction module is introduced. This module models different receptive fields for features using a multi-branch parallel convolutional structure and fuses the features from each branch to obtain rich multi-scale semantic representations. This module effectively improves the model's adaptability to targets at different scales and enhances the hierarchical representation of features.

[0015] (6) Output segmentation results: The multi-scale semantic representation obtained after fusion is restored to the input size by upsampling, and pixel-level segmentation results are generated by convolution.

[0016] (7) Model deployment: The model is trained through steps (1)-(6). After training, the image to be processed is input and the above steps are performed to obtain the segmentation result.

[0017] During training, the model parameters are updated by optimizing the loss function.

[0018] The Stem module is used to extract initial features from the input image. It includes two consecutive convolutional layers, each followed by a batch normalization layer and a non-linear activation function to generate an initial feature map for subsequent encoding processing. The convolution operation achieves layer-by-layer feature mapping of the input image to extract low-level texture and edge information. At the same time, the batch normalization layer is used to adjust the feature distribution, and the non-linear activation function is used to enhance the feature representation capability, so that the generated initial features have a stable and effective representation basis for further processing by the frequency-aware encoder.

[0019] To address the problem that existing methods struggle to balance global structural information with local detail information during feature representation modeling of global semantic information and local frequency information, this invention constructs a lightweight frequency-aware encoder. By performing path-by-path modeling on information from different frequency bands, it achieves the collaborative expression of global and detail information with lower computational complexity.

[0020] The frequency-aware encoder adopts a hierarchical cascaded structure and is designed with three encoding stages. Each encoding stage first achieves downsampling and expands the number of channels through convolution with a stride of 2. The low-frequency convolutional block (LFCB) includes a frequency domain transform unit and a spatial convolutional unit. It maps the input features to the frequency domain through a fast Fourier transform and performs low-frequency filtering through a central mask. The filtering result is restored to the spatial domain through an inverse transform and then added to the input features element by element. The spatial convolutional unit adopts a multi-branch structure and extracts multi-scale features through depth convolution and dilated convolution of different sizes to supplement feature information. The low-frequency convolutional block (LFCB) includes a frequency domain processing unit and a spatial feature extraction unit. It transforms the input features to the frequency domain using a fast Fourier transform and performs low-frequency filtering through a central mask. The filtering result is restored to the spatial domain through an inverse transform and fused element-wise with its original input features. The fused features are further input into the spatial convolutional unit (spatial feature extraction unit). This unit adopts a multi-branch structure and processes the features in parallel through different convolution scales and dilatations to supplement feature information, thereby achieving the supplementation and reconstruction of multi-scale information.

[0021] The High Frequency Mamba Block (HFMB) includes a frequency domain detail enhancement branch and a sequence modeling branch. The frequency domain detail enhancement branch obtains high-frequency features through Fourier transform combined with high-frequency filtering and enhances the detail information through convolution operations. The sequence modeling branch converts its input features (two-dimensional features) into a sequence form through linear mapping and introduces a state-space model to dynamically model the features in order to capture long-distance dependencies. The outputs of the two branches are fused in the channel dimension and integrated with the feedforward structure through normalization processing. The final output is obtained by combining residual connections.

[0022] To address the problem of insufficient utilization of detailed information in the multi-scale feature fusion process of existing methods, this invention designs a lightweight multi-level detail condensation decoder. It adopts a top-down feature reconstruction path, restores resolution step by step through upsampling and cross-layer connections, and achieves feature fusion through element-wise addition or channel splicing.

[0023] Step (4) The Contextual Details Enhancement (CDRB) module maps high-level features into a convolutional map to obtain a response feature map. This response map is used to perform element-wise weighting on low-level features. At the same time, edge information is combined and fused by element-wise addition or channel concatenation to enhance the expression of boundary and detail information. The Multi-Scale Feature Extraction (MSFEB) module includes multiple parallel convolutional branches. Each branch extracts features through dilated convolutions with different convolutional scales or different dilation rates. The outputs of each branch are aggregated in the channel dimension. Then, feature integration is completed through 1×1 convolution to obtain multi-scale features with unified expression.

[0024] The model optimization process in step (7) uses the cross-entropy loss function to train the model. The cross-entropy loss function is a pixel-level classification loss function used to measure the difference between the model's predicted class probability distribution and the true label distribution. The closer the prediction result is to the true label, the smaller the loss value. During the training process, stochastic gradient descent or adaptive optimization algorithms are used to iteratively update the model parameters, and a learning rate decay strategy is combined to improve convergence stability. The application of the method is characterized by its application in industrial defect detection, autonomous driving target recognition, and medical lesion segmentation.

[0025] Compared with the prior art, the present invention has the following significant advantages: This invention proposes a lightweight frequency-aware semantic segmentation method based on the collaborative modeling of image frequency information and multi-level detail information. By introducing low-frequency convolutional blocks and high-frequency Mamba blocks during the encoding stage, the input features are decoupled in the frequency domain, respectively characterizing global structural information and local texture detail information in the image. This avoids the problems of insufficient global semantic expression and missing high-frequency detail information caused by traditional convolutional networks modeling in a single spatial domain, thus achieving more comprehensive feature representation with lower computational overhead. Furthermore, this invention constructs a dual-path collaborative modeling mechanism, establishing an information interaction relationship between the frequency domain enhancement branch and the sequence modeling branch. This allows high-frequency detail information and long-range dependency information to be fused in a unified feature space, further enhancing the model's ability to characterize target structures in complex scenes.

[0026] Furthermore, this invention employs a multi-level detail condensation mechanism in the decoding stage. By introducing cross-layer information flow through a top-down feature reconstruction path, it effectively mitigates the attenuation problem of shallow detail information during deep layer transmission. A contextual detail enhancement module redistributes high-level semantic information, guiding low-level features to adaptively adjust their spatial location, thereby significantly improving the continuity and accuracy of the segmentation results in boundary regions. Further, a multi-scale feature extraction module performs parallel modeling of the fused features, enabling the model to meet the representational needs of targets at different scales within a unified framework, enhancing its ability to recognize small targets and complex structural regions.

[0027] Regarding model efficiency, this invention employs a lightweight design strategy to effectively control the model parameter size and computational complexity while ensuring feature representation capabilities. For example, low-frequency convolutional blocks replace some large receptive field convolution operations with frequency domain filtering, and high-frequency Mamba blocks utilize a state-space model to achieve efficient global dependency modeling, avoiding the computational overhead of traditional self-attention mechanisms under high-resolution input. Simultaneously, the multi-branch structure reduces redundant computation through parameter sharing and structure compression strategies, enabling the overall network to maintain good performance even on edge devices or in resource-constrained environments. Furthermore, the multi-level detail condensation decoding structure proposed in this invention employs lightweight upsampling and feature fusion methods, reducing the number of parameters while maintaining the stability of the feature reconstruction process, further enhancing the model's practicality and deployability.

[0028] In summary, this invention achieves efficient expression of image semantics and detail information without significantly increasing computational costs through joint modeling of frequency and spatial information, effective utilization of multi-level detail information, and lightweight structural design. It not only improves the precision and structural consistency of segmentation results but also balances accuracy and efficiency requirements in practical applications. It is suitable for image segmentation tasks in complex scenarios and has good engineering application value and promotion prospects. Attached Figure Description

[0029] Figure 1 This is the overall architecture and process framework of the present invention.

[0030] Figure 2 This is a structural diagram of the LFCB module of the present invention.

[0031] Figure 3 This is a structural diagram of the HFMB module of the present invention.

[0032] Figure 4 This is a structural diagram of the CDRB module of the present invention.

[0033] Figure 5 This is a structural diagram of the MSFEB module of the present invention. Detailed Implementation

[0034] Semantic segmentation, as a pixel-level dense prediction task, has significant application value in fields such as autonomous driving, underwater inspection, and industrial defect detection. However, existing lightweight segmentation methods struggle to achieve an effective balance between model expressive power, inference speed, and computational complexity. Furthermore, during feature extraction, low-frequency global semantic information gradually decays, and high-frequency local detail information is easily lost during global modeling, thus limiting further improvements in model performance. To address these issues, this invention proposes a frequency-aware lightweight image segmentation method (LFNet). This method decouples the input image in the frequency domain, decomposing image features into low-frequency global information and high-frequency local detail information, and models them separately. Specifically, in the encoding stage, low-frequency convolutional blocks (LFCB) and high-frequency Mamba blocks (HFMB) are designed to compensate for global semantic information and detail feature information, respectively. In the decoding stage, a multi-level detail condensation structure is constructed, cross-layer feature interaction is achieved through a context detail enhancement module (CDRB), and multi-scale semantic expression capability is further enhanced through a multi-scale feature extraction module (MSFEB).

[0035] Experimental results show that the present invention achieves superior segmentation performance compared to existing methods on multiple public datasets, while maintaining low model parameter count and computational complexity. It has good engineering application value and is suitable for scenarios such as underwater target detection, autonomous driving environmental perception, and industrial vision inspection. The present invention will be further described below with reference to the accompanying drawings.

[0036] The complete process framework of this invention is as follows: Figure 1 As shown. A lightweight image segmentation method based on frequency-aware multi-level detail condensation includes: Step 1: Input Image Preprocessing. Obtain the RGB input image and perform size normalization; extract initial features using the Stem module, which consists of two convolutional layers followed by a batch normalization layer and an activation function, outputting the initial feature map.

[0037] Step 2: Construct a frequency-aware encoder, which includes multiple encoding stages. Each stage includes a downsampling module and a low-frequency convolutional block (LFCB). Figure 2 (as shown) and high-frequency Mamba block HFMB (such as Figure 3 (As shown).

[0038] Step 3: Frequency-aware encoding. The initial features are input into the encoder. At each stage, the resolution and number of channels are adjusted by downsampling, and multi-scale features are extracted using the LFCB module and HFMB module of each stage.

[0039] Low-frequency information modeling. In LFCB, features are mapped to the frequency domain and low-pass filtered, then inversely transformed to restore the spatial domain, and finally fused with the input features and processed by convolution.

[0040] High-frequency information modeling. In HFMB, the features are divided into two branches: one branch extracts high-frequency responses and performs convolutional enhancement, and the other branch performs sequence modeling; the two feature branches are fused and output.

[0041] Step 4: Construct a multi-level detail condensation decoder. This includes the Context Detail Enhancement Module (CDRB) (such as...). Figure 4 (as shown) and the multi-scale feature extraction module MSFEB (as shown) Figure 5 (As shown).

[0042] Step 5: Multi-level detail condensation and decoding. The encoded features are input into the decoder, and the resolution is restored through progressive upsampling. Feature fusion is achieved by combining cross-layer connections.

[0043] Detail enhancement. A response map is generated using the Contextual Detail Enhancement (CDRB) module, which weights low-level features and fuses edge information.

[0044] Multi-scale modeling. Features are processed by parallel convolution through a multi-scale feature extraction module (MSFEB) and then fused to obtain a multi-scale representation.

[0045] Step 6: Output Results. Upsample the features to restore them to the input size, and generate pixel-level segmentation results through convolution.

[0046] Step 7: Model Training and Deployment. The model is trained and its parameters are updated using the cross-entropy loss function; during the inference phase, the input image is used to output the segmentation results.

[0047] The following specific examples will further illustrate this point.

[0048] Example The core of this invention lies in improving the semantic segmentation model's ability to express global structural information and local detailed information through collaborative modeling of frequency information and spatial features, and achieving step-by-step optimization and reconstruction of features through a multi-level detail condensation mechanism, thereby obtaining high-precision segmentation results while ensuring the model's lightweight nature.

[0049] 1. Image data acquisition and preprocessing First, RGB images of the scene to be processed are acquired using an industrial camera or vision sensor. In this invention, the input data is a three-channel red, green, and blue RGB image. To improve the stability of model training and inference, uniform preprocessing operations are performed on the images before inputting them into the network, including size normalization, numerical normalization, and standardization, to ensure that they meet the model input distribution requirements.

[0050] After preprocessing, the input image is fed into the Stem module for initial feature extraction. The Stem module consists of two consecutive convolutional layers, each followed by a batch normalization layer and a non-linear activation function. This allows for layer-by-layer feature mapping of the input image, extracting low-level texture and edge structure information to obtain an initial feature map. This initial feature map not only contains basic visual information but also provides stable input for subsequent frequency modeling. The overall network structure and processing flow of this invention are as follows: Figure 1 As shown.

[0051] 2. Construct a frequency-aware coding architecture This invention enhances feature representation by constructing a frequency-aware encoder to perform multi-stage modeling of input features. The encoder consists of multiple encoding stages and employs a progressive downsampling structure, gradually increasing the number of channels while reducing the spatial resolution of the feature map, thereby achieving a balance between feature compression and semantic enhancement. The overall structure of the encoder is as follows: Figure 1 The encoder section is shown in the image.

[0052] In each encoding stage, low-frequency convolutional blocks (LFCB) and high-frequency Mamba blocks (HFMB) are set up to model information in different frequency bands. In this way, traditional spatial domain feature extraction is extended to joint modeling of the frequency domain and sequence domain, thereby improving the model's ability to express complex structures and detailed information.

[0053] (1) Low-frequency information modeling: In the low-frequency convolution block, the input features are mapped to the frequency domain through fast Fourier transform. The low-frequency components are extracted by filtering the central region of the spectrum through a low-pass filter, thereby obtaining the global structural information and semantic information in the image. Then, the features are restored to the spatial domain through inverse transform and fused with the original input features element by element. The fused features are further input into a multi-branch convolution structure. Different branches extract features through different convolution kernel sizes and dilation rates, thereby realizing multi-scale contextual information modeling and enhancing the feature expression ability while ensuring computational efficiency.

[0054] (2) High-frequency information modeling: In the high-frequency Mamba block, the input features are divided into two processing paths. One path extracts high-frequency components through Fourier transform and high-pass filtering to obtain edge information and texture details in the image, and enhances them through convolution operation. The other path converts the features into a sequence form through linear mapping and inputs them into a state-space model-based structure for modeling to capture long-range dependencies and global context information. Finally, the outputs of the two branches are concatenated in the channel dimension and fused with the feedforward network through a normalization layer. Residual connections are also introduced to improve the stability of the model.

[0055] 3. Multi-layered detail-condensed decoder To achieve high-quality segmentation results, this invention constructs a multi-level detail condensation decoder to reconstruct the features output from the encoding stage step by step. The decoder adopts a top-down structure, restoring the spatial resolution of the feature map through step-by-step upsampling operations. At each layer, features from the corresponding encoding stage are introduced through cross-layer connections, achieving effective fusion between features at different scales. During the decoding process, by fusing high-level semantic features with low-level detail features layer by layer, the model maintains the integrity and consistency of feature representation while restoring spatial resolution, thus providing a good input foundation for subsequent detail enhancement modules. The overall decoding structure is as follows: Figure 1 The decoder section is shown in the image.

[0056] In the decoding stage, to further improve segmentation accuracy and boundary detail representation, this invention introduces a context detail enhancement module (CDRB) and a multi-scale feature extraction module (MSFEB).

[0057] (1) The context detail enhancement module generates a response map by performing convolutional mapping on high-level features, and uses the response map to perform positional weighting on low-level features. At the same time, it combines the structural information extracted by edge detection or high-frequency response, and fuses them by element-wise addition or channel splicing, thereby enhancing the discriminative ability of features in the target boundary region and effectively improving the continuity and precision of the segmentation results.

[0058] (2) The multi-scale feature extraction module adopts a multi-branch parallel structure. Each branch processes the features through dilated convolution with different kernel sizes or different dilation rates, thereby extracting semantic information under different receptive fields. Then, the outputs of each branch are fused in the channel dimension, and feature integration is completed through convolution operation to obtain a unified multi-scale feature representation.

[0059] Finally, the features output by the decoder are upsampled to restore the spatial resolution consistent with the input image, and pixel-level classification results are generated through convolutional layers to obtain the final semantic segmentation map. This segmentation result maintains global semantic consistency while effectively depicting the fine-grained structural information of the target, making it suitable for application scenarios with complex backgrounds, multi-scale targets, and blurred boundaries.

[0060] 4. Model Training Strategy During model training, labeled image data is used for supervised training of the network. A pixel-level cross-entropy loss function is employed to measure the difference between the model's predictions and the true labels, and the network parameters are updated using backpropagation. Furthermore, a learning rate decay strategy and regularization methods are combined during training to improve the model's convergence speed and training stability. This approach effectively reduces overfitting while maintaining segmentation accuracy, thereby enhancing the model's generalization ability across different scenarios.

[0061] 5. Experimental verification and effect evaluation In the experimental evaluation, to ensure comparability between different methods, the mean Intersection over Union (mIoU) was uniformly used as the evaluation metric to quantitatively analyze the model's segmentation performance. mIoU comprehensively reflects the model's segmentation accuracy across different categories and is an important evaluation standard in semantic segmentation tasks. This invention was experimentally validated on multiple publicly available datasets, including Cityscapes and CamVid datasets for urban road scenes, OUC-Crack and CFD datasets for building crack detection scenes, and LIACi and SUIM datasets for complex underwater scenes. Experimental results show that this invention outperforms many mainstream methods in both segmentation accuracy and boundary detail representation while maintaining low computational complexity.

[0062] Meanwhile, this invention is compared with several typical lightweight image segmentation methods, including those based on convolutional neural networks and those based on Transformer structures. The comparison results show that this invention achieves a better balance between global semantic modeling capabilities and local detail representation capabilities, demonstrating good engineering application value and promising prospects for wider application.

[0063] Experimental results show that the present invention achieves higher mIoU values ​​on all six datasets, verifying the effectiveness and practical value of the present invention. The mIoU results of the method of the present invention on the six datasets are shown in the table below: Table 1 shows the mIoU results of the method of this invention and other existing methods on six datasets.

[0064] Existing lightweight image segmentation methods often lack segmentation accuracy. This invention achieves significant improvements in mIoU (might and accuracy) across six datasets compared to other methods, reaching 48.9%, 50.9%, 52.0%, 20.6%, 76.8%, and 72.4% mIoU on the LIACI, SUIM, OUC-Crack, CFD, Cityscapes, and CamVid datasets, respectively. Compared to the superior LMFFNet method, this invention improves the mIoU by 1.0, 1.0, 1.2, 6.7, 1.7, and 0.4 percentage points on the six datasets, with the most significant improvement on the CFD dataset, indicating a stronger ability to represent fine-grained defects and complex boundaries.

[0065] The comprehensive experimental results show that the present invention effectively utilizes the frequency information in the image to achieve collaborative modeling of low-frequency global semantic information and high-frequency local detail information, overcoming the shortcomings of existing methods in feature extraction and demonstrating good generalization ability. It can be applied to tasks such as underwater target detection, autonomous driving environment perception, and industrial vision inspection.

Claims

1. A lightweight image segmentation method based on frequency-aware multi-level detail condensation, characterized in that... Includes the following steps: (1) Image preprocessing: Obtain the RGB image to be segmented, and perform initial feature extraction through the Stem module to obtain the initial feature map for subsequent encoding processing; (2) Constructing a frequency-aware encoder: Construct an encoder consisting of multiple encoding stages. Each encoding stage includes a downsampling module, several low-frequency convolutional blocks (LFCBs) and at least one high-frequency Mamba block (HFMB) for frequency decoupling modeling and extracting multi-scale features. (3) Frequency-aware coding: In each coding stage, downsampling, low-frequency information modeling, and high-frequency information modeling are performed sequentially, wherein: (3.1) Downsampling: The downsampling module compresses its input features by reducing spatial resolution and increasing the number of channels; (3.2) Low-frequency information modeling: The low-frequency convolutional block LFCB maps its input features to the frequency domain through fast Fourier transform, extracts low-frequency components using a low-frequency filter, restores them to the spatial domain through inverse transform, and fuses them with its input features; the fusion result is further enhanced by convolution. (3.3) High-frequency information modeling: The high-frequency Mamba block divides its input features in the channel dimension into two branches. One branch extracts high-frequency details through Fourier transform and high-frequency filtering, and enhances them through convolution. The other branch models long-range dependencies by linearly mapping the input state space model. The output results of the two branches are merged in the channel dimension to obtain multi-scale features, thereby realizing the collaborative modeling of high-frequency details and global information. (4) Constructing a multi-level detail condensation decoder: The decoder upsamples the multi-scale features output from the higher-order coding stage and then performs cross-layer fusion through the context detail enhancement module CDRB to restore spatial resolution and enhance feature expression, thereby realizing information interaction of features at different levels; (5) Multi-layered detailed decoding: (5.1) Feature extraction and upsampling: Extract content information, background information and edge information from the output of the higher-order encoding stage to enhance feature discrimination ability and improve segmentation boundary accuracy, and then upsample to the size of the output of the previous encoding stage; (5.2) Contextual detail enhancement: The Contextual Detail Enhancement (CDRB) module is used to fuse the upsampled result with the output of the previous encoding stage; (5.3) Multi-scale feature extraction module: After fusion, the multi-scale feature extraction module MSFEB is introduced. It models different receptive field features through multi-branch parallel convolution and fuses each branch to obtain multi-scale semantic representation, so as to improve the adaptability to targets of different sizes. (6) Output segmentation results: The multi-scale semantic representation obtained after fusion is restored to the input size by upsampling, and pixel-level segmentation results are generated by convolution; (7) Model deployment: The model is trained through steps (1)-(6). After training, the image to be processed is input and the above steps are performed to obtain the segmentation result.

2. The method according to claim 1, characterized in that... The Stem module used for initial feature extraction of the input image in step (1) includes two consecutive convolutional layers, and each convolutional layer is followed by a batch normalization layer and a non-linear activation function to generate an initial feature map for subsequent encoding processing.

3. The method according to claim 1, characterized in that... The frequency sensing encoder described in step (2) adopts a hierarchical cascaded structure and is designed with three encoding stages. Each encoding stage first achieves downsampling and expands the number of channels through convolution with a stride of 2. The low-frequency convolutional block (LFCB) includes a frequency domain transform unit and a spatial convolutional unit. It maps the input features to the frequency domain using a fast Fourier transform and performs low-frequency filtering through a central mask. The filtered result is then restored to the spatial domain by an inverse transform and fused element-wise with the input features. The fused features are further input into the spatial convolutional unit, which adopts a multi-branch structure and extracts multi-scale features through depth convolution and dilated convolution of different sizes to supplement feature information. The high-frequency Mamba block (HFMB) includes a frequency domain detail enhancement branch and a sequence modeling branch. The frequency domain detail enhancement branch extracts high-frequency features through Fourier transform and high-frequency filtering and performs convolution enhancement. The sequence modeling branch converts its input features into a sequence through linear mapping and inputs it into a state space model for modeling. The outputs of the two branches are concatenated and fused in the channel and processed through normalization and feedforward networks. At the same time, it combines residual connections to output multi-scale features.

4. The method according to claim 1, characterized in that... Step (4) The Contextual Details Enhancement Module (CDRB) generates a response map by convolutional decoding of high-level features. The response map is used to perform element-wise weighting of low-level features. At the same time, edge information is combined and fused by element-wise addition or channel concatenation to enhance the expression of boundary and detail information. The Multi-Scale Feature Extraction Module (MSFEB) includes multiple parallel convolutional branches. Each branch uses convolutional kernels of different sizes or dilated convolutions to extract features. The outputs of each branch are aggregated in the channel dimension and fused by 1×1 convolution to obtain a multi-scale feature representation.

5. The method according to claim 1, characterized in that... The model optimization process in step (7) uses the cross-entropy loss function to train the model. The cross-entropy loss function is a pixel-level classification loss function used to measure the difference between the model's predicted class probability distribution and the true label distribution. The closer the prediction result is to the true label, the smaller the loss value. During the training process, stochastic gradient descent or adaptive optimization algorithm is used to iteratively update the model parameters, and a learning rate decay strategy is combined to improve convergence stability.

6. The application of the method according to claim 1, characterized in that... The method is applied to industrial defect detection, autonomous driving target recognition, and medical lesion segmentation.

Images