Scene style generalization and dynamic space attention fusion underwater biological segmentation method based on improved SegFormer

Abstract

The invention belongs to the field of underwater biological image segmentation, and provides an underwater biological segmentation method based on scene style generalization and dynamic space attention fusion of improved SegFormer. The constructed model is based on SegFormer, and a multi-branch dynamic space attention module, an improved style mixing module and a detail-preserving context feature fusion module are introduced. Under the unfavorable environment of dim and complex underwater, the model provided by the invention significantly improves the segmentation precision of the target boundary, and shows more excellent segmentation performance under the complex scene. The model effectively solves the problems that an existing semantic segmentation model is poor in performance in an underwater scene and rough in segmentation boundary, and important value is provided for research of underwater target image semantic segmentation.

Description

Technical Field

[0001] This invention belongs to the field of underwater organism image segmentation, specifically involving an underwater organism segmentation method based on scene style generalization and dynamic spatial attention fusion of an improved SegFormer. Background Technology

[0002] Underwater semantic segmentation, as a crucial component of underwater visual perception, plays an irreplaceable role in tasks such as marine geological surveys, marine ranching monitoring, and underwater robot navigation and operations. Compared to natural terrestrial images, underwater images are affected by complex optical media, resulting in significant domain-specific visual characteristics. Light propagating in water undergoes significant scattering and absorption, leading to brightness attenuation, color distortion, and blurred contours in distant areas. Simultaneously, the large number of particles and plankton floating in seawater generates random speckle patterns known as "sea snow," significantly increasing image noise levels. These complex factors collectively contribute to unstable underwater image quality, uneven illumination distribution, and severe texture degradation, making it difficult to directly transfer and maintain stable performance from natural image semantic segmentation models.

[0003] To address the challenges posed by underwater imaging degradation, researchers have conducted extensive research on enhancing feature representation, multi-scale contextual aggregation, and robustness to noise. Shi et al. proposed the lightweight architecture CSPLiteNet, which enhances multi-scale feature extraction capabilities, enabling the network to more accurately identify target boundaries in weakly textured regions. Li et al. utilized dilated convolutions to construct multi-scale receptive fields, enhancing the network's ability to perceive complex backgrounds through combinations of different dilation rates. Wang et al. designed USNet, improving the quality of underwater images through multi-scale water network modules, mitigating illumination attenuation and blurring at the feature level. Ge's DINet employs a dual-iteration enhancement mechanism to explicitly model degradation factors, making the network more adaptable to real underwater environments. Meanwhile, Liu et al.'s improved architecture based on DeepLabV3+ effectively preserves the edge contours of marine targets by introducing depthwise separable convolutions and enhanced multi-scale fusion strategies.

[0004] In complex underwater scenarios, target contours are more severely affected by noise, making noise resistance and detail recovery key research areas. Zhang's RUSNet mitigates the impact of motion optical flow noise on segmentation through adaptive filtering, improving robustness to fast-moving targets. Other studies reduce noise in sonar or low-resolution images and improve feature consistency through non-local mean filtering and superpixel aggregation. The Transformer architecture has also been introduced into underwater semantic segmentation in recent years. For example, Pavithra et al. used the Swin Transformer to build a segmentation framework with stronger global modeling capabilities, demonstrating better structured feature extraction capabilities in low-light scenarios. Graph Neural Networks (GNNs) are also being used to model the relationships between underwater targets, achieving good results in scenarios with complex structures or fragmented targets.

[0005] While existing methods have made some progress in feature enhancement, multi-scale information fusion, and noise resistance, underwater images still commonly suffer from uncontrollable lighting conditions, weakened target boundaries, and missing texture information, resulting in segmentation results that often exhibit blurred edges and omissions of small targets. Therefore, constructing a model that can effectively address the challenges of underwater images such as blurring, color cast, and loss of detail, and significantly improve the accuracy and robustness of underwater scene semantic segmentation, is of great significance for promoting the development of underwater semantic segmentation technology. Summary of the Invention

[0006] This invention discloses an improved SegFormer architecture-based image semantic segmentation network for underwater scenes. The core of this network includes: a multi-branch dynamic spatial attention module, which enhances the model's ability to focus on key spatial regions and extract texture features from underwater images by combining a multi-head deformable sampling mechanism with gating units; an improved style mixing module, which introduces semantically aware pairing and channel-level style mixing mechanisms on top of the original MixStyle method to improve the model's generalization performance and training stability under style perturbations; and a detail-preserving contextual feature fusion module, which effectively fuses deep semantic features with shallow detail features through detail-sensitive enhancement and grouped context selection mechanisms, aiming to solve the problem of detail information being submerged during feature decoding and improve the accuracy of segmentation boundaries. Through the synergistic effect of these modules, this invention can effectively address challenges such as blurriness, color cast, and detail loss in underwater images, significantly improving the accuracy and robustness of underwater scene semantic segmentation.

[0007] This invention provides an underwater organism segmentation method based on scene style generalization and dynamic spatial attention fusion using an improved SegFormer, comprising the following steps: Step 1: Acquire underwater target images; Step 2: Based on the Segformer network backbone, construct an underwater target feature extraction model for feature extraction: The multi-branch dynamic spatial attention module is inserted after the output of each stage of the Encoder MiT of the Segformer network, and the improved style mixing module is inserted into the first two layers of the Encoder MiT of the Segformer network. The Multi-Branch Dynamic Spatial Attention Module (MH-DDSA) includes three parallel branches: a multi-head deformable attention branch, a spatial frequency domain approximation branch, and a deep convolution enhancement branch, which are used to capture dynamic spatial attention, frequency domain features, and local details, respectively. The multi-branch dynamic spatial attention module uses the features output by the Stage as the input feature map; Adaptive fusion is performed after processing through three parallel branches: The input feature map enters the multi-head deformable attention branch. First, the offset is learned using the convolutional layer, and dynamic offset sampling points are generated for each pixel, so that the model can adaptively focus on the key spatial regions of the underwater image. Pyramid pooling is then used to construct a multi-scale receptive field, aggregate global context information, and generate a low-rank dynamic convolutional kernel based on the aggregated global context information. The low-rank dynamic convolutional kernel is then used to perform weighted aggregation on the features extracted based on the dynamic offset sampling points to generate a spatial attention map, which is then multiplied with the input feature map to achieve dynamic emphasis on key features and suppression of noise. The spatial frequency domain approximation branch employs a spatial domain convolution approximation strategy on the input feature map. By using convolution kernels initialized with Laplacian and mean operators, it directly separates the high-frequency and low-frequency features of the input feature map in the spatial domain. High-frequency features are used to enhance edges and textures, while low-frequency features are used to capture the overall structure and background. High- and low-frequency features are concatenated and then fused using 1x1 convolution. This process avoids the overhead of complex number calculations while compensating for the loss of high-frequency details caused by underwater scattering.

[0008] The depthwise convolution enhancement branch uses depthwise separable convolutions to perform lightweight processing on the input feature map, and utilizes 3x3 convolution kernels to efficiently capture local spatial details, thereby reducing the number of parameters and enhancing the model's ability to express the surface texture of underwater organisms.

[0009] The output features of the multi-head deformable attention branch, the spatial frequency domain approximation branch, and the deep convolutional enhancement branch are fed into the gate control unit for adaptive fusion to obtain the fused feature map; the gate control unit generates dynamic weights based on the semantic content of each pixel position, enabling each pixel to autonomously select the enhancement mode; The fused feature map is enhanced with channel attention through the SE (Squeeze-and-Excitation) module: first, the spatial dimension is compressed by global average pooling to generate channel weights; then, the channel importance is recalibrated through fully connected layers and activation functions to improve the model's robustness to color distortion and illumination changes in underwater images; finally, the input feature map is added to the enhanced feature map through residual connections to ensure training stability and avoid gradient vanishing.

[0010] The improved style mixing module, based on the traditional MixStyle (MixStyle Neural Networks for Domain Generalization and Adaptation) module, introduces a semantically aware sample pairing strategy to ensure that the mixing operation is performed between similar semantic features, and adopts channel-level mixing intensity. It allows each channel to adjust the mixing ratio independently; This design endows features with more fine-grained style perturbation capabilities and stronger structure preservation, effectively solving the semantic ambiguity problem caused by the original global mixing, and significantly enhancing the model's generalization ability in complex underwater lighting and cross-domain scenarios.

[0011] The improved style mixing module takes the feature maps from the first two layers of Encoder MiT as input feature maps, applies semantic-aware pairing to the input feature maps, calculates the sample similarity within a batch and selects the optimal pairing; based on the pairing results, it performs channel-level style mixing to generate a mixed feature map and passes it to subsequent network layers. Step 3: Feature Fusion The All-MLP Decoder of the SegFormer network is replaced with a detail-preserving contextual feature fusion module. The standard SegFormer Decoder simply uses a simple MLP to map features to a common dimension and then sums them. This "brute-force concatenation" ignores the semantic interference between features at different levels and does not fully utilize high-level semantics to "guide" the selection of low-level details. For noisy underwater images, it is easy to introduce redundant information. The detail-preserving contextual feature fusion module introduces a stronger cross-scale interaction mode, making the fusion process more selective, dynamic, and fine-grained.

[0012] A cascaded detail-preserving decoding network is constructed based on the three detail-preserving context feature fusion modules, using a top-down path: the top-down path is specifically Stage 4-Stage 3-Stage 2-Stage 1. Each fusion stage uses a detail-preserving context feature fusion module, using high-level features as context guides to dynamically filter effective textures in low-level features and suppress noise. The detail-preserving contextual feature fusion module includes a Grouped Adaptive Combiner, which contains a global semantic path and a local detail path; The input to the global semantic path is high-level features. A global weight vector is generated using AdaptiveAvgPool(1)+MLP to adjust the importance of different channels.

[0013] The input to the local detail path is high-level features. With low-level features Concat ( , Edge information is extracted through 3x3 convolution, BN, and ReLU to generate a pixel-level spatial attention map.

[0014] The fusion weight A of the grouped adaptive combiner is obtained by adding the two paths mentioned above and then applying the Sigmoid function: ; It is a weight term of the global semantic path. It is the weight term of the local detail path. Based on the fusion weight A, the output features of the grouped adaptive combiner are obtained. ; Underwater images typically exhibit the following harsh characteristics, and the detail-preserving contextual feature fusion module is specifically optimized to address these issues: local detail paths are focused on edge information extraction through 3x3 convolutions. Compared to SegFormer's direct upsampling, DPCF progressively introduces high-resolution features at lower levels and utilizes an attention mechanism to "sharpen" edges, significantly improving the edge fit of the segmentation mask. Furthermore, in complex underwater environments, if a certain area is merely noise (responding in lower-level features but not supported by higher-level semantics), the fusion weight A tends to suppress the lower-level features at that location, thereby filtering out background suspended particle noise.

[0015] Step 4: Obtain images of underwater organisms to be segmented, input them into the underwater target segmentation model, and obtain the underwater segmentation results.

[0016] Preferably, in step 2, the improved style mixing module introduces a semantically aware sample pairing strategy to ensure that the mixing operation is performed between similar semantic features, and adopts channel-level mixing intensity. It allows each channel to adjust the mixing ratio independently, specifically including: Improvement 1: The original MixStyle module randomly selects a sample to mix, but random pairing may mix completely dissimilar objects, leading to semantic corruption.

[0017] In the improved style mixing module, the random sample selection strategy of the MixStyle module is changed to: first extracting each sample using the GAP method. semantic representation : ; Calculate the cosine similarity matrix for the semantic representation of all samples within the batch: ; and These represent the semantic representations of two different samples; represent and The cosine similarity matrix; Then, based on the cosine similarity matrix, the most similar sample is selected for each sample: ; Improvement 2: In the original MixStyle, all channels share a [1, 1] λ matrix, but different channels represent different semantics, so they obviously should not share the same mixing intensity.

[0018] The improved style blending module generates a λ matrix of shape [C, 1, 1] based on the channel dimension C of the input feature map, which is used to independently control the blending ratio of each channel, thereby more effectively simulating complex underwater style changes: Preferably, in step 3, for the current layer Stage N, the higher-level features refer to the output from a deeper level of the encoder, namely Stage N+1, or the output features from the previous level detail-preserving context feature fusion module, while the lower-level features come from the output features from a shallower level of the encoder, namely Stage N. Preferably, in step 1, the types of underwater targets include: sea cucumbers, sea urchins, scallops, and starfish.

[0019] Preferably, step 1 further includes: setting the length and width of the underwater target image to 512×512; and performing regularization processing on the underwater target image to map it to a normal distribution function.

[0020] Preferably, the output features of the grouped adaptive combiner are obtained based on the fusion weights A. Specifically, it includes: Output features The expression is as follows: ; For background noise regions with suspended particles, the weighting term of the local detail path The value is positive due to capturing the texture of suspended particles; it is a weight term of the global semantic path. The region was determined to be a background lacking semantic support, hence the negative value. + <0; the weights A obtained by mapping with the Sigmoid function tend to 0. Therefore, the model automatically “chooses” numerically to suppress low-level features and instead utilizes the smoothness of high-level features to filter out background noise.

[0021] For the biological contour region, the weight term of the local detail path is used as the target edge region. The weights of the global semantic path are positive because they capture significant gradient changes. Because the region is determined to have a semantic boundary for objects and is therefore a positive value, the weight A obtained by mapping with the Sigmoid function tends towards 1. Therefore, the model numerically "selects" to prioritize preserving low-level features, maximizing the recovery of the fine contours of underwater organisms.

[0022] The beneficial effects of this invention are as follows: In the challenging, dimly lit underwater environment, the model provided by this invention significantly improves the segmentation accuracy of target boundaries and exhibits superior segmentation performance in complex scenes. This model effectively addresses the shortcomings of existing semantic segmentation models in underwater scenarios, such as poor performance and coarse segmentation boundaries, providing significant value for research on semantic segmentation of underwater target images. Attached Figure Description

[0023] Figure 1 This is a flowchart of the Segformer backbone network algorithm constructed in this invention; Figure 2 This is an example diagram of the improved multi-branch dynamic spatial attention module of this invention; Figure 3 This is an example diagram of the style mixing module improved by the present invention; Figure 4This is an example diagram of the improved detail-preserving context feature fusion module of the present invention; Figure 5 This is a comparison diagram of the segmentation results of the present invention. Detailed Implementation

[0024] The invention will now be further described with reference to the accompanying drawings.

[0025] like Figure 1 As shown, to address the problem of dark and complex semantic segmentation in underwater target images, this invention proposes an improved image semantic segmentation network for underwater scenes based on the SegFormer architecture. This network significantly improves segmentation accuracy and robustness, especially when dealing with low contrast, blurred boundaries, and difficult-to-identify targets. This invention includes the following steps: Step 1: Acquiring underwater target images The DUT-USEG dataset primarily contains underwater target image data across four different categories, including 1487 natural underwater target images and their ground truth semantic labels, as well as a test set of 110 images collected in ocean exploration and human-robot collaborative experiments. This dataset provides pixel-level annotations for eight object categories: sea cucumber, sea urchin, scallop, and starfish.

[0026] Prepare the dataset and perform preprocessing. The dataset includes 1487 underwater target images; of which 1338 images were designated for training and 149 images were reserved for testing. The images were preprocessed in a 9:1 ratio to form the dataset. The dimensions of the underwater target images were set to 512×512. The underwater biological images were regularized and mapped to a normal distribution function.

[0027] Step 2: Extract features.

[0028] An underwater target feature extraction model is constructed, based on the backbone network of the SegFormer network. Given an image of size H×W×3, it is first divided into 4×4 blocks using overlapping block division. The encoder uses these image blocks as input to a hierarchical Transformer encoder to obtain multi-level features at the original image resolutions {1 / 4, 1 / 8, 1 / 16, 1 / 32}. The MixStyleImproved module output is inserted into the Transformer blocks of Stage 1 and Stage 2. After each Stage output (F1..F4), MH_DDSA is applied to achieve local / global alignment and enhancement of multi-head deformable spatial attention. The processed multi-scale features (after MH_DDSA F1..F4) are upsampled / projected and then fed into the SegFormer MLP Decoder.

[0029] The Multi-Branch Dynamic Spatial Attention Module (MH_DDSA module) is an innovative component proposed in this invention for underwater image segmentation tasks, such as... Figure 2 As shown, its core lies in capturing dynamic spatial attention, frequency domain features, and local details through three parallel branches, and finally adaptively fusing them through a gating mechanism to address the problems of blurring, noise, and detail loss in underwater images. Specifically, it includes the following core components and processing flow: Input feature map processing Multi-head deformable attention branch: This branch aims to address the problem of distorted and non-rigid deformation of object shapes in underwater images. The input feature map first uses convolutional layers to learn offsets, generating dynamic offset sampling points for each pixel, enabling the model to adaptively focus on key spatial regions of the underwater image; Multi-scale receptive fields are constructed using pyramid pooling to aggregate global contextual information, and low-rank dynamic convolutional kernels are generated based on the aggregated global contextual information. The low-rank dynamic convolutional kernels are then used to weight and aggregate the features extracted based on dynamic offset sampling points to generate a spatial attention map, which is then multiplied with the input feature map to achieve dynamic emphasis on key features and suppression of noise. Spatial frequency domain approximation branch: The spatial frequency domain approximation branch employs a spatial domain convolution approximation strategy on the input feature map, using convolution kernels initialized with Laplacian and mean operators to directly separate the high-frequency and low-frequency features of the input feature map in the spatial domain.

[0030] By initializing the input feature map as convolutional layers with Laplacian kernels and Mean kernels, the high-frequency components (edges, texture details) and low-frequency components (smooth background, overall structure) of the image are approximately extracted, respectively. High-frequency features are used to enhance edges and textures, while low-frequency features are used to capture the overall structure and background. The extracted high-frequency and low-frequency features are concatenated and then fused using 1x1 convolution. This avoids the overhead of complex computations while compensating for the loss of frequency domain information caused by underwater scattering.

[0031] Depthwise convolutional augmentation branch: This branch employs depthwise separable convolutions to perform lightweight processing on the input feature map, enhancing local feature representation while maintaining low computational cost. A 3x3 convolution kernel is used to efficiently capture local spatial details, reducing the number of parameters and enhancing the model's ability to represent the surface texture of underwater organisms. The 3x3 kernel size effectively captures local texture details in underwater images.

[0032] Multi-branch gating fusion mechanism The outputs of the three branches (deformable attention features, frequency domain approximation features, and local convolutional features) are fused through a gating unit: the features from each branch are concatenated, passed through a lightweight "Conv-ReLU-Conv" network, and normalized in the channel dimension using the Softmax function to generate mutually exclusive dynamic gating weights. Based on the generated weights, the outputs of the three branches are weighted and summed. This allows the model to select the optimal enhancement strategy for each pixel. SE (Squeeze-and-Excitation) module and residual connection The fused feature maps are further enhanced with channel attention through the SE module: first, the spatial dimension is compressed by global average pooling to generate channel weights, and then the channel importance is recalibrated by fully connected layers and activation functions (such as ReLU) to improve the model’s robustness to color distortion and illumination changes in underwater images.

[0033] Finally, the module adds the input feature map to the enhanced feature map through residual connections, ensuring training stability and avoiding gradient vanishing.

[0034] Core processing flow (input-processing-output) Input: Multi-scale feature maps (e.g., F1 to F4) from SegFormer Encoder, with a resolution of H×W×C.

[0035] Processing procedure: The input feature map is fed into the multi-head deformable attention branch, frequency domain enhancement branch, and depth convolution enhancement branch in parallel.

[0036] The outputs of each branch are weighted and fused through a gating fusion mechanism to generate intermediate features.

[0037] The intermediate features are enhanced by the SE module and then added to the input features through residual connections.

[0038] Output: Enhanced feature map with a resolution of H×W×C, possessing stronger spatial attention capabilities and detail preservation.

[0039] This module significantly improves the model's ability to focus on key areas of underwater images through the synergistic effect of the aforementioned multi-branch architecture, and solves the performance degradation problem of traditional segmentation models under blurred boundaries and noise interference.

[0040] Step 3: Feature Fusion. The standard SegFormer Decoder simply uses a simple MLP to map features to a common dimension and then sums them. The detail-preserving contextual feature fusion module introduces a stronger cross-scale interaction method, making the fusion process more selective, dynamic, and fine-grained.

[0041] Since the output features F1–F4 of MiT have different resolutions and number of channels, DPCF first unifies them: it aligns the spatial resolution step by step through upsampling, uses 1×1 Conv to unify the channel dimension C, and then calculates the importance of each layer through a dynamic weight generation module. During the progressive propagation process from high to low layers, it achieves complementary fusion of semantics and details, enabling the final features to possess both stronger semantic consistency and more detailed structural expression capabilities. The four multi-layer features Fi obtained in step 2 are input into the feature fusion model to obtain a feature matrix of size (H, W, 256), which is then fed into the multi-scale patch segmentation head. Here, (H, W) represents the size of the underwater biological image input, and 256 represents the number of channels in the image. The underwater target segmentation model is then trained using the dataset from step 1 to obtain a trained underwater target segmentation model. Step 4: Obtain the underwater organism image to be segmented, input it into the trained underwater target segmentation model, and obtain the underwater segmentation result.

[0042] The following sections further introduce the SegFormer network, the improved style blending module, and the detail-preserving context feature fusion module.

[0043] 1. SegFormer network SegFormer stands for "SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers." Its core goal is to build a semantic segmentation framework that is both simple and efficient, as well as powerful in performance. By eliminating some of the complex and computationally expensive designs found in traditional Visual Transformers (ViT), such as positional encoding and complex decoders, it successfully improves computational efficiency and the model's robustness to different input resolutions while maintaining high accuracy.

[0044] The model consists of two major innovative parts: a hierarchical Transformer Encoder and a lightweight all-MLP Decoder.

[0045] 2. Improved style mixing module: Improved style hybrid module structure such as Figure 3 As shown, this module improves the generalization performance and training stability of underwater image segmentation models under style perturbations. Based on the original MixStyle method, it enhances semantic integrity and improves the model's adaptability to lighting and texture variations in underwater images through a semantically aware pairing mechanism and channel-level style mixing.

[0046] An improved style blending module is inserted into the first two layers of the Encoder MiT in the SegFormer network (i.e., after the outputs of Stage 1 and Stage 2). Its core function is to enhance the diversity of feature representations through style blending, thereby improving the model's robustness to underwater domain shifts (such as color cast and uneven illumination). The output features of this module maintain the same dimensionality as the input features.

[0047] Improvement 1: The original version randomly selects a sample to mix, but random pairing may mix completely dissimilar objects, leading to semantic corruption.

[0048] The improved method first extracts the semantic representation of each sample (via GAP): ; Calculate the cosine similarity matrix for all samples within a batch, which is then used for semantically-aware pairing. ; Then, the similarity between samples within the batch is calculated, and the best pairing is selected, which means selecting the most similar sample for each sample: ; Improvement 2: In the original MixStyle, all channels share a [1, 1] λ matrix, but different channels represent different semantics, so they obviously shouldn't share the same mixing intensity. The improved style mixing module generates a [C, 1, 1] λ matrix based on the channel dimension C of the input feature map, achieving independent mixing intensity for each channel. To address the shortcomings of the original version where all channels share the same blending intensity λ, and to adapt to the semantic differences between different channels (such as some channels encoding textures and others encoding shapes).

[0049] When blending styles, the original version uses a scalar λ to control the blending intensity; the improved version generates channel-level blending weights λ with a shape of [C, 1, 1], allowing each channel to adjust the blending intensity independently.

[0050] The weights λ are generated through learnable parameters or channel attention based on feature maps, ensuring that high-frequency channels (such as edge details) and low-frequency channels (such as semantic structure) adopt a differentiated mixing strategy.

[0051] The formula for mixing is: , ; In the above formula, Bold text represents tensors; Feature Reconstruction: + ; x: The feature map of the current input; Features of the most similar samples selected through semantically aware pairing (cosine similarity); , : The mean and standard deviation of the feature map in the spatial dimensions (H, W).

[0052] , The mean and standard deviation after mixing.

[0053] Output features Detail-preserving context feature fusion module Unlike the original detail-preserving context feature fusion module for CNNs, the SegFormer-detail-preserving context feature fusion module proposed in this invention has made structural improvements to Transformer multi-scale token features, including channel alignment, semantically driven dynamic weight generation, and a globally consistent cross-scale fusion strategy, making it more suitable for the multi-scale encoding structure of SegFormer and significantly improving the ability to preserve small targets and details.

[0054] The detail-preserving context feature fusion module proposed in this invention has the following structure: Figure 4 Used to replace the All-MLP Decoder in the SegFormer network; A cascaded detail-preserving decoding network is constructed based on the three detail-preserving context feature fusion modules, adopting a top-down path: Stage 4 - Stage 3 - Stage 2 - Stage 1. Each fusion stage uses a detail-preserving context feature fusion module, utilizing high-level features as context guides to dynamically filter effective textures in low-level features and suppress noise. For the current stage N, the high-level features refer to the output from a deeper level of the encoder, i.e., Stage N+1, or the output features from the previous level detail-preserving context feature fusion module, while the low-level features come from the output features of a shallower level of the encoder, i.e., Stage N. The detail-preserving contextual feature fusion module includes a Grouped Adaptive Combiner, which contains a global semantic path and a local detail path; The input to the global semantic path is high-level features. A global weight vector is generated using AdaptiveAvgPool(1)+MLP to adjust the importance of different channels.

[0055] The input to the local detail path is high-level features. With low-level features Concat ( , Edge information is extracted through 3x3 convolution, BN, and ReLU to generate a pixel-level spatial attention map.

[0056] The fusion weight A of the grouped adaptive combiner is obtained by adding the two paths mentioned above and then applying the Sigmoid function: ; It is a weight term of the global semantic path. It is the weight term of the local detail path. Based on the fusion weight A, the output features of the grouped adaptive combiner are obtained. Output features The generation of this reflects the soft handover mechanism: Output features The expression is as follows: ; For background noise regions with suspended particles, the weighting term of the local detail path The value is positive due to capturing the texture of suspended particles; it is a weight term of the global semantic path. The region was determined to be a background lacking semantic support, hence the negative value. + <0; the weights A obtained by mapping with the Sigmoid function tend to 0. Therefore, the model automatically “chooses” numerically to suppress low-level features and instead utilizes the smoothness of high-level features to filter out background noise.

[0057] For the biological contour region, the weight term of the local detail path is used as the target edge region. The weights of the global semantic path are positive because they capture significant gradient changes. Because the region is determined to have a semantic boundary for objects and is therefore a positive value, the weight A obtained by mapping with the Sigmoid function tends towards 1. Therefore, the model numerically "selects" to prioritize preserving low-level features, maximizing the recovery of the fine contours of underwater organisms.

[0058] Underwater images typically exhibit the following harsh characteristics, and the detail-preserving contextual feature fusion module is specifically optimized to address these issues: local detail paths are focused on edge information extraction through 3x3 convolutions. Compared to SegFormer's direct upsampling, DPCF progressively introduces high-resolution features at lower levels and utilizes an attention mechanism to "sharpen" edges, significantly improving the edge fit of the segmentation mask. Furthermore, in complex underwater environments, if a certain area is merely noise (responding in lower-level features but not supported by higher-level semantics), the fusion weight A tends to suppress the lower-level features at that location, thereby filtering out background suspended particle noise.

[0059] 4. Ablation test To verify the effectiveness of the improved modules, ablation experiments were conducted on the multi-branch dynamic spatial attention module, the improved style mixing module, and the detail-preserving contextual feature fusion module. The results are shown in Table 1. Table 1. Ablation experiments of different Basicblocks

[0060] Compared to the original Segformer, the multi-branch dynamic spatial attention module, the improved style mixing module, and the detail-preserving contextual feature fusion module all result in a stable improvement in the model's mIoU, mPA, and accuracy, greatly enhancing the model's segmentation performance.

[0061] Output underwater segmentation results After training the model, the test set is input into the trained model to obtain the final underwater organism segmentation results. Simultaneously, the test set is input into other different pre-trained benchmark models to obtain corresponding classification results. Then, the algorithm performance is compared and analyzed.

[0062] Nine different networks were tested on the dataset. Each experiment ran for 100 epochs, and the average of the last 10 epochs was used to prevent errors. Table 2 shows the experimental results for different models.

[0063] Table 2 Experimental results on the dataset

[0064] According to the experimental results in Table 2, on the DUT-USEG dataset, the mIoU index of the GDA-SegFormer network provided by this invention can reach 62.14%, which has a significant advantage compared with classic networks such as DeeplabV3+, U-Net, segformer, and SegNet, and also has certain advantages compared with networks proposed in recent years such as UWSegFormer and SAN.

[0065] We used the validation set of the DUT-USEG dataset to make predictions in the SegFormer and GDA-SegFormer models respectively, and the results are as follows. Figure 5 As shown. From Figure 5 It can be seen that GDA-SegFormer can segment underwater scenes with high detection and recognition accuracy. At the same time, the boundary accuracy and accuracy of the segmented underwater objects are higher than those of traditional algorithms.

[0066] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. An underwater organism segmentation method based on improved SegFormer scene style generalization and dynamic spatial attention fusion, characterized in that, Includes the following steps: Step 1: Acquire underwater target images; Step 2: Based on the Segformer network backbone, construct an underwater target feature extraction model for feature extraction: The multi-branch dynamic spatial attention module is inserted after the output of each stage of the Encoder MiT of the Segformer network, and the improved style mixing module is inserted into the first two layers of the Encoder MiT of the Segformer network. The Multi-Branch Dynamic Spatial Attention Module (MH-DDSA) includes three parallel branches: a multi-head deformable attention branch, a spatial frequency domain approximation branch, and a deep convolution enhancement branch, which are used to capture dynamic spatial attention, frequency domain features, and local details, respectively. The multi-branch dynamic spatial attention module uses the features output by the Stage as the input feature map; Adaptive fusion is performed after processing through three parallel branches: The input feature map enters the multi-head deformable attention branch. First, the offset is learned using the convolutional layer, and dynamic offset sampling points are generated for each pixel, so that the model can adaptively focus on the key spatial regions of the underwater image. Pyramid pooling is then used to construct a multi-scale receptive field, aggregate global context information, and generate a low-rank dynamic convolutional kernel based on the aggregated global context information. The low-rank dynamic convolutional kernel is then used to perform weighted aggregation on the features extracted based on the dynamic offset sampling points to generate a spatial attention map, which is then multiplied with the input feature map to achieve dynamic emphasis on key features and suppression of noise. The spatial frequency domain approximation branch employs a spatial domain convolution approximation strategy on the input feature map. By using convolution kernels initialized with Laplacian and mean operators, it directly separates the high-frequency and low-frequency features of the input feature map in the spatial domain. High-frequency features are used to enhance edges and textures, while low-frequency features are used to capture the overall structure and background. High- and low-frequency features are concatenated and then fused using 1x1 convolution, which avoids the overhead of complex number calculations and compensates for the loss of high-frequency details caused by underwater scattering. The depthwise convolution enhancement branch uses depthwise separable convolutions to perform lightweight processing on the input feature map, and uses 3x3 convolution kernels to efficiently capture local spatial details, thereby reducing the number of parameters and enhancing the model's ability to express the surface texture of underwater organisms. The output features of the multi-head deformable attention branch, the spatial frequency domain approximation branch, and the deep convolutional enhancement branch are fed into the induction control unit for adaptive fusion to obtain the fused feature map. The gating unit generates dynamic weights based on the semantic content of each pixel location, enabling each pixel to autonomously select an enhancement mode; The fused feature maps are enhanced with channel attention through the SE (Squeeze-and-Excitation) module: first, the spatial dimension is compressed by global average pooling to generate channel weights; then, the channel importance is recalibrated by fully connected layers and activation functions to improve the model’s robustness to color distortion and illumination changes in underwater images. Then, the input feature map is added to the enhanced feature map through residual connections; The improved style mixing module, based on the MixStyle module, introduces a semantically aware sample pairing strategy to ensure that the mixing operation is performed between similar semantic features, and adopts channel-level mixing intensity. It allows each channel to adjust the mixing ratio independently; The improved style mixing module takes the feature maps from the first two layers of Encoder MiT as input feature maps, applies semantic-aware pairing to the input feature maps, calculates the sample similarity within a batch and selects the optimal pairing; based on the pairing results, it performs channel-level style mixing to generate a mixed feature map and passes it to subsequent network layers. Step 3: Feature Fusion Replace the All-MLP Decoder of the SegFormer network with a detail-preserving contextual feature fusion module; A cascaded detail-preserving decoding network is constructed based on the three detail-preserving context feature fusion modules described above, using a top-down path: each fusion stage uses a detail-preserving context feature fusion module, utilizing high-level features as context guides to dynamically filter effective textures in low-level features and suppress noise; The detail-preserving contextual feature fusion module includes a Grouped Adaptive Combiner, which contains a global semantic path and a local detail path; The input to the global semantic path is high-level features. A global weight vector is generated using AdaptiveAvgPool(1)+MLP to adjust the importance of different channels; The input to the local detail path is high-level features. With low-level features Concat () , Edge information is extracted through 3x3 convolution, BN, and ReLU to generate a pixel-level spatial attention map. The fusion weight A of the grouped adaptive combiner is obtained by adding the two paths mentioned above and then applying the Sigmoid function: ； It is a weight term of the global semantic path. It is the weight term of the local detail path. Based on the fusion weight A, the output features of the grouped adaptive combiner are obtained. ; Step 4: Obtain images of underwater organisms to be segmented, input them into the underwater target segmentation model, and obtain the underwater segmentation results.

2. The underwater organism segmentation method based on scene style generalization and dynamic spatial attention fusion according to claim 1, characterized in that, In step 2, the improved style mixing module introduces a semantically aware sample pairing strategy to ensure that the mixing operation is performed between similar semantic features, and adopts channel-level mixing intensity. It allows each channel to adjust the mixing ratio independently, specifically including: In the improved style mixing module, the random sample selection strategy of the MixStyle module is changed to: first extracting each sample using the GAP method. semantic representation : ； Calculate the cosine similarity matrix for the semantic representation of all samples within the batch: ； and These represent the semantic representations of two different samples; represent and The cosine similarity matrix; Then, based on the cosine similarity matrix, the most similar sample is selected for each sample: ； The improved style blending module generates a λ matrix of shape [C, 1, 1] based on the channel dimension C of the input feature map, which is used for independent control of the blending ratio of each channel.

3. The underwater organism segmentation method based on scene style generalization and dynamic spatial attention fusion according to claim 1, characterized in that, In step 3, the top-down path is specifically: Stage 4 - Stage 3 - Stage 2 - Stage 1; For the current Stage N, the higher-level features refer to the output of the encoder at a deeper level, namely Stage N+1, or the output features of the previous level detail-preserving context feature fusion module, while the lower-level features come from the output features of the encoder at a shallower level, namely Stage N.

4. The underwater organism segmentation method based on scene style generalization and dynamic spatial attention fusion according to claim 1, characterized in that, In step 1, the types of underwater targets include: sea cucumbers, sea urchins, scallops, and starfish.

5. The underwater organism segmentation method based on scene style generalization and dynamic spatial attention fusion according to claim 2, characterized in that, Step 1 also includes: setting the width and height of the underwater target image to 512×512; and performing regularization processing on the underwater target image to map it to a normal distribution function.

6. The underwater organism segmentation method based on scene style generalization and dynamic spatial attention fusion according to claim 1, characterized in that, The output features of the grouped adaptive combiner are obtained based on the fusion weight A. Specifically, it includes: Output features The expression is as follows: ； For background noise regions with suspended particles, the weighting term of the local detail path The value is positive due to capturing the texture of suspended particles; it is a weight term of the global semantic path. The region was determined to be a background lacking semantic support, hence the negative value. + <0; the weight A obtained by mapping with the Sigmoid function tends to 0; For the biological contour region, the weight term of the local detail path is used as the target edge region. The weights of the global semantic path are positive because they capture significant gradient changes. Since the region is determined to have a semantic boundary for objects and is therefore a positive value, the weight A obtained by mapping through the Sigmoid function tends to 1.

Images