A two-stage edge-guided semantic segmentation method, system, and medium for camouflaged objects.

Abstract

This disclosure provides a semantic segmentation method, system, and medium for camouflaged objects based on two-stage edge guidance. The method utilizes SAM2 and Hiera backbone networks to extract multi-scale features and generates an edge prior map based on these features. Then, a multi-granularity edge feature perception module provides a feature basis for two-stage edge guidance through its edge representation capabilities at different scales and directions, resulting in enhanced multi-scale features. Based on the enhanced multi-scale features and the edge prior map, and combined with the edge confidence map of the upper-layer prediction results, the features are dynamically optimized layer by layer from low to high scale, generating intermediate feature results containing fine-grained edge information. Based on the dynamically optimized intermediate feature results, a cross-scale edge-assisted decoder is used to integrate and reconstruct the intermediate feature results. A layer-by-layer edge reconstruction feedback mechanism is used to achieve feature refinement and edge enhancement, ultimately outputting the semantic segmentation result of the camouflaged object.

Description

Technical Field

[0001] This disclosure relates to the field of image segmentation, and in particular to a method, system, and medium for semantic segmentation of camouflaged objects based on two-stage edge guidance. Background Technology

[0002] Semantic segmentation of camouflaged objects is more challenging than ordinary object detection. The main difficulty lies in the fact that the "camouflage" makes the target and the background highly similar in color, texture, and structure, making it difficult for the model to accurately distinguish between the foreground and the background. Currently, most semantic segmentation methods for camouflaged objects still rely on single-stage or single-path detection, lacking joint modeling of target edge details and subject regions. This easily leads to problems such as blurred boundaries and missing details, thus affecting the overall segmentation accuracy. Summary of the Invention

[0003] This disclosure provides at least one method, system, and medium for semantic segmentation of camouflaged objects based on two-stage edge guidance, in order to solve at least one of the above-mentioned technical problems.

[0004] According to one aspect of this disclosure, a semantic segmentation method for camouflaged objects based on two-stage edge guidance is provided, comprising:

[0005] The camouflaged object image to be segmented is input into the SAM2 and Hiera backbone networks, and multiple feature maps of different scales output by the SAM2 and Hiera backbone networks are obtained. The highest-scale feature map and the lowest-scale feature map are input into the gradient edge perception module, and the gradient edge perception module fuses local edge details and high-level semantic information to obtain the edge prior map of the camouflaged object image. The multiple feature maps of different scales are input into the multi-granularity edge feature perception module, and the enhanced multi-scale features corresponding to each feature map output by the multi-granularity edge feature perception module are obtained.

[0006] The coarse prediction module is used to process each enhanced multi-scale feature to obtain a coarse segmentation result, and the coarse segmentation result is used as the upper-level prediction result corresponding to the lowest scale feature map.

[0007] Edge-guided optimization and decoding are performed layer by layer from the lowest-scale feature map to the highest-scale feature map to obtain the prediction results at each scale:

[0008] The current-scale feature map, edge prior map, enhanced multi-scale features corresponding to the current feature map, and upper-layer prediction results are input into the dynamic edge optimization module, and the intermediate feature results after dynamic edge optimization are obtained from the output of the dynamic edge optimization module.

[0009] The intermediate feature results and upper-level prediction results corresponding to the current scale are input into the cross-scale edge-assisted decoder, and the edge prediction map output by the cross-scale edge-assisted decoder is obtained.

[0010] Based on the edge prediction map, the prediction result at the current scale is obtained by decoding.

[0011] The prediction result corresponding to the highest scale is taken as the final semantic segmentation result.

[0012] In one possible implementation, the ViT image encoder in the Hiera backbone network of the SAM2 Hiera backbone network is frozen, and only the feature transfer function of the ViT image encoder is retained; a lightweight adapter module is inserted before each scale block in the Hiera backbone network to achieve fine-grained feature tuning.

[0013] In one possible implementation, the multi-granularity edge feature perception module includes an input feature preprocessing unit, a direction-aware convolutional branch group, and a gated channel attention unit.

[0014] The input feature preprocessing unit includes a convolutional layer with a kernel size of 1×1, which is used to perform channel compression and preliminary feature transformation on the input features to obtain a standardized intermediate feature map.

[0015] The orientation-aware convolutional branch group includes a first group of side branches, a second group of side branches, a third group of side branches, and a fourth group of side branches. The first group of side branches, the second group of side branches, the third group of side branches, and the fourth group of side branches process the intermediate feature map respectively, and obtain the first scale feature, the second scale feature, the third scale feature, and the fourth scale feature respectively. The orientation-aware convolutional branch group is also used to concatenate and fuse the first scale feature, the second scale feature, the third scale feature, and the fourth scale feature in the channel dimension to obtain a multi-granularity feature representation.

[0016] The gated channel attention unit is used to perform channel attention mapping on the multi-granularity feature representation through a convolution operation with a kernel size of 1×1, calculate the weights corresponding to the first scale feature, the second scale feature, the third scale feature and the fourth scale feature respectively, and use the weights to perform weighted fusion of the first scale feature, the second scale feature, the third scale feature and the fourth scale feature to obtain the enhanced multi-scale feature.

[0017] In one possible implementation, the first set of side branches does not include strip convolution and dilated convolution, retains the structural information of the intermediate feature map of the original input, and generates the first scale feature;

[0018] The second set of side branches includes strip convolutional layers with kernel sizes of 1*3 and 3*1 connected in sequence, and dilated convolutions with an inflation rate of 3, which are used to process the intermediate feature maps to obtain the second scale features;

[0019] The third set of side branches includes strip convolutional layers with kernel sizes of 1*5 and 5*1 connected in sequence, and dilated convolution with an inflation rate of 5, which are used to process the intermediate feature map to obtain the third scale feature.

[0020] The fourth set of side branches includes: strip convolutional layers with kernel sizes of 1*7 and 7*1 connected in sequence, and dilated convolution with an inflation rate of 7, which are used to process the intermediate feature maps to obtain the fourth scale features.

[0021] In one possible implementation, the coarse prediction module includes three lightweight, cross-scale edge-assisted decoders.

[0022] In one possible implementation, the dynamic edge optimization module extracts edge responses from the upper-layer prediction results using the Laplacian operator to generate an edge confidence map; and performs element-wise interaction and weighting of the edge confidence map, the upper-layer prediction results, and the edge prior map with the feature map at the current scale to obtain multi-source edge guidance information; and uses a lightweight convolution module to jointly enhance the multi-source edge guidance information in both spatial and channel directions to generate an attention mask; and uses a channel attention mechanism to adaptively weight and fuse the multi-source edge guidance information in the channel dimension based on the attention mask to obtain intermediate feature results.

[0023] In one possible implementation, the cross-scale edge-assisted decoder includes a bidirectional feature fusion module and a fourth-order Runge-Kutta-based edge reconstruction module;

[0024] The bidirectional feature fusion module is used to input the intermediate feature results and the upper-layer prediction results into two branches respectively, calculate the query, key, and value using the two branches respectively, generate adaptive attention weights based on the query, key, and value through dual-path Softmax operation; and perform weighted fusion of the intermediate feature results and the upper-layer prediction results using the adaptive attention weights; and perform feature enhancement and boundary refinement on the weighted fused features using convolutional layers and Sigmoid activation to obtain edge-enhanced and refined features.

[0025] The edge reconstruction module is used to reconstruct the edge enhancement and refinement features using the fourth-order Runge-Kutta numerical integration method to obtain edge reconstruction features; and to jointly optimize the edge enhancement and refinement features and the edge reconstruction features, and generate an edge prediction map based on the optimization results.

[0026] In one possible implementation, the fourth-order Runge-Kutta edge reconstruction module includes four recursive update units, each of which is used to perform convolution, normalization, and nonlinear activation operations.

[0027] According to another aspect of this disclosure, a semantic segmentation system for camouflaged objects based on two-stage edge guidance is provided, comprising:

[0028] The SAM2 Hiera backbone network is used to process the camouflaged object image to be segmented and output multiple feature maps of different scales.

[0029] The gradient edge perception module is used to fuse local edge details and high-level semantic information between the highest-scale feature map and the lowest-scale feature map to obtain the edge prior map of the camouflaged object image.

[0030] The multi-granularity edge feature perception module is used to process feature maps at each scale separately and obtain the enhanced multi-scale features corresponding to each feature map.

[0031] The coarse prediction module is used to process each enhanced multi-scale feature to obtain a coarse segmentation result; the coarse segmentation result is used as the upper-level prediction result corresponding to the lowest-scale feature map.

[0032] The dynamic edge optimization module is used to process feature maps, edge prior maps, enhanced multi-scale features corresponding to each feature map, and upper-layer prediction results at each scale, and obtain intermediate feature results after dynamic edge optimization at each scale.

[0033] A cross-scale edge-assisted decoder is used to process the intermediate feature results and upper-level prediction results corresponding to each scale, and obtain the edge prediction map corresponding to each scale respectively; and to perform decoding processing based on the edge prediction map to obtain the prediction results of each scale; wherein, the prediction result corresponding to the highest scale is taken as the final semantic segmentation result.

[0034] According to another aspect of this disclosure, a computer-readable storage medium is provided, wherein a computer program is stored therein, and the computer program, when executed by a processor, implements the method described in any of the preceding claims.

[0035] This disclosure discloses a semantic segmentation method, system, and medium for camouflaged objects based on two-stage edge guidance. It utilizes SAM2 and Hiera backbone networks to extract multi-scale features and generates an edge prior map based on these features. Then, a multi-granularity edge feature perception module provides a feature foundation for two-stage edge guidance through its edge representation capabilities at different scales and directions, resulting in enhanced multi-scale features. Based on these enhanced multi-scale features and the edge prior map, and combined with the edge confidence map of the upper-layer prediction results, the features are dynamically optimized layer by layer from low to high scale, generating intermediate feature results containing fine-grained edge information. Furthermore, based on the dynamically optimized intermediate feature results, a cross-scale edge-assisted decoder is used to integrate and reconstruct the intermediate feature results. A layer-by-layer edge reconstruction feedback mechanism is used to achieve feature refinement and edge enhancement, ultimately outputting the semantic segmentation result of the camouflaged object, achieving clear separation and saliency enhancement of target edges in complex backgrounds.

[0036] This disclosure achieves layer-by-layer guidance and edge refinement from coarse to fine by constructing a two-stage dynamic edge optimization and edge reconstruction feedback mechanism, enabling the model to more accurately perceive the edge contours and salient regions of camouflaged targets. This disclosure has broad application prospects in tasks such as polyp segmentation, camouflaged plant identification, and target detection in agricultural scenarios, effectively improving segmentation accuracy and robustness in complex backgrounds.

[0037] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0038] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0039] Figure 1 This is a flowchart of a semantic segmentation method for camouflaged objects based on two-stage edge guidance in one embodiment of this disclosure;

[0040] Figure 2 This is a schematic diagram of the structure of a multi-granularity edge feature perception module in one embodiment of this disclosure;

[0041] Figure 3 This is a schematic diagram of the structure of the dynamic edge optimization module in one embodiment of this disclosure;

[0042] Figure 4 This is a schematic diagram of the structure of a cross-scale edge-assisted decoder in one embodiment of this disclosure;

[0043] Figure 5A This is a schematic diagram of a camouflaged object image to be segmented in one embodiment of this disclosure;

[0044] Figure 5B This is a schematic diagram of the truth labels of a camouflaged object image to be segmented in one embodiment of this disclosure;

[0045] Figure 5C This is a schematic diagram illustrating the effect of applying the dual-stage edge-guided semantic segmentation method for camouflaged objects according to embodiments of this disclosure;

[0046] Figure 5D This is a schematic diagram illustrating the effect of applying EG-SAM;

[0047] Figure 5E This is a schematic diagram illustrating the effect of applying COMPPrompter;

[0048] Figure 5F This is a schematic diagram illustrating the effect of applying CamoDiffusion;

[0049] Figure 6 This is a structural diagram of the entire model in one embodiment of this disclosure;

[0050] Figure 7 This is a schematic diagram of the structure of a camouflaged object semantic segmentation system based on two-stage edge guidance in one embodiment of this disclosure. Detailed Implementation

[0051] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0052] This disclosure addresses the shortcomings of camouflage object segmentation, such as blurred boundaries, lack of detail, and low segmentation accuracy. It proposes a semantic segmentation method, system, and medium for camouflage objects based on a two-stage edge-guided approach. By constructing two stages—dynamic edge optimization and edge reconstruction feedback—this disclosure achieves layer-by-layer guidance and edge refinement from coarse to fine, enabling the model to more accurately perceive the edge contours and salient regions of camouflaged targets. The technical solution of this disclosure has broad application prospects in tasks such as polyp segmentation, camouflaged plant identification, and target detection in agricultural scenarios, effectively improving segmentation accuracy and robustness in complex backgrounds.

[0053] The technical solution of this disclosure will be described below through specific embodiments.

[0054] like Figure 1The diagram shown is a flowchart of the semantic segmentation method for camouflaged objects based on two-stage edge guidance in this embodiment. The execution subject of this embodiment is a computing device or component with data processing capabilities. Specifically, the method of this embodiment may include the following steps:

[0055] S110. Input the camouflaged object image to be segmented into the SAM2 and Hiera backbone networks, and obtain multiple feature maps of different scales output by the SAM2 and Hiera backbone networks; input the highest scale feature map and the lowest scale feature map into the gradient edge perception module, and fuse local edge details and high-level semantic information through the gradient edge perception module to obtain the edge prior map of the camouflaged object image; input the multiple feature maps of different scales into the multi-granularity edge feature perception module respectively, and obtain the enhanced multi-scale features corresponding to each feature map output by the multi-granularity edge feature perception module.

[0056] The ViT image encoder in the Hiera backbone network is in a frozen state, and only the feature transfer function of the ViT image encoder is retained; a lightweight adapter module is inserted before each scale block in the Hiera backbone network to achieve fine-grained feature tuning.

[0057] The multi-granularity edge feature perception module includes an input feature preprocessing unit, a direction-aware convolutional branch group, and a gated channel attention unit.

[0058] The input feature preprocessing unit includes a convolutional layer with a kernel size of 1×1, which is used to perform channel compression and preliminary feature transformation on the input features to obtain a standardized intermediate feature map.

[0059] The orientation-aware convolutional branch group includes a first group of side branches, a second group of side branches, a third group of side branches, and a fourth group of side branches. The first group of side branches, the second group of side branches, the third group of side branches, and the fourth group of side branches process the intermediate feature map respectively, and obtain the first scale feature, the second scale feature, the third scale feature, and the fourth scale feature respectively. The orientation-aware convolutional branch group is also used to concatenate and fuse the first scale feature, the second scale feature, the third scale feature, and the fourth scale feature in the channel dimension to obtain a multi-granularity feature representation.

[0060] The first set of side branches does not include bar convolutions and dilated convolutions, retaining the structural information of the intermediate feature maps of the original input to generate first-scale features; the second set of side branches includes bar convolutional layers with kernel sizes of 1*3 and 3*1 connected in sequence, and dilated convolutions with a dilation rate of 3, used to process the intermediate feature maps to obtain second-scale features; the third set of side branches includes bar convolutional layers with kernel sizes of 1*5 and 5*1 connected in sequence, and dilated convolutions with a dilation rate of 5, used to process the intermediate feature maps to obtain third-scale features; the fourth set of side branches includes bar convolutional layers with kernel sizes of 1*7 and 7*1 connected in sequence, and dilated convolutions with a dilation rate of 7, used to process the intermediate feature maps to obtain fourth-scale features.

[0061] The gated channel attention unit is used to perform channel attention mapping on the multi-granularity feature representation through a convolution operation with a kernel size of 1×1, calculate the weights corresponding to the first scale feature, the second scale feature, the third scale feature and the fourth scale feature respectively, and use the weights to perform weighted fusion of the first scale feature, the second scale feature, the third scale feature and the fourth scale feature to obtain the enhanced multi-scale feature.

[0062] The reason for adopting a joint backbone structure of SAM2 and Hiera is that SAM2 has powerful global semantic modeling and cross-domain generalization capabilities, while Hiera can effectively capture hierarchical spatial details and local structural information. The combination of the two can maintain global consistency and enhance the expression of edge and detail features, thereby improving the model's ability to identify and segment camouflaged objects in complex backgrounds.

[0063] This step takes the camouflaged object image to be segmented as input, uses the SAM2 Hiera backbone to extract multi-scale features and edge priors of the image to be segmented, and uses the edge perception capabilities of the multi-granularity edge feature perception module at different scales and directions to provide a feature basis for subsequent two-stage edge guidance.

[0064] S120. The coarse prediction module is used to process each enhanced multi-scale feature to obtain a coarse segmentation result, and the coarse segmentation result is used as the upper-level prediction result corresponding to the lowest-scale feature map.

[0065] The coarse prediction module includes three lightweight, cross-scale edge-aided decoders.

[0066] S130. Perform edge-guided optimization and decoding layer by layer from the lowest-scale feature map to the highest-scale feature map to obtain the prediction results at each scale:

[0067] The current-scale feature map, edge prior map, enhanced multi-scale features corresponding to the current feature map, and upper-level prediction results are input into the dynamic edge optimization module, and the intermediate feature results after dynamic edge optimization are obtained from the output of the dynamic edge optimization module. The intermediate feature results corresponding to the current scale and the upper-level prediction results are input into the cross-scale edge auxiliary decoder, and the edge prediction map output by the cross-scale edge auxiliary decoder is obtained. The edge prediction map is decoded to obtain the prediction result at the current scale.

[0068] The dynamic edge optimization module extracts edge responses from the upper-layer prediction results using the Laplacian operator to generate an edge confidence map. It then performs element-wise interaction and weighting of the edge confidence map, the upper-layer prediction results, and the edge prior map with the feature map at the current scale to obtain multi-source edge guidance information. Finally, it uses a lightweight convolution module to jointly enhance the multi-source edge guidance information in both spatial and channel directions to generate an attention mask. Finally, it utilizes a channel attention mechanism to adaptively weight and fuse the multi-source edge guidance information along the channel dimension based on the attention mask to obtain intermediate feature results.

[0069] The cross-scale edge-assisted decoder includes a bidirectional feature fusion module and a fourth-order Runge-Kutta-based edge reconstruction module.

[0070] The bidirectional feature fusion module is used to input the intermediate feature results and the upper-layer prediction results into two branches respectively, and to calculate the query, key and value using the two branches respectively. Adaptive attention weights are generated based on the query, key and value through dual-path Softmax operation. The intermediate feature results and the upper-layer prediction results are then weighted and fused using the adaptive attention weights. Finally, the weighted and fused features are enhanced and the boundaries are refined using convolutional layers and sigmoid activation to obtain edge-enhanced and refined features.

[0071] The edge reconstruction module is used to reconstruct the edge enhancement and refinement features using the fourth-order Runge-Kutta numerical integration approach, obtaining edge reconstruction features; and to jointly optimize the edge enhancement and refinement features and the edge reconstruction features, generating an edge prediction map based on the optimization results. The fourth-order Runge-Kutta edge reconstruction module includes four recursive update units, each of which is used to implement convolution, normalization, and nonlinear activation operations.

[0072] The reason for introducing the Dynamic Edge Optimization Module (DEOM) is that this module achieves dynamic edge guidance of features by fusing the edge confidence of intermediate predictions with the initial edge prior during the layer-by-layer optimization process from low to high. This enables the model to adaptively focus on the true boundary region of the disguised object, effectively reducing the interference of blurry or false edges.

[0073] The reason for adopting the cross-scale edge-assisted decoder (CEAD) is that this module achieves hierarchical refinement and information interaction of features through cross-scale feature integration and edge reconstruction feedback mechanism during the decoding stage. It can fully integrate deep semantic information and shallow edge features, thereby gradually restoring the complete structure and true outline of the disguised object. In addition, the bidirectional feature fusion structure (BFF) in CEAD establishes an information interaction channel between deep features and shallow features, realizes the collaborative optimization of edges and semantics, effectively avoids the error accumulation problem caused by static edge priors, and further enhances the recognizability of the disguised object boundary and the generalization ability of the model.

[0074] This step, based on the enhanced multi-scale features and edge priors, and combining the edge confidence and edge priors of the upper-layer prediction results, performs layer-by-layer edge-guided optimization on the enhanced multi-scale features from low to high scales to obtain intermediate feature results after dynamic edge optimization; and, based on the intermediate feature results after dynamic edge optimization, a cross-scale edge-assisted decoder is used to integrate and reconstruct the multi-granularity features, and further optimizes the decoding process through a layer-by-layer edge reconstruction feedback mechanism, finally yielding the prediction results of each layer (i.e., the prediction results of each scale).

[0075] S140. The prediction result corresponding to the highest scale is taken as the final semantic segmentation result.

[0076] like Figure 6 As shown, this embodiment includes multi-scale feature extraction, a perception enhancement stage, a dynamic edge optimization stage, and an edge reconstruction-assisted decoding stage. The multi-scale feature extraction and perception enhancement stage uses the SAM2 Hiera backbone network to perform multi-scale feature extraction and perception optimization on the camouflaged object image to be segmented. Specifically, the input image is input to the model after being standardized in size. The ViT encoder in the Hiera backbone is frozen, and lightweight adapter modules are inserted before each layer to achieve efficient feature extraction and transfer of parameters. Based on the extracted multi-scale features and edge prior information, a multi-granularity edge feature perception module is constructed. Multi-scale convolution, directional feature filtering, and attention weighting mechanisms enhance the model's edge perception capabilities at different scales and directions, effectively improving the model's ability to recognize blurred, broken, and gradual boundaries, providing a feature foundation for subsequent dynamic edge optimization.

[0077] For the dynamic edge optimization stage, a Dynamic Edge Optimization Module (DEOM) is employed. This module uses the edge confidence of the upper-level prediction results as a guide and combines edge prior information to perform layer-by-layer optimization of multi-scale features from low to high scale. Through the synergistic effect of the edge confidence generation module, the attention modulation module, and the efficient channel attention enhancement module, dynamic adjustment and enhancement of edge regions in the feature space are achieved. This enables the model to adaptively focus on the true boundary region of the camouflaged object and suppress the interference of background noise.

[0078] For the edge reconstruction-assisted decoding stage, a cross-scale edge-assisted decoder (CEAD) is employed. This stage combines a bidirectional feature fusion module (BFF) with an edge reconstruction module (RK4) based on the fourth-order Runge-Kutta method to perform layer-by-layer decoding and edge feedback reconstruction of the dynamically optimized intermediate feature results. This stage enables cross-layer integration and semantic alignment of multi-scale features, and gradually restores the complete structure and fine boundaries of the camouflaged object under the guidance of edge feedback, ultimately outputting high-quality camouflaged object segmentation prediction results.

[0079] In step S110 above, to fully leverage the strong representational capabilities of SAM2 in transfer learning, the ViT image encoder part of the Hiera backbone network is frozen, retaining only its feature transfer function. Simultaneously, lightweight adapter modules are inserted before each multi-scale block of the Hiera structure to achieve efficient fine-grained feature tuning, thereby improving the model's adaptability in camouflaged target scenarios. Through this method, four hierarchically processed feature maps at different scales are obtained; the highest-scale feature map is then compared with the lowest-scale feature map. Figure 1 The input is fed into the gradient edge awareness module (EGA), which fuses local edge details with high-level semantic information to generate an edge prior map of the image to be segmented. Based on the four feature maps of different scales, they are respectively input into the multi-granularity edge feature awareness module (EFPM), such as... Figure 2 As shown, to enhance the edge perception capability of features at different scales and directions, the enhanced first feature map, second feature map, third feature map and fourth feature map are obtained.

[0080] In steps S120 and S130 above, the first feature map, second feature map, third feature map, and fourth feature map are first uniformly input into the coarse prediction module (CPM) to generate a preliminary prediction map, i.e., a coarse segmentation result; then, edge-guided optimization is performed layer by layer from the lowest scale (fourth feature map) to the highest scale (first feature map). Figure 3As shown, the Dynamic Edge Optimization Module (DEOM) of each layer takes the edge prior, the current layer enhanced feature map, and the upper layer prediction result as input (where the preliminary prediction map is used as the upper layer prediction result input of the lowest scale layer). It evaluates the edge confidence of the upper layer prediction result through the Laplacian operator, and combines the edge prior map and the perceptually enhanced feature map to achieve layer-by-layer edge-guided optimization, and obtains the intermediate feature result after dynamic edge optimization.

[0081] The Dynamic Edge Optimization Module (DEOM) establishes a layer-by-layer dynamic optimization mechanism between the current layer feature map and the previous layer prediction result. It extracts edge responses from the coarse prediction result using the Laplacian operator to generate an edge confidence map, reflecting the confidence level of the prediction boundary. Subsequently, this edge confidence map, the previous layer prediction result, and the edge prior are interacted and weighted element-wise with the multi-scale features of the current layer to form multi-source edge guidance information, achieving deep synergy between features and edge information. Based on this, a lightweight convolution module jointly enhances the fused features in both spatial and channel directions, generating an attention mask to strengthen the salient regions of the disguised target. Finally, an efficient channel attention (ECA) mechanism is introduced to adaptively weight the optimized features in the channel dimension, improving the ability to distinguish between edges and main body regions, thereby outputting a dynamically optimized feature map that provides more structurally guided semantic feature support for subsequent layer predictions.

[0082] Based on the intermediate feature results optimized by dynamic edges, the edge reconstruction stage achieves layer-by-layer decoding from low to high scales through an edge reconstruction feedback mechanism. For example... Figure 4 As shown, each layer of the cross-scale edge-assisted decoder (CEAD) takes the optimized features of the current layer and the prediction results of the upper layer as input, and uses a bidirectional feature fusion module (BFF) to realize the dynamic interaction of high and low layer semantics. It also combines a reconstruction unit based on fourth-order Runge-Kutta (RK4) to generate an edge prediction map of the corresponding scale. This prediction map is further fed back to optimize the segmentation results of the current layer and serves as the prediction feature input for the upper layer. The lowest scale layer takes the preliminary prediction map as input, and the segmentation output of the highest scale layer is the final prediction result of the model.

[0083] The aforementioned CEAD includes a bidirectional feature fusion module (BFF) and a fourth-order Runge-Kutta-based edge reconstruction module (RK4); BFF and RK4 are interconnected.

[0084] The BFF (Browser-Focused Front) mechanism establishes a bidirectional interaction mechanism between shallow and deep features, enabling the dynamic fusion of fine-grained semantic information and global structural information of the camouflaged target. Specifically, it inputs feature maps from different levels into two branches to calculate the query, key, and value respectively. Adaptive attention weights are generated through dual-path Softmax operations to characterize the complementary relationship between shallow and deep features. After the weighted features are fused, feature enhancement and boundary refinement are achieved through convolutional layers and sigmoid activation, thereby improving the model's ability to transmit information across scales in complex contexts.

[0085] The RK4 module described above utilizes the fourth-order Runge-Kutta numerical integral concept to perform multi-step dynamic reconstruction of the edge features of camouflaged objects, achieving progressive optimization of boundary prediction through multi-stage feature iteration updates. Specifically, the module consists of four recursive update units, each containing convolution, normalization, and non-linear activation operations to progressively adjust the amplitude and direction of the edge response, ultimately obtaining a more continuous and detailed edge reconstruction result. Simultaneously, it is jointly optimized with the fused features output by the BFF to generate the final prediction result of the camouflaged object.

[0086] The technical solution of this disclosure is implemented based on the PyTorch framework and trained using an NVIDIA RTX 3090 GPU. A pre-trained SAM2 Hiera is used as the backbone network. Model parameters are updated using the Adam optimizer, with 80 training epochs and an initial learning rate of 1. e-4 The batch size is 6, and cosine annealing is used for adjustment, with a minimum learning rate of 1. e-7 It is worth noting that during training and testing, the original view size was adjusted to 382. 382.

[0087] Dataset Splitting: This public case was evaluated on three widely used public datasets: CAMO, COD10K, and NC4K. CAMO contains 1250 camouflaged images, with 1000 used as training images and 250 as test images. CHAMELEON is a small dataset collected from the internet via Google search, consisting of 76 manually labeled images. COD10K is a more challenging large-scale dataset containing 10,000 images, including 5066 camouflaged images, 3000 background images, and 1934 uncamouflaged images. NC4K is another COD dataset used for testing, including 4121 test images.

[0088] Table 1 presents the quantitative comparison results with 19 state-of-the-art camouflaged object semantic segmentation models, including SINet, BGNet, ZoomNet, FEDER, HitNet, FSPNet, EAMNet, GLCONet, ZoomNetxt, VSCode-T, RISNet, BDCL-Net, CamoDiffusin, SAM, EG-SAM, COMPOMPter, SAM2, SAM2-DFBCNet, and HFS-SAM2.

[0089] Table 1

[0090] Figures 5A-5F The document presents a qualitative performance comparison chart showing the technical solution of this disclosure embodiment with three state-of-the-art models (EG-SAM, COMPromter, and CamoDiffusion). Specifically, Figure 5A This is a schematic diagram of a camouflaged object image to be segmented in one embodiment of this disclosure; Figure 5B This is a schematic diagram of the truth labels of a camouflaged object image to be segmented in one embodiment of this disclosure; Figure 5C This is a schematic diagram illustrating the effect of applying the dual-stage edge-guided semantic segmentation method for camouflaged objects according to embodiments of this disclosure; Figure 5D This is a schematic diagram illustrating the effect of applying EG-SAM; Figure 5E This is a schematic diagram illustrating the effect of applying COMPPrompter; Figure 5F This is a schematic diagram illustrating the effect of applying CamoDiffusion. As can be seen from the diagram, the segmentation effect of the disclosed solution is optimal.

[0091] Based on the same inventive concept, this disclosure provides a semantic segmentation system for camouflaged objects based on two-stage edge guidance. The steps performed by the components of this system are the same as or similar to those described above, therefore, similar details will not be repeated. Figure 7 As shown, the semantic segmentation system for camouflaged objects based on two-stage edge guidance in this embodiment includes:

[0092] SAM2 and Hiera backbone network 710 are used to process the camouflaged object image to be segmented and output multiple feature maps of different scales.

[0093] The gradient edge perception module 720 is used to fuse local edge details and high-level semantic information between the highest-scale feature map and the lowest-scale feature map to obtain the edge prior map of the camouflaged object image.

[0094] The multi-granularity edge feature perception module 730 is used to process the feature maps at each scale separately and obtain the enhanced multi-scale features corresponding to each feature map.

[0095] The coarse prediction module 740 is used to process each enhanced multi-scale feature to obtain a coarse segmentation result; the coarse segmentation result is used as the upper-level prediction result corresponding to the lowest-scale feature map.

[0096] The dynamic edge optimization module 750 is used to process the feature maps, edge prior maps, enhanced multi-scale features corresponding to each feature map, and upper-level prediction results at each scale, and obtain the intermediate feature results after dynamic edge optimization at each scale.

[0097] A cross-scale edge-assisted decoder 760 is used to process the intermediate feature results and upper-level prediction results corresponding to each scale, and obtain the edge prediction map corresponding to each scale respectively; and to perform decoding processing based on the edge prediction map to obtain the prediction results of each scale; wherein the prediction result corresponding to the highest scale is used as the final semantic segmentation result.

[0098] Based on the same inventive concept, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the scheme of any of the above embodiments. Therefore, similar details will not be repeated.

[0099] The various embodiments of the techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0103] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0104] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0105] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0106] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A semantic segmentation method for camouflaged objects based on two-stage edge guidance, characterized in that, include: The camouflaged object image to be segmented is input into the SAM2 and Hiera backbone networks, and multiple feature maps of different scales output by the SAM2 and Hiera backbone networks are obtained. The highest-scale feature map and the lowest-scale feature map are input into the gradient edge perception module, and the gradient edge perception module fuses local edge details and high-level semantic information to obtain the edge prior map of the camouflaged object image. The multiple feature maps of different scales are input into the multi-granularity edge feature perception module, and the enhanced multi-scale features corresponding to each feature map output by the multi-granularity edge feature perception module are obtained. The coarse prediction module is used to process each enhanced multi-scale feature to obtain a coarse segmentation result, and the coarse segmentation result is used as the upper-level prediction result corresponding to the lowest scale feature map. Edge-guided optimization and decoding are performed layer by layer from the lowest-scale feature map to the highest-scale feature map to obtain the prediction results at each scale: The current-scale feature map, edge prior map, enhanced multi-scale features corresponding to the current feature map, and upper-layer prediction results are input into the dynamic edge optimization module, and the intermediate feature results after dynamic edge optimization are obtained from the output of the dynamic edge optimization module. The intermediate feature results and upper-level prediction results corresponding to the current scale are input into the cross-scale edge-assisted decoder, and the edge prediction map output by the cross-scale edge-assisted decoder is obtained. Based on the edge prediction map, the prediction result at the current scale is obtained by decoding. The prediction result corresponding to the highest scale is taken as the final semantic segmentation result; Specifically, the dynamic edge optimization module extracts edge responses from the upper-layer prediction results using the Laplacian operator to generate an edge confidence map; it then performs element-wise interaction and weighting of the edge confidence map, the upper-layer prediction results, and the edge prior map with the feature map at the current scale to obtain multi-source edge guidance information; it further enhances the multi-source edge guidance information in both spatial and channel directions using a lightweight convolution module to generate an attention mask; and finally, it uses a channel attention mechanism to adaptively weight and fuse the multi-source edge guidance information in the channel dimension based on the attention mask to obtain intermediate feature results. The cross-scale edge-assisted decoder includes a bidirectional feature fusion module and a fourth-order Runge-Kutta-based edge reconstruction module.

2. The method according to claim 1, characterized in that, The ViT image encoder in the Hiera backbone network is in a frozen state, and only the feature transfer function of the ViT image encoder is retained; a lightweight adapter module is inserted before each scale block in the Hiera backbone network to achieve fine-grained feature tuning.

3. The method according to claim 1, characterized in that, The multi-granularity edge feature perception module includes an input feature preprocessing unit, a direction-aware convolutional branch group, and a gated channel attention unit. The input feature preprocessing unit includes a convolutional layer with a kernel size of 1×1, which is used to perform channel compression and preliminary feature transformation on the input features to obtain a standardized intermediate feature map. The orientation-aware convolutional branch group includes a first group of side branches, a second group of side branches, a third group of side branches, and a fourth group of side branches. The first group of side branches, the second group of side branches, the third group of side branches, and the fourth group of side branches process the intermediate feature map respectively and obtain the first scale feature, the second scale feature, the third scale feature, and the fourth scale feature respectively. The orientation-aware convolutional branch group is also used to splice and fuse the first-scale features, second-scale features, third-scale features and fourth-scale features in the channel dimension to obtain a multi-granularity feature representation; The gated channel attention unit is used to perform channel attention mapping on the multi-granularity feature representation through a convolution operation with a kernel size of 1×1, calculate the weights corresponding to the first scale feature, the second scale feature, the third scale feature and the fourth scale feature respectively, and use the weights to perform weighted fusion of the first scale feature, the second scale feature, the third scale feature and the fourth scale feature to obtain the enhanced multi-scale feature.

4. The method according to claim 3, characterized in that, The first set of side branches does not include strip convolution and dilated convolution, but retains the structural information of the intermediate feature map of the original input to generate the first scale feature; The second set of side branches includes strip convolutional layers with kernel sizes of 1*3 and 3*1 connected in sequence, and dilated convolutions with an inflation rate of 3, which are used to process the intermediate feature maps to obtain the second scale features; The third set of side branches includes strip convolutional layers with kernel sizes of 1*5 and 5*1 connected in sequence, and dilated convolution with an inflation rate of 5, which are used to process the intermediate feature map to obtain the third scale feature. The fourth set of side branches includes: strip convolutional layers with kernel sizes of 1*7 and 7*1 connected in sequence, and dilated convolution with an inflation rate of 7, which are used to process the intermediate feature maps to obtain the fourth scale features.

5. The method according to claim 1, characterized in that, The coarse prediction module includes three lightweight, cross-scale edge-assisted decoders.

6. The method according to claim 1, characterized in that, The bidirectional feature fusion module is used to input the intermediate feature results and the upper-level prediction results into two branches respectively, and use the two branches to calculate the query, key and value respectively. Adaptive attention weights are generated based on the query, key and value through dual-path Softmax operation. Furthermore, adaptive attention weights are used to weight and fuse intermediate feature results and upper-level prediction results; Furthermore, convolutional layers and sigmoid activation are used to enhance and refine the weighted fused features, resulting in edge-enhanced and refined features. The edge reconstruction module is used to reconstruct the edge enhancement and refinement features using the fourth-order Runge-Kutta numerical integration method to obtain edge reconstruction features; and to jointly optimize the edge enhancement and refinement features and the edge reconstruction features, and generate an edge prediction map based on the optimization results.

7. The method according to claim 6, characterized in that, The fourth-order Runge-Kutta edge reconstruction module includes four recursive update units, each of which is used to implement convolution, normalization, and nonlinear activation operations.

8. A semantic segmentation system for camouflaged objects based on two-stage edge guidance, characterized in that, include: SAM2 and Hiera backbone network are used to process the camouflaged object image to be segmented and output multiple feature maps of different scales. The gradient edge perception module is used to fuse local edge details and high-level semantic information between the highest-scale feature map and the lowest-scale feature map to obtain the edge prior map of the camouflaged object image. The multi-granularity edge feature perception module is used to process feature maps at each scale separately and obtain the enhanced multi-scale features corresponding to each feature map. The coarse prediction module is used to process each enhanced multi-scale feature to obtain a coarse segmentation result; the coarse segmentation result is used as the upper-level prediction result corresponding to the lowest-scale feature map. The dynamic edge optimization module is used to process feature maps, edge prior maps, enhanced multi-scale features corresponding to each feature map, and upper-layer prediction results at each scale, and obtain intermediate feature results after dynamic edge optimization at each scale. A cross-scale edge-assisted decoder is used to process the intermediate feature results and upper-level prediction results corresponding to each scale, and obtain the edge prediction map corresponding to each scale respectively; and to perform decoding processing based on the edge prediction map to obtain the prediction results of each scale; wherein, the prediction result corresponding to the highest scale is taken as the final semantic segmentation result. Specifically, the dynamic edge optimization module extracts edge responses from the upper-layer prediction results using the Laplacian operator to generate an edge confidence map; it then performs element-wise interaction and weighting of the edge confidence map, the upper-layer prediction results, and the edge prior map with the feature map at the current scale to obtain multi-source edge guidance information; it further enhances the multi-source edge guidance information in both spatial and channel directions using a lightweight convolution module to generate an attention mask; and finally, it uses a channel attention mechanism to adaptively weight and fuse the multi-source edge guidance information in the channel dimension based on the attention mask to obtain intermediate feature results. The cross-scale edge-assisted decoder includes a bidirectional feature fusion module and a fourth-order Runge-Kutta-based edge reconstruction module.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images