Camouflage target segmentation method based on SAM-guided reversible unfolding network

Abstract

This invention relates to the field of computer vision and camouflage target segmentation, specifically proposing a camouflage target segmentation method based on a SAM-guided reversible unfolding network. First, a foreground space prior map, a background space prior map, and a high-quality SAM pseudo-mask are constructed using the Segmentation Model (SAM). Prior redundancy is eliminated and the separability between the foreground and background is enhanced through low-dimensional orthogonal subspace projection. Then, pixel-level and gradient-level feature fitting terms and SAM subspace prior constraints are fused to construct an overall objective function. This objective function is then unfolded into a multi-stage alternating iterative process of a foreground optimization submodule (SFOS) and a background optimization submodule (SBOS), progressively refining the foreground and background feature maps. Finally, a camouflage target segmentation mask is generated based on the iteratively optimized foreground feature map. Through large-model prior guidance, two-level feature modeling, and multi-stage unfolding optimization, the completeness and accuracy of camouflage target segmentation are significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and particularly to the task of camouflaged target segmentation. Specifically, it is a method and system for camouflaged target segmentation that integrates the strong visual prior of Segment Anything Model (SAM) with a reversible unfolding framework. It is applicable to image segmentation scenarios such as camouflaged target segmentation, polyp image segmentation, and transparent target segmentation. Background Technology

[0002] Camouflage target segmentation (COS) is a fundamental and highly challenging task in computer vision. Its core objective is to accurately segment target regions from images that are highly visually integrated with the background. It is widely used in military reconnaissance, medical image analysis, and industrial inspection. The core challenges of camouflage target segmentation are mainly reflected in two aspects: First, the foreground and background of camouflage targets have inherent similarities in visual features, making it difficult to extract effective discriminative features; second, the subtle edges, textures, and other fine-grained features of camouflage targets are easily obscured by the background, resulting in incomplete segmentation results and blurred edges.

[0003] With the development of deep learning technology, significant progress has been made in camouflage target segmentation methods based on deep networks, and various deep models have been applied to this task to improve segmentation performance. However, existing methods still have many problems that need to be solved: First, some methods rely on hard masks to establish the association between the foreground and the original image. The rigid constraints brought by hard masks will cut off the subtle discriminative features of the camouflage target, resulting in the loss of target edges and details. Second, existing methods lack effective prior guidance from large-scale visual base models, and the regularization terms are mostly manually designed, resulting in limited generalization ability. Third, most methods only perform pixel-level data fitting in the objective function, ignoring the modeling of gradient-level fine-grained features (such as edges and textures) that are crucial to distinguishing camouflage targets from the background, and thus failing to capture the detailed structure of the camouflage target.

[0004] Segment Anything Model (SAM), a powerful visual foundational model proposed in recent years, has been trained on ultra-large-scale masked data and possesses excellent zero-shot generalization ability and universal segmentation performance, providing effective visual priors for various segmentation tasks. SAM can output the soft mask probability of targets in an image, providing reliable prior information for camouflaged target segmentation. However, current technologies only utilize SAM priors at the level of generating a single pseudo-mask, failing to explore the potential of SAM to provide spatially adaptive priors for the unfolding framework of camouflaged target segmentation. Furthermore, there is a lack of methods to effectively integrate SAM's strong visual priors with model-driven reversible unfolding frameworks, thus failing to fully exploit the prior value of SAM for camouflaged target segmentation.

[0005] Therefore, there is an urgent need for a novel camouflage target segmentation method that can effectively integrate the advantages of SAM's strong visual priors and reversible unfolding framework, solve the problems of hard mask constraints, lack of large model priors, and neglect of gradient feature modeling in existing methods, improve the accuracy, robustness and generalization ability of camouflage target segmentation, and adapt to complex camouflage target segmentation tasks such as small targets, multiple targets, and degenerate scenes. Summary of the Invention

[0006] To address the aforementioned technical problems of existing camouflage target segmentation methods, this invention proposes a camouflage target segmentation method based on a SAM-guided reversible unfolding network. The core idea is to deeply integrate the strong visual prior of SAM with the reversible unfolding framework. The camouflage target segmentation model is reconstructed from three dimensions: spatial prior guidance, pixel-gradient dual-level feature modeling, and multi-stage reversible unfolding optimization. This eliminates the rigid constraints of hard masks, effectively injects large-scale visual priors into the segmentation framework, strengthens the modeling capability for fine-grained features, and ultimately achieves accurate and robust segmentation of camouflage targets. The term "reversible unfolding" in this application does not refer to a strictly reversible neural network, nor does it require that every network layer mapping satisfy a bidirectional one-to-one correspondence; rather, it refers to an alternating iterative optimization framework obtained by unfolding a unified objective function. Specifically, firstly, both the foreground optimization submodule SFOS and the background optimization submodule SBOS ​​are derived from the same overall objective function, and there is a clear correspondence between the forward updates of each stage of the network and the corresponding optimization sub-problems; secondly, the stage updates of the foreground feature map and the background feature map can be derived from the corresponding optimization sub-problems to obtain closed-form update formulas, which have a clear mathematical origin; thirdly, the output of each stage can be traced back to the data fitting terms, prior constraint terms and the state of the previous stage in the objective function, thus making the entire segmentation framework highly interpretable and backtrackable.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] A camouflaged target segmentation method based on a SAM-guided reversible unfolding network includes the following steps:

[0009] Step 1: Receive and preprocess the input camouflaged target image to obtain the original image;

[0010] Step 2: Input the original image into the Segmentation Model (SAM), construct and optimize the foreground space prior map, background space prior map, and high-quality SAM pseudo-mask to achieve adaptive foreground-background separation;

[0011] Step 3: Perform dimensionality reduction and orthogonalization on the foreground space prior map and the background space prior map to eliminate feature redundancy and keep the foreground and background subspaces orthogonal, so as to obtain the orthogonal reconstructed foreground prior map and the orthogonal reconstructed background prior map.

[0012] Step 4: Based on the original image, the gradient feature map of the original image, the foreground space prior map, the background space prior map, the orthogonally reconstructed foreground prior map, the orthogonally reconstructed background prior map, and the high-quality SAM pseudomask, construct the objective function and expand the objective function as follows: The foreground and background update processes are alternately iterated in stages, and foreground feature maps and background feature maps are obtained step by step.

[0013] Step 5, place the first Input foreground feature map obtained from the first iteration The convolutional layer is activated by the Sigmoid function to obtain a segmentation probability map. The segmentation probability map is then binarized to obtain the final segmentation mask of the disguised target.

[0014] The preprocessing step in step one involves normalizing the input RGB camouflage target image to a uniform image size. ,in, Indicates the image height. The image width is represented by 3, and the number of image channels corresponds to the three RGB color channels. A zero-padding strategy is used to ensure size consistency in subsequent gradient extraction and convolution operations. The preprocessed image is denoted as the original image. ;

[0015] The specific implementation steps for step two are as follows:

[0016] Step 1: Extract the original image conduct Geometric augmentation, including flipping, rotating, and scaling, yields multiple augmented views. To increase the number of views;

[0017] Step 2: Input each augmented view into SAM to obtain the foreground soft response map under a single view. Background soft response map in single view And the native segmentation mask, and then the inverse transformation is used to map these outputs back to the original image coordinate system to ensure that the outputs of all views are aligned in the same space;

[0018] Step 3: Calculate the foreground-background prediction entropy for each pixel location;

[0019] Step 4: Weight and fuse the foreground / background soft response maps of each view using pixel-level and image-level weights respectively to obtain the final foreground space prior map. and background space prior graph ;

[0020] Step 5: Weighted fusion of the original segmentation masks from multiple views yields a soft-fused pseudo-mask. This pseudo-mask is then processed through thresholding, space filling, and maximum connected component filtering to finally obtain a high-quality SAM pseudo-mask. This is used for subsequent subspace prior constraints.

[0021] Step three involves dimensionality reduction and orthogonalization, including:

[0022] Step 1: Create a two-dimensional foreground space prior image. and background space prior graph Flatten them into lengths of A one-dimensional vector, denoted as the prior flattened vector in the foreground space. and background space prior flattened vector Then perform min-max normalization to [0,1] and concatenate them into a joint feature matrix. ,in This represents the total number of pixels in the image. Indicates the image height. Indicates the image width;

[0023] Step 2: For the joint feature matrix After decentralization, the covariance matrix is ​​calculated, and the projection matrix is ​​obtained through eigenvalue decomposition. Then and Projecting onto a low-dimensional subspace yields a low-dimensional foreground feature vector. and low-dimensional background feature vectors ;

[0024] Step 3, for and Perform Gram-Schmidt orthogonalization to obtain mutually orthogonal low-dimensional orthogonal foreground vectors. and low-dimensional orthogonal background vector ;

[0025] Step 4, obtain and Through projection matrix Inversely projected back to the original pixel space and reshaped to a size of From the two-dimensional feature map, an orthogonal reconstructed foreground prior map is obtained. and orthogonal reconstruction background prior graph .

[0026] In step four, we first construct an objective function that integrates the pixel-gradient two-level feature fitting term and the SAM subspace prior constraint term, which serves as the basis for subsequent optimization. Specifically:

[0027] Step 1, Two-level feature fitting term This includes pixel-level data fitting and gradient-level feature fitting, ensuring that the weighted foreground and background can reconstruct the original image and its gradient features. The formula is:

[0028] .

[0029] in Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. , These are the original image, foreground feature map, and background feature map, respectively. Foreground space prior graph, This is the background space prior image. For Hadama accumulation, This is the gradient feature map of the original image. For gradient extraction operators, The square of the L2 norm;

[0030] Step 2: Define the foreground prior graph reconstructed from orthogonal priors. Orthogonal reconstruction of background prior graph Guided foreground response map and background response map :

[0031] .

[0032] in, This represents a fixed response mapping operator used to map a foreground or background feature map weighted by prior knowledge to a single-channel response space consistent with the pseudomask.

[0033] Step 3: Construct subspace prior constraints :

[0034] .

[0035] in, Foreground response map, For background response image, For high-quality SAM pseudomasks;

[0036] Step 4, Overall Objective Function: ,in The fixed weights of the subspace prior constraints control the contribution of the SAM prior.

[0037] Among them, the gradient feature map of the original image The gradients in the x / y directions are obtained by extracting them using the Sobel operator. The specific process is as follows:

[0038] Step 1: Define a 3×3 Sobel horizontal convolution kernel. Convolution kernels perpendicular to Sobel Sobel convolution is performed on the three RGB channels of the normalized original image to obtain the gradient response in the x / y direction of each channel;

[0039] Step 2: Average and fuse the gradients from the three channels to obtain the global gradient value in the x-direction. Global y-axis gradient value ;

[0040] Step 3, , Standardize to [0,1], and splice according to channel. Original image gradient feature map .

[0041] Step four, specifically the alternating foreground and background update sections, comprises two alternating sub-modules: the SAM-guided foreground optimization sub-module SFOS and the SAM-guided background optimization sub-module SBOS. The specific implementation of SFOS is as follows:

[0042] Step 1, construct the first The phase prospect optimization sub-problem, with the fixed first... Stage background feature map Ignoring constant terms irrelevant to the foreground, the overall objective function is simplified to a function that depends only on the foreground feature map. The optimized formula;

[0043] Step 2: Solve the optimization subproblem based on the proximal gradient algorithm, and derive the... Stage Prospect Feature Map Closed-form update solution:

[0044] .

[0045] in:

[0046] The coefficient matrix for the prospect optimization subproblem. This is the weight matrix of the foreground feature map from the previous stage.

[0047] The foreground residual term is composed of both pixel-level and gradient-level fitting terms. To reconstruct the foreground prior graph from orthogonal and high-quality SAM pseudomask The prior alignment terms that are jointly formed,

[0048] The fixed weights are the prior constraints for the subspace. It is the first Stage foreground feature map, initial value =0, which is a matrix containing all zeros;

[0049] The specific implementation of SBOS ​​is as follows:

[0050] Step 1, construct the first The phase background optimization sub-problem, with the fixed first... Stage Prospect Feature Map Ignoring constant terms irrelevant to the background, the overall objective function is simplified to a function that depends only on the background feature map. The optimized formula;

[0051] Step 2: Solve the optimization subproblem based on the proximal gradient algorithm, and derive the... Stage background feature map Closed-form update solution:

[0052] .

[0053] in:

[0054] Optimize the coefficient matrix of the subproblem in the background. This is the weight matrix of the background feature map from the previous stage.

[0055] The background residual term is composed of pixel-level fitting terms and gradient-level fitting terms. To reconstruct the background prior graph using orthogonal methods and high-quality SAM pseudomask complement The prior alignment terms that are jointly formed, The fixed weights are the prior constraints for the subspace. It is the first Stage background feature map, initial value =0, which is a matrix of all zeros.

[0056] The SFOS and SBOS ​​submodules adopt The update rules for alternating phases are as follows:

[0057] .

[0058] in , =0、 =0 is the initial all-zero matrix. The number of network stages represents the foreground and background outputs of the previous stage, which serve as the inputs for the next stage, progressively refining the feature maps.

[0059] The SAM-guided reversible unfolding segmentation module employs a stage-weighted segmentation loss function to guide iterative optimization. This segmentation loss function is the weighted sum of the segmentation losses at each stage, with stage weights set exponentially decaying. The formula is as follows:

[0060] .

[0061] The loss function will include all The losses from each stage are weighted and summed, with stage weights set exponentially decaying to focus the model on later stages where the results are more refined.

[0062] It is the foreground feature map of the k-th stage The generated segmentation mask, It is the Sigmoid activation function. It is a 1×1 convolutional layer;

[0063] It is the true segmentation mask used to disguise the target;

[0064] It is a weighted binary cross-entropy loss, used to solve the class imbalance problem in camouflaged target segmentation tasks;

[0065] It is the weighted intersection-union loss, used to measure the degree of overlap between the predicted mask and the true mask;

[0066] It is the first The stage weights corresponding to the first-order loss decay exponentially with the number of stages, allowing the model to focus on later stages with more refined foreground / background features.

[0067] The specific process of step five is as follows: The final foreground feature map obtained after stage iterative optimization is input into a 1×1 convolutional layer, and a pixel-level segmentation probability map is generated by the Sigmoid activation function. The probability map is then binarized to obtain the final segmentation mask of the disguised target.

[0068] This invention also constructs a camouflaged target segmentation system based on a SAM-guided reversible unfolding network. This system comprises five core functional modules: a camouflaged image input module, a SAM space prior construction module, a low-dimensional orthogonal subspace projection module, a SAM-guided reversible unfolding segmentation module, and a segmentation result output module.

[0069] The camouflage image input module is used to receive and preprocess the input camouflage target image;

[0070] The SAM spatial prior construction module is used to build and optimize the foreground spatial prior map and background spatial prior map from the native output of Segment Anything Model (SAM), replacing hard mask to achieve adaptive foreground-background separation;

[0071] The low-dimensional orthogonal subspace projection module is used to reduce the dimensionality and orthogonalize the foreground and background space prior maps, eliminate feature redundancy and force the foreground and background subspaces to be strictly orthogonal.

[0072] The SAM-guided reversible unfolding segmentation module is used to fuse pixel-gradient dual-level feature fitting with subspace prior constraints, and achieves progressive segmentation of camouflaged targets through alternating iterative foreground optimization and background optimization sub-modules.

[0073] The segmentation result output module is used to generate the final segmentation mask of the camouflaged target based on the iteratively optimized foreground feature map;

[0074] The modules are connected sequentially according to the data processing flow, with the output of the previous module serving as the input of the next module. Together, they achieve a complete segmentation process from camouflaged image input to final segmentation mask output. The specific implementation methods and core improvements of each module are as follows:

[0075] The camouflage image input module, as the initial step in the entire segmentation method, primarily functions to perform standardization preprocessing on the input camouflage target image. This ensures the consistency of size and numerical stability in subsequent module operations, laying the foundation for the subsequent segmentation process. This module first receives an RGB format camouflage target image of arbitrary size, denoted as the camouflage target image. ,in , Here, represents the image height and width, respectively, and 3 represents the RGB three-channel dimension. The RGB pixel values ​​are then normalized to the [0,1] range to avoid numerical overflow affecting subsequent calculations. The normalization formula is:

[0076] .

[0077] in, To normalize the image at position Place, No. The pixel values ​​of each channel. To disguise target images In position Place, No. The pixel values ​​of each channel. The row and column coordinates of the image. This is the color channel index, with values ​​of R, G, and B. For image In the Two-dimensional image corresponding to each channel For the first The minimum value of all pixels in each channel. For the first The maximum value of all pixels in each channel.

[0078] After channel normalization, this module scales the normalized image to a fixed size set in the experiment and uses a zero-padding strategy to fill the image edges, ensuring that the output feature maps of subsequent gradient extraction and convolution operations are exactly the same size as the input image. Finally, the preprocessed original image is output, denoted as the original image. , which serves as the input to the SAM space prior construction module and the SAM-guided reversible unfolding segmentation module.

[0079] The improvements in this module are as follows: by using a unified normalization, fixed-size scaling and boundary completion strategy, it provides stable input for subsequent spatial prior modeling and multi-stage iterative optimization, avoiding feature distortion and training instability caused by inconsistent sizes or fluctuations in numerical range, thereby improving the engineering feasibility and cross-scene adaptability of the entire segmentation process.

[0080] The core function of the SAM spatial prior construction module is to construct a foreground space prior graph from the native output of SAM. Background space prior graph and high-quality SAM pseudomask It replaces the hard mask in traditional methods with adaptive pixel-wise weights to achieve foreground-background separation without hard constraints. At the same time, it optimizes the quality of the prior image through a multi-augmentation fusion strategy, effectively preserving the subtle features of the camouflaged target.

[0081] This module first processes the original image. Inputting a pre-trained SAM model yields foreground soft response maps and background soft response maps under a single view, denoted as follows: and Both are related to Two-dimensional feature maps of uniform size, wherein, The continuous probability response that represents a pixel belonging to a camouflaged foreground is used to characterize the pixel. This module represents the continuous probability response of a pixel belonging to the background. To improve the stability of the SAM output in complex camouflage scenes, this module further refines the original image. generate An augmented view, denoted as ,in, Indicates the first A geometric transformation operation Indicates the first An augmented view corresponding to a geometric transformation, the geometric transformation including flipping, rotating, and scaling.

[0082] Input each augmented view into SAM to get the first... Foreground soft response map of an augmented view , No. The background soft response map of the augmented view Then, the outputs of each augmented view are mapped back to the original image coordinate system through inverse transformation to obtain...

[0083] .

[0084] in, For the first Foreground soft response map of an augmented view, For the first The background soft response map of the augmented view. For the first Geometric transformation operations The inverse transformation operation is used to map the augmented view coordinates back to the original image coordinate system. , For the first The foreground soft response map after mapping the augmented view back to the original image coordinate system, the first The background soft response map is generated after the augmented view is mapped back to the original image coordinate system.

[0085] Since the output quality of SAM varies across different augmented views, direct mean fusion would introduce noise. Therefore, this invention employs an uncertainty-based weighted fusion strategy to improve prior quality. Specifically, the first... The foreground-background prediction entropy of the augmented view at pixel position (x, y) is:

[0086] .

[0087] in, Characterizing the uncertainty at this location, For the first The foreground soft response value at pixel position (x, y) after the augmented view is mapped back to the original image. For the first The background soft response value at pixel position (x,y) after the augmented view is mapped back to the original image.

[0088] The smaller the entropy value, the more stable the foreground-background determination at that location in the current view; the larger the entropy value, the greater the uncertainty at that location.

[0089] Based on this, pixel-level fusion weights are defined as follows:

[0090] .

[0091] in, For the first Pixel-level blending weights at pixel position (x, y) for each augmented view. It is an exponential function. For the first The foreground-background prediction entropy of an augmented view at pixel location (x,y).

[0092] Meanwhile, to reflect the overall reliability of the entire view, image-level quality weights are defined:

[0093] .

[0094] in, For the first Image-level quality weights for each augmented view The averaging coefficient is the average value applied to all pixels of the entire image. For the first The foreground-background prediction entropy of an augmented view at pixel location (x,y).

[0095] Based on this, the foreground space prior map and the background space prior map are defined as the weighted fusion results of the multi-view soft response:

[0096] .

[0097] .

[0098] in, Foreground space prior graph, This is the background space prior image. It represents the Hadamah accumulation. For the first Image-level quality weights for each augmented view For the first Pixel-level fusion weight map of each augmented view For the first The foreground soft response map after mapping the augmented view back to the original image coordinate system. For the first The background soft response map is generated after the augmented view is mapped back to the original image coordinate system.

[0099] Furthermore, a similar weighted fusion is performed on the native segmentation mask of the augmented view to obtain a soft-fused pseudo-mask. After post-processing operations such as thresholding, space filling, and maximum connected component filtering, a high-quality SAM pseudo-mask is finally obtained. This is used for subsequent subspace prior constraints.

[0100] Finally, the module outputs a foreground space prior map. Background space prior graph and high-quality SAM pseudomask .in, and Input a low-dimensional orthogonal subspace projection module and a SAM-guided reversible unfolding and segmentation module. Input the subspace prior constraint construction process in the low-dimensional orthogonal subspace projection module, and simultaneously input the SAM-guided reversible expansion and segmentation module.

[0101] The improvements in this module are as follows: On the one hand, it utilizes the native output of SAM to construct soft priors for the foreground / background, avoiding the problem of hard masks forcibly truncating the subtle edges and weak texture information of the camouflaged target; on the other hand, it improves the quality and stability of the priors through a multi-augmentation fusion strategy, enabling subsequent unfolding optimization to perform foreground / background separation under more reliable spatial cues, which is especially suitable for camouflage scenes with blurred boundaries, mixed textures, and low contrast.

[0102] The core function of the low-dimensional orthogonal subspace projection module is to and Dimensionality reduction, redundancy removal, and orthogonalization are performed to eliminate feature redundancy in the spatial prior and force strict orthogonality between the foreground and background subspaces. Simultaneously, subspace prior constraints are constructed based on the projection results, effectively injecting the large-scale visual prior of the SAM model into the subsequent reversible unfolding segmentation framework. The core improvement of this invention lies in employing a combined strategy of linear PCA dimensionality reduction and Gram-Schmidt orthogonalization. The entire process involves fixed mathematical operations with no learnable parameters, and can be pre-computed on the CPU. This module first transforms the two-dimensional foreground space prior map... and background space prior graph Flatten them into lengths of One-dimensional vector: Foreground space prior flattened vector and background space prior flattened vector ,in This represents the total number of pixels in the image. Indicates the image height. This represents the image width. Then, respectively... and After performing min-max normalization to the [0,1] interval, the joint feature matrix is ​​constructed by concatenating the columns. To eliminate the effect of the overall bias, the joint characteristic matrix is... Decentralization is performed, and the mean vector is... Then the decentralized joint feature matrix can be written as Then calculate the covariance matrix of the decentralized joint feature matrix.

[0103]

[0104] in The transpose operation represents the matrix transformation operation, applied to the covariance matrix. Perform eigenvalue decomposition, and take the eigenvectors corresponding to the first d largest eigenvalues ​​to form a projection matrix. , where d is the dimension of the low-dimensional subspace. Due to the joint feature matrix... Since it only contains two columns, foreground and background, its rank is at most 2, and the corresponding low-dimensional subspace dimension can be d=2. Then, the decentralized foreground and background prior vectors are projected onto the low-dimensional subspace to obtain the low-dimensional foreground feature vector and low-dimensional background feature vector, respectively:

[0105] .

[0106] Subsequently , Perform Gramm-Schmidt orthogonalization. First, use the low-dimensional foreground feature vector... Based on this, normalize:

[0107] .

[0108] Then, the low-dimensional background feature vector Middle Directional components are removed to obtain a low-dimensional orthogonal background vector.

[0109] .

[0110] Therefore,

[0111] .

[0112] That is, the foreground and background are strictly orthogonal in the low-dimensional subspace.

[0113] Then, the orthogonalized low-dimensional vector is inversely projected back into the original pixel space to obtain high-dimensional orthogonal foreground vectors and high-dimensional orthogonal background vectors.

[0114] .

[0115] in, , Due to the projection matrix The column vectors are pairwise orthogonal and satisfy the following conditions: ,in It is a d-dimensional identity matrix, therefore:

[0116] .

[0117] This ensures that the foreground prior and background prior remain strictly orthogonal in high-dimensional space after inverse projection. Finally, and Reshape them to size From the two-dimensional feature map, an orthogonal reconstructed foreground prior map is obtained. Orthogonal reconstruction of background prior graph .

[0118] Subsequently, a foreground response map is defined in the mask semantic space. and background response map :

[0119] .

[0120] in, and These represent the foreground feature map and the background feature map, respectively. Indicates the Hadamah accumulation; when or When it is a multi-channel feature map, and Broadcast along the corridor. This represents a fixed response mapping operator used to map a foreground or background feature map weighted by prior knowledge to a single-channel response space consistent with the pseudomask.

[0121] Based on this, subspace prior constraints are constructed.

[0122] .

[0123] The first requirement is that the foreground response map obtained by orthogonally reconstructing the foreground prior map must be matched with a high-quality SAM pseudomask. Alignment should be maximized; the second requirement is that the background response map obtained by orthogonally reconstructing the background prior map be the complement of the high-quality SAM pseudomask. Align as much as possible; It is the square of the L2 norm.

[0124] This module ultimately outputs an orthogonally reconstructed foreground prior map. Orthogonal reconstruction of background prior graph and subspace prior constraints All inputs are into the SAM-guided reversible expansion and segmentation module.

[0125] The improvements in this module are as follows: a fixed mathematical transformation of "PCA dimensionality reduction combined with orthogonalization constraints" is adopted to compress prior redundancy and enhance foreground / background separability without introducing additional learnable parameters. High-quality SAM priors are injected into the subsequent unfolding network in a more stable form, taking into account interpretability, computational efficiency and prior utilization efficiency.

[0126] The SAM-guided reversible unfolding segmentation module is the core module of this invention. Its core function is to construct an overall objective function by fusing pixel-gradient dual-level feature fitting terms with SAM subspace prior constraints. It designs two alternating iterative reversible sub-modules, deriving closed-form update solutions for the foreground and background based on the proximal gradient algorithm. Through multi-stage iteration, it gradually refines the foreground and background feature maps, ultimately achieving accurate segmentation of camouflaged targets. This module retains the interpretability of model-driven methods and the strong fitting capability of data-driven methods, and all components are seamlessly integrated into the multi-stage reversible unfolding framework. This module first completes the construction of the overall objective function. To achieve gradient-level feature modeling, it first extracts the original image using the Sobel operator. Gradient feature maps are used to capture fine-grained features such as edges and textures in an image. First, a 3×3 Sobel convolution kernel is defined.

[0127] .

[0128] The horizontal and vertical edges are extracted separately. Then, Sobel convolution operations are performed on the RGB channels of the normalized original image to obtain the x / y gradient responses of each channel. A zero-padding strategy is used during the convolution process. Finally, the average of the three channel gradient responses is taken and fused to obtain the global x / y gradient.

[0129] .

[0130] in, Let be the global x-axis gradient value at pixel position (i,j) after fusion. Let be the fused global y-axis gradient value at pixel position (i,j). For the first The gradient response of each color channel in the x-direction at (i,j). For the first The gradient response of each color channel in the y-direction at (i,j). This is the color channel index, with values ​​of R, G, and B.

[0131] Then take the global x-direction gradient value Global y-axis gradient value Standardize each feature to the [0,1] interval and concatenate them by channel to form a gradient feature map:

[0132] .

[0133] in, This is the normalized gradient value in the x-direction. This is the normalized gradient value in the y-direction.

[0134] .

[0135] in, This is the gradient feature map of the original image. This is a splicing operation at the channel dimension.

[0136] Based on this gradient feature map, a pixel-gradient two-level feature fitting term is constructed:

[0137] .

[0138] in Learnable weights for pixel-level fitting terms. The learnable weights for the gradient-level fitting term are initialized according to the characteristics of the COS task. =1.0、 =0.5, adaptively adjusted during training. This fitting term includes two parts: pixel-level data fitting and gradient-level feature fitting. It ensures that the weighted foreground and background can reconstruct both the original image and its gradient features, achieving pixel-gradient bilevel modeling. The pixel-gradient bilevel feature fitting term... subspace prior constraints By merging, we obtain the overall objective function of the SAM-guided COS model:

[0139] .

[0140] in Fixed weights for prior constraints in the subspace, empirical values =0.3, used to control the contribution of the SAM large model prior to the segmentation model. The only variable to be optimized in this objective function is the foreground feature map. and background feature map This ensures the simplicity of model optimization.

[0141] Based on the overall objective function, this module designs two alternating iterative reversible submodules: the SAM-guided Foreground Optimization Submodule (SFOS) and the SAM-guided Background Optimization Submodule (SBOS), which are responsible for the foreground feature map, respectively. and background feature map The iterative updates are based on the proximal gradient algorithm to derive closed-form update solutions, ensuring the efficiency and interpretability of the iterative optimization.

[0142] The SFOS submodule is responsible for the first Stage (1≤k≤ Foreground Feature Map The module first fixes the iterative update of the first... Stage background feature map Ignoring constant terms irrelevant to the foreground, the overall objective function is simplified to a function that depends only on the foreground feature map. Optimization subproblems:

[0143]

[0144] .

[0145] in, Indicates the first Stage prospect feature map Index for the current iteration phase, For the first Stage background feature map, Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. The fixed weights are the prior constraints for the subspace. For the original image, Foreground feature map, Foreground space prior graph, This is the background space prior image. This is the gradient feature map of the original image. For gradient extraction operators, To reconstruct the foreground prior graph orthogonally, For high-quality SAM pseudomasks, For fixed response mapping operators, Optimize the proximal regularization term for the foreground. This is the decay factor for the foreground near-end iterations, used to ensure the stability of iterative updates.

[0146] Then, based on the proximal gradient algorithm, the optimization subproblem is solved. The optimization process is decomposed into gradient descent steps and proximal mapping steps. After differentiation and simplification, the foreground feature map of the k-th stage is obtained. Closed-form update solution:

[0147] .

[0148] in:

[0149] . The coefficient matrix for the prospect optimization subproblem. It is a diagonal matrix constructed from the input vector or the flattened feature map. For fixed response mapping operators, For the transpose operator of the fixed response mapping operator, For gradient extraction operators, For the transpose operator of the gradient extraction operator, Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. The fixed weights are the prior constraints for the subspace. It is the identity matrix. The decay factor for the foreground proximal iteration;

[0150] , This is the weight matrix of the foreground feature map from the previous stage;

[0151] .

[0152] The foreground residual term is composed of pixel-level fitting terms and gradient-level fitting terms.

[0153] , To reconstruct the foreground prior graph from orthogonal and high-quality SAM pseudomask The prior alignment terms that are jointly formed;

[0154] It is the first Stage foreground feature map, initial value =0, which is a matrix containing all zeros;

[0155] It is the first Stage background feature map, initial value =0, which is a matrix of all zeros.

[0156] The SBOS ​​submodule is responsible for the first Stage (1≤k≤ Background feature map The module first fixes the iterative update of the first... Stage Prospect Feature Map Ignoring constant terms irrelevant to the background, the overall objective function is simplified to a function that depends only on the background feature map. Optimization subproblems:

[0157]

[0158] .

[0159] in, Indicates the first Stage background feature map, This represents the foreground feature map at stage k. Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. The fixed weights are the prior constraints for the subspace. For the original image, Background feature map, Foreground space prior graph, This is the background space prior image. This is the gradient feature map of the original image. For gradient extraction operators, To orthogonally reconstruct the background prior graph, For high-quality SAM pseudomasks, The complement of a high-quality SAM pseudomask. For fixed response mapping operators, Optimize the near-end regularization term for the background. The decay factor for the near-end iteration is used as the background.

[0160] Then, based on the proximal gradient algorithm, the optimization subproblem is solved. After differentiation and simplification, the 1st... Stage background feature map Closed-form update solution:

[0161] .

[0162] in:

[0163] .

[0164] Optimize the coefficient matrix of the subproblem in the background. It is a diagonal matrix constructed from the input vector or the flattened feature map. For fixed response mapping operators, For the transpose operator of the fixed response mapping operator, For gradient extraction operators, For the transpose operator of the gradient extraction operator,

[0165] , This is the weight matrix of the background feature map from the previous stage;

[0166] .

[0167] The background residual term is composed of pixel-level fitting terms and gradient-level fitting terms.

[0168] , To reconstruct the background prior graph using orthogonal methods and high-quality SAM pseudomask complement The prior alignment terms that are jointly formed;

[0169] This is the foreground feature map of stage k;

[0170] It is the first Stage background feature map, initial value =0, which is a matrix of all zeros.

[0171] SFOS and SBOS ​​submodules adopt The update rule for alternating phases and the number of phases. experience points =4, balancing segmentation performance and computational efficiency, the output of the previous stage is used as the input of the next stage, progressively refining the foreground and background feature maps, with the iterative update rule being:

[0172] .

[0173] in This represents the number of stages in the network (set to 4 in the experiment to balance segmentation performance and computational efficiency). After K iterations, the final foreground and background feature maps are obtained. To guide the end-to-end training of the model, this module also designs a stage-weighted segmentation loss function:

[0174] .

[0175] This loss function will... The losses from each stage are weighted and summed, with stage weights set exponentially decaying to focus the model on later stages where the results are more refined.

[0176] It is the foreground feature map of the k-th stage The generated segmentation mask, It is the Sigmoid activation function. It is a 1×1 convolutional layer;

[0177] It is the true segmentation mask used to disguise the target;

[0178] It is a weighted binary cross-entropy loss, used to solve the class imbalance problem in camouflaged target segmentation tasks;

[0179] It is the weighted intersection-union loss, used to measure the degree of overlap between the predicted mask and the true mask;

[0180] It is the first The stage weights corresponding to the first-order loss decay exponentially with the number of stages, allowing the model to focus on later stages with more refined foreground / background features.

[0181] The module's final output is after... The final foreground feature map after stage iterations and loss function optimization is used as input to the segmentation result output module.

[0182] The improvement of this module lies in the joint design of "pixel domain, gradient domain, subspace prior and multi-stage alternating optimization", which not only enhances the model's ability to perceive the edges of camouflaged targets and fine-grained textures, but also makes the foreground update and background update have clear optimization meaning and mathematical structure. Compared with the pure data-driven black box segmentation network, it has stronger interpretability, stability and robustness to complex scenes.

[0183] The segmentation result output module, as the final stage of the entire segmentation method, has the core function of converting the final foreground feature map output by the SAM-guided reversible unfolding segmentation module into a binary segmentation mask for the camouflaged target, thus completing the entire camouflaged target segmentation process. This module first inputs the final foreground feature map into a 1×1 convolutional layer, converting the multi-channel feature map into a single-channel feature map. Then, it processes the map using a Sigmoid activation function to generate a pixel-level segmentation probability map with probability values ​​ranging from [0,1], representing the probability that each pixel is a camouflaged target. Subsequently, a 0.5 threshold method is used to binarize the segmentation probability map. Pixels with a probability value ≥ 0.5 are marked as foreground with a pixel value of 1, while pixels with a probability value < 0.5 are marked as background with a pixel value of 0. Finally, the binarized camouflaged target segmentation mask is output as the final result of this method.

[0184] The improvement of this module lies in the fact that its output does not come directly from the result of a single forward propagation, but from the final foreground feature map obtained after multi-stage alternating optimization. Therefore, the final segmentation result is superior to existing methods in terms of region integrity, boundary clarity, and adaptability to complex camouflage scenes.

[0185] This invention innovates and improves upon existing camouflaged target segmentation technology in multiple dimensions, with core innovations in four aspects:

[0186] One approach is to propose an adaptive spatial prior guided by SAM to replace hard masking. This approach constructs a foreground / background spatial prior from the native output of SAM and replaces the traditional hard mask with pixel-wise adaptive weights. This eliminates the rigid constraints of hard masking and enables the model to adaptively separate the foreground and background according to the spatial structure of the image, effectively preserving the subtle edge and texture features of the camouflaged target.

[0187] Secondly, a low-dimensional orthogonal subspace projection strategy combining PCA and Gram-Schmidt was designed. By reducing the dimensionality of linear PCA, feature redundancy of spatial priors was eliminated. Gram-Schmidt orthogonalization was combined to force strict orthogonality between the foreground and background subspaces. At the same time, subspace prior constraints were constructed to inject the SAM large model prior into the reversible expansion framework, which significantly improved the generalization ability of the model.

[0188] Thirdly, a pixel-gradient dual-level feature fitting term is constructed. On the basis of traditional pixel-level data fitting, a gradient-level feature fitting term is added to enhance the modeling of fine-grained features such as the edge and texture of the camouflaged target. This enables the model to capture the core discrimination clues of the COS task and solves the problem of incomplete segmentation caused by the lack of fine feature modeling in existing methods.

[0189] Fourthly, closed-form update formulas for foreground and background feature maps are derived based on a unified objective function, and a multi-stage expansion optimization framework is constructed. The "reversible expansion" is mainly reflected in: the expansion based on the unified objective function yields two alternating iterative sub-modules, SFOS and SBOS, with each stage of forward update corresponding to a clear optimization sub-problem; both foreground and background updates can be derived from closed-form update formulas using the proximal gradient algorithm, exhibiting clear mathematical solution relationships; simultaneously, the network iteration process maintains a correspondence with the expansion process of the objective function, thus combining the interpretability of model-driven methods with the fitting capability of data-driven methods.

[0190] This invention proposes a camouflaged target segmentation method based on a SAM-guided reversible unfolding network. This method effectively addresses the shortcomings of existing methods, such as hard mask constraints, lack of large model priors, and neglect of gradient feature modeling. Extensive experimental verification demonstrates significant advantages: Segmentation accuracy is significantly improved. Through adaptive guidance of SAM spatial priors and pixel-gradient dual-level feature modeling, the model can accurately capture subtle edge and texture features of camouflaged targets. The completeness and accuracy of the segmentation results surpass existing state-of-the-art methods, achieving state-of-the-art performance on the COS task dataset. Generalization ability is greatly enhanced. The strong visual priors of SAM are effectively injected into the reversible unfolding framework, enabling the model to maintain excellent segmentation performance and exhibit good adaptability to different scenes and types of camouflaged targets. Robustness is excellent. The multi-stage iterative reversible unfolding framework combined with exponentially decaying segmentation loss allows the model to perform well even with small targets, multiple targets, and degraded COS targets with blurred / noisy images. It exhibits strong robustness in the S-scene, effectively solving the segmentation failure problem in complex scenarios; the model has strong interpretability. This invention constructs a segmentation framework based on model-driven optimization, and its reversibility is reflected in the alternating iterative submodule design, closed-form update solution, and the one-to-one correspondence between objective function expansion and reconstruction. Therefore, the update process of foreground and background has clear mathematical meaning and a backtrackable solution path; it has good engineering feasibility. The construction of the SAM space prior and the projection of the low-dimensional orthogonal subspace are pre-computation operations with no additional training overhead. The number of stages in the multi-stage iteration can be flexibly adjusted according to hardware resources, balancing segmentation performance and computational efficiency, and is suitable for practical engineering deployment; it has a wide range of applications. The method of this invention is applicable to various camouflaged target segmentation scenarios, including camouflaged target detection in military reconnaissance, polyp / tumor segmentation in medical imaging, transparent defect detection in industrial inspection, and camouflaged target recognition in security monitoring, etc., and has extremely high practical application value. Attached Figure Description

[0191] Figure 1 This is a flowchart of the present invention.

[0192] Figure 2 This is a model block diagram of the present invention.

[0193] Figure 3 This is a comparison chart of the segmentation results of the present invention and eight advanced models in different scenarios. Detailed Implementation

[0194] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0195] Example: The camouflaged target segmentation method based on a SAM-guided reversible unfolding network proposed in this invention consists of five core modules connected sequentially according to the data processing flow: a camouflaged image input module, a SAM space prior construction module, a low-dimensional orthogonal subspace projection module, a SAM-guided reversible unfolding segmentation module, and a segmentation result output module. The output of each module serves as the input of the next module, achieving end-to-end intelligent segmentation from camouflaged target image input to binary segmentation mask output. Extensive experiments have verified the superiority of this method. Specifically, the steps employed by the above system are as follows:

[0196] In this embodiment, the core empirical parameters are all set according to experimental optimal values: image augmentation amount. =18, PCA low-dimensional subspace dimension =2. Number of stages in a reversible unfolding network =4. Fixed weights of subspace prior constraint terms =0.3, learnable weights of pixel-level fitting term =1.0, learnable weights of gradient-level fitting term =0.5, decay factor for foreground-proximal iterations =0.1, the decay factor of the background near-end iteration. =0.1. The above parameters can be flexibly adjusted according to actual hardware resources and segmentation task scenarios to balance segmentation performance and computational efficiency.

[0197] The camouflage image input module, as the initial step in the entire segmentation method, primarily functions to perform standardization preprocessing on the input camouflage target image. This ensures the consistency of size and numerical stability in subsequent module operations, laying the foundation for the subsequent segmentation process. This module first receives an RGB format camouflage target image of arbitrary size, denoted as the camouflage target image. ,in , Here, represents the image height and width, respectively, and 3 represents the RGB three-channel dimension. The RGB pixel values ​​are then normalized to the [0,1] range to avoid numerical overflow affecting subsequent calculations. The normalization formula is:

[0198] .

[0199] in, To normalize the image at position Place, No. The pixel values ​​of each channel. To disguise target images In position Place, No. The pixel values ​​of each channel. The row and column coordinates of the image. This is the color channel index, with values ​​of R, G, and B. For image In the Two-dimensional image corresponding to each channel For the first The minimum value of all pixels in each channel. For the first The maximum value of all pixels in each channel.

[0200] After channel normalization, this module scales the normalized image to a fixed size set in the experiment and uses a zero-padding strategy to fill the image edges, ensuring that the output feature maps of subsequent gradient extraction and convolution operations are exactly the same size as the input image. Finally, the preprocessed original image is output, denoted as the original image. , which serves as the input to the SAM space prior construction module and the SAM-guided reversible unfolding segmentation module.

[0201] The core function of the SAM spatial prior construction module is to construct a foreground space prior graph from the native output of SAM. Background space prior graph and high-quality SAM pseudomask It replaces the hard mask in traditional methods with adaptive pixel-wise weights to achieve foreground-background separation without hard constraints. At the same time, it optimizes the quality of the prior image through a multi-augmentation fusion strategy, effectively preserving the subtle features of the camouflaged target.

[0202] This module first processes the original image. Inputting a pre-trained SAM model yields foreground soft response maps and background soft response maps under a single view, denoted as follows: and Both are related to Two-dimensional feature maps of uniform size, wherein, The continuous probability response that represents a pixel belonging to a camouflaged foreground is used to characterize the pixel. This module represents the continuous probability response of a pixel belonging to the background. To improve the stability of the SAM output in complex camouflage scenes, this module further refines the original image. generate An augmented view, denoted as ,in, Indicates the first A geometric transformation operation Indicates the first An augmented view corresponding to a geometric transformation, the geometric transformation including flipping, rotating, and scaling.

[0203] Input each augmented view into SAM to get the first... Foreground soft response map of an augmented view , No. The background soft response map of the augmented view Then, the outputs of each augmented view are mapped back to the original image coordinate system through inverse transformation to obtain...

[0204] .

[0205] in, For the first Foreground soft response map of an augmented view, For the first The background soft response map of the augmented view. For the first Geometric transformation operations The inverse transformation operation is used to map the augmented view coordinates back to the original image coordinate system. , For the first The foreground soft response map after mapping the augmented view back to the original image coordinate system, the first The background soft response map is generated after the augmented view is mapped back to the original image coordinate system.

[0206] Since the output quality of SAM varies across different augmented views, direct mean fusion would introduce noise. Therefore, this invention employs an uncertainty-based weighted fusion strategy to improve prior quality. Specifically, the first... The foreground-background prediction entropy of the augmented view at pixel position (x, y) is:

[0207] .

[0208] in, Characterizing the uncertainty at this location, For the first The foreground soft response value at pixel position (x, y) after the augmented view is mapped back to the original image. For the first The background soft response value at pixel position (x,y) after the augmented view is mapped back to the original image.

[0209] The smaller the entropy value, the more stable the foreground-background determination at that location in the current view; the larger the entropy value, the greater the uncertainty at that location.

[0210] Based on this, pixel-level fusion weights are defined as follows:

[0211] .

[0212] in, For the first Pixel-level blending weights at pixel position (x, y) for each augmented view. It is an exponential function. For the first The foreground-background prediction entropy of an augmented view at pixel location (x,y).

[0213] Meanwhile, to reflect the overall reliability of the entire view, image-level quality weights are defined:

[0214] .

[0215] in, For the first Image-level quality weights for each augmented view The averaging coefficient is the average value applied to all pixels of the entire image. For the first The foreground-background prediction entropy of an augmented view at pixel location (x,y).

[0216] Based on this, the foreground space prior map and the background space prior map are defined as the weighted fusion results of the multi-view soft response:

[0217] .

[0218] .

[0219] in, This represents the foreground space prior graph. This represents the background space prior graph. It represents the Hadamah accumulation. For the first Image-level quality weights for each augmented view For the first Pixel-level fusion weight map of each augmented view For the first The foreground soft response map after mapping the augmented view back to the original image coordinate system. For the first The background soft response map is generated after the augmented view is mapped back to the original image coordinate system.

[0220] Furthermore, a similar weighted fusion is performed on the native segmentation mask of the augmented view to obtain a soft-fused pseudo-mask. After post-processing operations such as thresholding, space filling, and maximum connected component filtering, a high-quality SAM pseudo-mask is finally obtained. This is used for subsequent subspace prior constraints.

[0221] Finally, the module outputs a foreground space prior map. Background space prior graph and high-quality SAM pseudomask .in, and Input a low-dimensional orthogonal subspace projection module and a SAM-guided reversible unfolding and segmentation module. Input the subspace prior constraint construction process in the low-dimensional orthogonal subspace projection module, and simultaneously input the SAM-guided reversible expansion and segmentation module.

[0222] The core function of the low-dimensional orthogonal subspace projection module is to and Dimensionality reduction, redundancy removal, and orthogonalization are performed to eliminate feature redundancy in the spatial prior and force strict orthogonality between the foreground and background subspaces. Simultaneously, subspace prior constraints are constructed based on the projection results, effectively injecting the large-scale visual prior of the SAM model into the subsequent reversible unfolding segmentation framework. The core improvement of this invention lies in employing a combined strategy of linear PCA dimensionality reduction and Gram-Schmidt orthogonalization. The entire process involves fixed mathematical operations with no learnable parameters, and can be pre-computed on the CPU. This module first transforms the two-dimensional foreground space prior map... and background space prior graph Flatten them into lengths of One-dimensional vector: Foreground space prior flattened vector and background space prior flattened vector ,in This represents the total number of pixels in the image. Indicates the image height. This represents the image width. Then, respectively... and After performing min-max normalization to the [0,1] interval, the joint feature matrix is ​​constructed by concatenating the columns. To eliminate the effect of the overall bias, the joint characteristic matrix is... Decentralization is performed, and the mean vector is... Then the decentralized joint feature matrix can be written as Then calculate the covariance matrix of the decentralized joint feature matrix.

[0223]

[0224] in The transpose operation represents the matrix transformation operation, applied to the covariance matrix. Perform eigenvalue decomposition, and take the eigenvectors corresponding to the first d largest eigenvalues ​​to form a projection matrix. , where d is the dimension of the low-dimensional subspace. Due to the joint feature matrix... Since it only contains two columns, foreground and background, its rank is at most 2, and the corresponding low-dimensional subspace dimension can be d=2. Then, the decentralized foreground and background prior vectors are projected onto the low-dimensional subspace to obtain the low-dimensional foreground feature vector and low-dimensional background feature vector, respectively:

[0225] .

[0226] Subsequently , Perform Gramm-Schmidt orthogonalization. First, use the low-dimensional foreground feature vector... Based on this, normalize:

[0227]

[0228] Then, the low-dimensional background feature vector Middle Directional components are removed to obtain a low-dimensional orthogonal background vector.

[0229] .

[0230] Therefore,

[0231]

[0232] That is, the foreground and background are strictly orthogonal in the low-dimensional subspace.

[0233] Then, the orthogonalized low-dimensional vector is inversely projected back into the original pixel space to obtain high-dimensional orthogonal foreground vectors and high-dimensional orthogonal background vectors.

[0234] .

[0235] in, , Due to the projection matrix The column vectors are pairwise orthogonal and satisfy the following conditions: ,in It is a d-dimensional identity matrix, therefore:

[0236] .

[0237] This ensures that the foreground prior and background prior remain strictly orthogonal in high-dimensional space after inverse projection. Finally, and Reshape them to size From the two-dimensional feature map, an orthogonal reconstructed foreground prior map is obtained. Orthogonal reconstruction of background prior graph .

[0238] Subsequently, a foreground response map is defined in the mask semantic space. and background response map :

[0239] .

[0240] in, and These represent the foreground feature map and the background feature map, respectively. Indicates the Hadamah accumulation; when or When it is a multi-channel feature map, and Broadcast along the corridor. This represents a fixed response mapping operator used to map a foreground or background feature map weighted by prior knowledge to a single-channel response space consistent with the pseudomask.

[0241] Based on this, subspace prior constraints are constructed.

[0242] .

[0243] The first requirement is that the foreground response map obtained by orthogonally reconstructing the foreground prior map must be matched with a high-quality SAM pseudomask. Alignment should be maximized; the second requirement is that the background response map obtained by orthogonally reconstructing the background prior map be the complement of the high-quality SAM pseudomask. Align as much as possible; It is the square of the L2 norm.

[0244] This module ultimately outputs an orthogonally reconstructed foreground prior map. Orthogonal reconstruction of background prior graph and subspace prior constraints All inputs are into the SAM-guided reversible expansion and segmentation module.

[0245] The SAM-guided reversible unfolding segmentation module is the core module of this invention. Its core function is to construct an overall objective function by fusing pixel-gradient dual-level feature fitting terms with SAM subspace prior constraints. It designs two alternating iterative reversible sub-modules, deriving closed-form update solutions for the foreground and background based on the proximal gradient algorithm. Through multi-stage iteration, it gradually refines the foreground and background feature maps, ultimately achieving accurate segmentation of camouflaged targets. This module retains the interpretability of model-driven methods and the strong fitting capability of data-driven methods, and all components are seamlessly integrated into the multi-stage reversible unfolding framework. This module first completes the construction of the overall objective function. To achieve gradient-level feature modeling, it first extracts the original image using the Sobel operator. Gradient feature maps are used to capture fine-grained features such as edges and textures in an image. First, a 3×3 Sobel convolution kernel is defined.

[0246] .

[0247] The horizontal and vertical edges are extracted separately. Then, Sobel convolution operations are performed on the RGB channels of the normalized original image to obtain the x / y gradient responses of each channel. A zero-padding strategy is used during the convolution process. Finally, the average of the three channel gradient responses is taken and fused to obtain the global x / y gradient.

[0248] .

[0249] in, Let be the global x-axis gradient value at pixel position (i,j) after fusion. Let be the fused global y-axis gradient value at pixel position (i,j). For the first The gradient response of each color channel in the x-direction at (i,j). For the first The gradient response of each color channel in the y-direction at (i,j). This is the color channel index, with values ​​of R, G, and B.

[0250] Then take the global x-direction gradient value Global y-axis gradient value Standardize each feature to the [0,1] interval and concatenate them by channel to form a gradient feature map:

[0251] .

[0252] in, This is the normalized gradient value in the x-direction. This is the normalized gradient value in the y-direction.

[0253]

[0254] in, This is the gradient feature map of the original image. This is a splicing operation at the channel dimension.

[0255] Based on this gradient feature map, a pixel-gradient two-level feature fitting term is constructed:

[0256] .

[0257] in Learnable weights for pixel-level fitting terms. The learnable weights for the gradient-level fitting term are initialized according to the characteristics of the COS task. =1.0、 =0.5, adaptively adjusted during training. , These are the original image, foreground feature map, and background feature map, respectively. Foreground space prior graph, This is the background space prior image. For Hadama accumulation, This is the gradient feature map of the original image. For gradient extraction operators, The fitting term, representing the square of the L2 norm, comprises pixel-level data fitting and gradient-level feature fitting. This ensures that the weighted foreground and background reconstruct both the original image and its gradient features, achieving a two-level pixel-gradient modeling. The pixel-gradient two-level feature fitting term... subspace prior constraints By merging, we obtain the overall objective function of the SAM-guided COS model:

[0258] .

[0259] in Fixed weights for prior constraints in the subspace, empirical values =0.3, used to control the contribution of the SAM large model prior to the segmentation model. The only variable to be optimized in this objective function is the foreground feature map. and background feature map This ensures the simplicity of model optimization.

[0260] Based on the overall objective function, this module designs two alternating iterative reversible submodules: the SAM-guided Foreground Optimization Submodule (SFOS) and the SAM-guided Background Optimization Submodule (SBOS), which are responsible for the foreground feature map, respectively. and background feature map The iterative updates are based on the proximal gradient algorithm to derive closed-form update solutions, ensuring the efficiency and interpretability of the iterative optimization.

[0261] The SFOS submodule is responsible for the first Stage (1≤k≤ Foreground Feature Map The module first fixes the iterative update of the first... Stage background feature map Ignoring constant terms irrelevant to the foreground, the overall objective function is simplified to a function that depends only on the foreground feature map. Optimization subproblems:

[0262]

[0263] .

[0264] in, Indicates the first Stage prospect feature map Index for the current iteration phase, For the first Stage background feature map, Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. The fixed weights are the prior constraints for the subspace. For the original image, Foreground feature map, Foreground space prior graph, This is the background space prior image. This is the gradient feature map of the original image. For gradient extraction operators, To reconstruct the foreground prior graph orthogonally, For high-quality SAM pseudomasks, For fixed response mapping operators, Optimize the proximal regularization term for the foreground. This is the decay factor for the foreground near-end iterations, used to ensure the stability of iterative updates.

[0265] Then, based on the proximal gradient algorithm, the optimization subproblem is solved. The optimization process is decomposed into gradient descent steps and proximal mapping steps. After differentiation and simplification, the foreground feature map of the k-th stage is obtained. Closed-form update solution:

[0266] .

[0267] in:

[0268] . The coefficient matrix for the prospect optimization subproblem. It is a diagonal matrix constructed from the input vector or the flattened feature map. For fixed response mapping operators, For the transpose operator of the fixed response mapping operator, For gradient extraction operators, For the transpose operator of the gradient extraction operator, Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. The fixed weights are the prior constraints for the subspace. It is the identity matrix. The decay factor for the foreground proximal iteration;

[0269] , This is the weight matrix of the foreground feature map from the previous stage;

[0270] .

[0271] The foreground residual term is composed of pixel-level fitting terms and gradient-level fitting terms.

[0272] , To reconstruct the foreground prior graph from orthogonal and high-quality SAM pseudomask The prior alignment terms that are jointly formed;

[0273] It is the first Stage foreground feature map, initial value =0, which is a matrix containing all zeros;

[0274] It is the first Stage background feature map, initial value =0, which is a matrix of all zeros.

[0275] The SBOS ​​submodule is responsible for the first Stage (1≤k≤ Background feature map The module first fixes the iterative update of the first... Stage Prospect Feature Map Ignoring constant terms irrelevant to the background, the overall objective function is simplified to a function that depends only on the background feature map. Optimization subproblems:

[0276]

[0277] .

[0278] in, Indicates the first Stage background feature map, This represents the foreground feature map at stage k. Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. The fixed weights are the prior constraints for the subspace. For the original image, Background feature map, Foreground space prior graph, This is the background space prior image. This is the gradient feature map of the original image. For gradient extraction operators, To orthogonally reconstruct the background prior graph, For high-quality SAM pseudomasks, The complement of a high-quality SAM pseudomask. For fixed response mapping operators, Optimize the near-end regularization term for the background. The decay factor for the near-end iteration is used as the background.

[0279] Then, based on the proximal gradient algorithm, the optimization subproblem is solved. After differentiation and simplification, the 1st... Stage background feature map Closed-form update solution:

[0280] .

[0281] in:

[0282] . Optimize the coefficient matrix of the subproblem in the background. It is a diagonal matrix constructed from the input vector or the flattened feature map. For fixed response mapping operators, For the transpose operator of the fixed response mapping operator, For gradient extraction operators, For the transpose operator of the gradient extraction operator,

[0283] , This is the weight matrix of the background feature map from the previous stage;

[0284] . The background residual term is composed of pixel-level fitting terms and gradient-level fitting terms.

[0285] , To reconstruct the background prior graph using orthogonal methods and high-quality SAM pseudomask complement The prior alignment terms that are jointly formed;

[0286] This is the foreground feature map of stage k;

[0287] It is the first Stage background feature map, initial value =0, which is a matrix of all zeros.

[0288] SFOS and SBOS ​​submodules adopt The update rule for alternating phases and the number of phases. experience points =4, balancing segmentation performance and computational efficiency, the output of the previous stage is used as the input of the next stage, progressively refining the foreground and background feature maps, with the iterative update rule being:

[0289] .

[0290] in This represents the number of stages in the network (set to 4 in the experiment to balance segmentation performance and computational efficiency). After K iterations, the final foreground and background feature maps are obtained. To guide the end-to-end training of the model, this module also designs a stage-weighted segmentation loss function:

[0291] .

[0292] This loss function will... The losses from each stage are weighted and summed, with stage weights set exponentially decaying to focus the model on later stages where the results are more refined.

[0293] It is the foreground feature map of the k-th stage The generated segmentation mask, It is the Sigmoid activation function. It is a 1×1 convolutional layer;

[0294] It is the true segmentation mask used to disguise the target;

[0295] It is a weighted binary cross-entropy loss, used to solve the class imbalance problem in camouflaged target segmentation tasks;

[0296] It is the weighted intersection-union loss, used to measure the degree of overlap between the predicted mask and the true mask;

[0297] It is the first The stage weights corresponding to the first-order loss decay exponentially with the number of stages, allowing the model to focus on later stages with more refined foreground / background features.

[0298] The module's final output is after... The final foreground feature map after stage iterations and loss function optimization is used as input to the segmentation result output module.

[0299] The segmentation result output module, as the final stage of the entire segmentation method, has the core function of converting the final foreground feature map output by the SAM-guided reversible unfolding segmentation module into a binary segmentation mask for the camouflaged target, thus completing the entire camouflaged target segmentation process. This module first inputs the final foreground feature map into a 1×1 convolutional layer, converting the multi-channel feature map into a single-channel feature map. Then, it processes the map using a Sigmoid activation function to generate a pixel-level segmentation probability map with probability values ​​ranging from [0,1], representing the probability that each pixel is a camouflaged target. Subsequently, a 0.5 threshold method is used to binarize the segmentation probability map. Pixels with a probability value ≥ 0.5 are marked as foreground with a pixel value of 1, while pixels with a probability value < 0.5 are marked as background with a pixel value of 0. Finally, the binarized camouflaged target segmentation mask is output as the final result of this method.

[0300] To verify the effectiveness of the proposed method, the algorithm was compared with eight state-of-the-art methods on four publicly available camouflage target detection datasets, including SINet, LSR, FEDER, FGANet, SegMaR, PreyNet, ASBI, and SurANet. All networks were implemented in PyTorch 1.8.0, and model training and testing were performed using an NVIDIA GeForce RTX 3090 GPU. A pre-trained ResNet50 model was used as the backbone network for the encoding feature extraction module, with Adam as the optimizer. The initial learning rate was set to 1e-4, decreasing to 0.1 times the original rate every 60 training epochs. The batch size was 36, and the network was trained for a total of 180 epochs. The model was trained using a hybrid dataset consisting of CAMOTrain (1000 images) and COD10K-Train (3040 images), and the performance of the proposed method was verified and tested on four publicly available datasets: CAMO, COD10K, CHAMELEON, and NC4K. Among them, the CHAMELEON dataset contains 76 images and serves as a small-sample robustness test dataset; the CAMO dataset contains 250 images with high scene complexity; the COD10K test set contains 2026 images, covering 78 types of camouflaged targets; and the NC4K dataset contains 4121 images, which can effectively evaluate the model's cross-domain generalization ability in different scenarios such as nature, city, and military.

[0301] The detection performance of each method on the CAMO, COD10K, CHAMELEON, and NC4K test datasets is shown in Tables 1 and 2.

[0302]

[0303]

[0304] As can be seen from the comparison figures in Tables 1 and 2 and Figure 3, the camouflage target segmentation method based on the SAM-guided reversible unfolding network proposed in this invention outperforms most existing state-of-the-art methods in terms of segmentation accuracy and robustness in camouflage target detection tasks, achieving superior performance. Furthermore, the qualitative visualization results clearly demonstrate the segmentation differences between this invention and other segmentation methods in complex scenes, enabling more accurate capture of target edges and details, and effectively avoiding undersegmentation and oversegmentation problems.

[0305] Quantitative data and qualitative visualization results together show that, compared with existing methods, the present invention has better segmentation effect and stronger environmental adaptability in various camouflaged target segmentation scenarios, and can be stably applied to the accurate segmentation task of multiple types of camouflaged targets.

[0306] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for camouflaged target segmentation based on a SAM-guided reversible unfolding network, characterized in that, Includes the following steps: Step 1: Receive and preprocess the input camouflaged target image to obtain the original image; Step 2: Input the original image into the Segmentation Model (SAM), construct and optimize the foreground space prior map, background space prior map, and high-quality SAM pseudo-mask to achieve adaptive foreground-background separation; Step 3: Perform dimensionality reduction and orthogonalization on the foreground space prior map and the background space prior map to eliminate feature redundancy and keep the foreground and background subspaces orthogonal, so as to obtain the orthogonal reconstructed foreground prior map and the orthogonal reconstructed background prior map. Step 4: Based on the original image, the gradient feature map of the original image, the foreground space prior map, the background space prior map, the orthogonally reconstructed foreground prior map, the orthogonally reconstructed background prior map, and the high-quality SAM pseudomask, construct the objective function and expand the objective function as follows: The foreground and background update processes are alternately iterated in stages, and foreground feature maps and background feature maps are obtained step by step. Step 5, place the first Input foreground feature map obtained from the first iteration The convolutional layer is activated by the Sigmoid activation function to obtain a segmentation probability map. The segmentation probability map is then binarized to obtain the final segmentation mask of the disguised target. In step four, a target function is first constructed that integrates the pixel-gradient two-level feature fitting term and the SAM subspace prior constraint term, serving as the basis for subsequent optimization. Specifically: Step 1, Two-level feature fitting term This includes pixel-level data fitting and gradient-level feature fitting, ensuring that the weighted foreground and background can reconstruct the original image and its gradient features. The formula is: in Learnable weights for pixel-level fitting terms. Learnable weights for the gradient-level fitting term. , These are the original image, foreground feature map, and background feature map, respectively. Foreground space prior graph, This is the background space prior image. For Hadama accumulation, This is the gradient feature map of the original image. For gradient extraction operators, The square of the L2 norm; Step 2: Define the foreground prior graph reconstructed from orthogonal priors. Orthogonal reconstruction of background prior graph Guided foreground response map and background response map : in, This represents a fixed response mapping operator used to map a foreground or background feature map weighted by prior knowledge to a single-channel response space consistent with the pseudomask. Step 3: Construct subspace prior constraints , in, Foreground response map, For background response image, For high-quality SAM pseudomasks; Step 4, Overall Objective Function: ,in The fixed weights of the subspace prior constraints control the contribution of the SAM prior.

2. The camouflaged target segmentation method based on a SAM-guided reversible unfolding network according to claim 1, characterized in that, The preprocessing steps in step one are as follows: The input RGB camouflage target image is normalized to a uniform image size. ,in, Indicates the image height. The image width is represented by 3, and the number of image channels corresponds to the three RGB color channels. A zero-padding strategy is used to ensure size consistency in subsequent gradient extraction and convolution operations. The preprocessed image is denoted as the original image. .

3. The camouflaged target segmentation method based on SAM-guided reversible unfolding network according to claim 1, characterized in that, The specific implementation steps for step two are as follows: Step 1: Extract the original image conduct Geometric augmentation, including flipping, rotating, and scaling, yields multiple augmented views. To increase the number of views; Step 2: Input each augmented view into SAM to obtain the foreground soft response map under a single view. Background soft response map in single view And the native segmentation mask, and then the inverse transformation is used to map these outputs back to the original image coordinate system to ensure that the outputs of all views are aligned in the same space; Step 3: Calculate the foreground-background prediction entropy for each pixel location; Step 4: Weight and fuse the foreground / background soft response maps of each view using pixel-level and image-level weights respectively to obtain the final foreground space prior map. and background space prior graph ; Step 5: Weighted fusion of the original segmentation masks from multiple views yields a soft-fused pseudo-mask. This pseudo-mask is then processed through thresholding, space filling, and maximum connected component filtering to finally obtain a high-quality SAM pseudo-mask. This is used for subsequent subspace prior constraints.

4. The camouflaged target segmentation method based on a SAM-guided reversible unfolding network according to claim 1, characterized in that, Step three involves dimensionality reduction and orthogonalization, including: Step 1: Create a two-dimensional foreground space prior image. and background space prior graph Flatten them into lengths of A one-dimensional vector, denoted as the prior flattened vector in the foreground space. and background space prior flattened vector Then perform min-max normalization to [0,1] and concatenate them into a joint feature matrix. ,in This represents the total number of pixels in the image. Indicates the image height. Indicates the image width; Step 2: For the joint feature matrix After decentralization, the covariance matrix is ​​calculated, and the projection matrix is ​​obtained through eigenvalue decomposition. Then and Projecting onto a low-dimensional subspace yields a low-dimensional foreground feature vector. and low-dimensional background feature vectors ; Step 3, for and Perform Gram-Schmidt orthogonalization to obtain mutually orthogonal low-dimensional orthogonal foreground vectors. and low-dimensional orthogonal background vector ; Step 4, obtain and Through projection matrix Inversely projected back to the original pixel space and reshaped to a size of From the two-dimensional feature map, an orthogonal reconstructed foreground prior map is obtained. and orthogonal reconstruction background prior graph .

5. The camouflaged target segmentation method based on SAM-guided reversible unfolding network according to claim 4, characterized in that, Original image gradient feature map The gradients in the x / y directions are obtained by extracting them using the Sobel operator. The specific process is as follows: Step 1: Define a 3×3 Sobel horizontal convolution kernel. Convolution kernels perpendicular to Sobel Sobel convolution is performed on the three RGB channels of the normalized original image to obtain the gradient response in the x / y direction of each channel; Step 2: Average and fuse the gradients from the three channels to obtain the global gradient value in the x-direction. Global y-axis gradient value ; Step 3, , Standardize to [0,1], and splice according to channel. Original image gradient feature map .

6. The camouflaged target segmentation method based on a SAM-guided reversible unfolding network according to claim 1, characterized in that, The alternating foreground and background update part in step four includes two alternating sub-modules: the SAM-guided foreground optimization sub-module SFOS and the SAM-guided background optimization sub-module SBOS, wherein SFOS is specifically implemented as follows: Step 1, construct the first The phase prospect optimization sub-problem, with the fixed first... Stage background feature map Ignoring constant terms irrelevant to the foreground, the overall objective function is simplified to a function that depends only on the foreground feature map. The optimized formula; Step 2: Solve the optimization subproblem based on the proximal gradient algorithm, and derive the... Stage Prospect Feature Map Closed-form update solution: in: The coefficient matrix for the prospect optimization subproblem. This is the weight matrix of the foreground feature map from the previous stage. The foreground residual term is composed of both pixel-level and gradient-level fitting terms. To reconstruct the foreground prior graph from orthogonal and high-quality SAM pseudomask The prior alignment terms that are jointly formed, The fixed weights are the prior constraints for the subspace. It is the first Stage foreground feature map, initial value =0, which is a matrix containing all zeros; The specific implementation of SBOS ​​is as follows: Step 1, construct the first The phase background optimization sub-problem, with the fixed first... Stage Prospect Feature Map Ignoring constant terms irrelevant to the background, the overall objective function is simplified to a function that depends only on the background feature map. The optimized formula; Step 2: Solve the optimization subproblem based on the proximal gradient algorithm, and derive the... Stage background feature map Closed-form update solution: in: Optimize the coefficient matrix of the subproblem in the background. This is the weight matrix of the background feature map from the previous stage. The background residual term is composed of pixel-level fitting terms and gradient-level fitting terms. To reconstruct the background prior graph using orthogonal methods and high-quality SAM pseudomask complement The prior alignment terms that are jointly formed, The fixed weights are the prior constraints for the subspace. It is the first Stage background feature map, initial value =0, which is a matrix of all zeros.

7. The camouflaged target segmentation method based on a SAM-guided reversible unfolding network according to claim 6, characterized in that, The SFOS and SBOS ​​submodules adopt The update rules for alternating phases are as follows: in , =0、 =0 is the initial all-zero matrix. The number of network stages is represented by the foreground and background outputs of the previous stage, which serve as the inputs for the next stage, thus gradually refining the feature maps. The SAM-guided reversible unfolding segmentation module employs a stage-weighted segmentation loss function to guide iterative optimization. This segmentation loss function is the weighted sum of the segmentation losses at each stage, with stage weights set exponentially decaying. The formula is as follows: The loss function will include all The losses from each stage are weighted and summed, with stage weights set exponentially decaying to focus the model on later stages where the results are more refined. It is the foreground feature map of the k-th stage The generated segmentation mask, It is the Sigmoid activation function. It is a 1×1 convolutional layer; It is the true segmentation mask used to disguise the target; It is a weighted binary cross-entropy loss, used to solve the class imbalance problem in camouflaged target segmentation tasks; It is the weighted intersection-union loss, used to measure the degree of overlap between the predicted mask and the true mask; It is the first The stage weights corresponding to the first-order loss decay exponentially with the number of stages, allowing the model to focus on later stages with more refined foreground / background features.

8. The camouflaged target segmentation method based on a SAM-guided reversible unfolding network according to claim 1, characterized in that, The specific process for step five is as follows: The final foreground feature map obtained after stage iterative optimization is input into a 1×1 convolutional layer, and a pixel-level segmentation probability map is generated by the Sigmoid activation function. The probability map is then binarized to obtain the final segmentation mask of the disguised target.

9. A camouflaged target segmentation system based on a SAM-guided reversible unfolding network, characterized in that, The system is used to implement the camouflaged target segmentation method based on a SAM-guided reversible unfolding network as described in any one of claims 1-8, the system comprising: The camouflage image input module is used to receive and preprocess the input camouflage target image; The SAM spatial prior construction module is used to build and optimize the foreground spatial prior map and background spatial prior map from the Segment Anything Model, i.e., the native output of SAM, to replace hard masking and achieve adaptive foreground-background separation. The low-dimensional orthogonal subspace projection module is used to reduce the dimensionality and orthogonalize the foreground and background space prior maps, eliminate feature redundancy and force the foreground and background subspaces to be strictly orthogonal. The SAM-guided reversible unfolding segmentation module is used to fuse pixel-gradient dual-level feature fitting with subspace prior constraints, and achieves progressive segmentation of camouflaged targets through alternating iterative foreground optimization and background optimization sub-modules. The segmentation result output module is used to generate the final segmentation mask of the camouflaged target based on the iteratively optimized foreground feature map.

Images