Weak supervision camouflage target segmentation method based on long-distance diffusion

Abstract

The invention discloses a weak supervision camouflage target segmentation method based on long-distance diffusion, and belongs to the technical field of data identification, and a weak supervision camouflage target segmentation model adopts a two-stage training method and comprises a backbone network, an RFB-modify module, an LCC module, a Trans-decorator module, an RUp module and a GLSC loss function. The backbone network generates backbone features for the RFB-modify module and the LCC module to extract local features from the backbone features, the Trans-decorator module receives the backbone features and the local features in sequence to update a global feature token and the local features, and the RUp module receives output of the Trans-decorator module and generates prediction features of a current layer. According to the method, through a one-way gating mechanism of GLSC loss, noise interference is suppressed, and the global diffusion efficiency of sparse labeling is improved; through combination of a two-stage training strategy and an extrusion effect, the confidence and sharpness of the edge of the camouflage target are significantly improved; and long-range dependence modeling is realized with relatively low calculation cost through the Trans-Decrorator module, and downsampling information loss is compensated in combination with the RUp module, so that lightweight global modeling is realized.

Description

Technical Field

[0001] The present invention relates to weakly-supervised camouflaged object segmentation in images, belonging to the technical field of data recognition, and specifically relates to a weakly-supervised camouflaged object segmentation method based on long-distance diffusion. Background Art

[0002] Due to the high similarity of camouflaged objects with the background in terms of features such as color and texture, traditional fully-supervised segmentation methods rely on densely annotated data, but the cost of obtaining pixel-level annotations is high in practical applications. Existing weakly-supervised camouflaged object segmentation methods have the following deficiencies: 1. The utilization rate of annotation information is low, and the supervision signals of sparse annotations (such as points, scribbles) are difficult to effectively spread to the whole, resulting in blurred object edges and false detections in the background area. 2. The long-range dependence modeling is insufficient. Existing methods rely on local receptive fields and are difficult to capture global context information, resulting in limited segmentation performance in complex scenarios. 3. The edge sharpening ability is weak. Existing loss functions rely on a bidirectional diffusion mechanism, and the annotation information is easily interfered by the noise in the unannotated area, suppressing the edge sharpening effect. Summary of the Invention

[0003] The purpose of the present invention is to overcome the above deficiencies in the prior art and propose a weakly-supervised camouflaged object segmentation method based on long-distance diffusion to improve the ability of weakly-supervised camouflaged object detection.

[0004] The purpose of the present invention is achieved through the following technical solutions: A weakly-supervised camouflaged object segmentation method based on long-distance diffusion inputs an image containing a camouflaged object to be segmented into a pre-constructed and trained weakly-supervised camouflaged object segmentation model for camouflaged object segmentation; wherein, the weakly-supervised camouflaged object segmentation model adopts a two-stage training method including a backbone network, an RFB-modify module, an LCC module, a Trans-decorator module, an RUp module, and a GLSC loss function; the backbone network generates backbone features for the RFB-modify module and the LCC module to extract local features therefrom, the Trans-decorator module successively receives the backbone features and local features to update the global feature token and local features, the RUp module receives the output of the Trans-decorator module and generates the prediction features of the current layer, and the four-layer prediction features are fused to generate the final prediction map.

[0005] The backbone network extracts multi-scale backbone features of the image containing the camouflaged object to be segmented, including four layers of features F k , k ∈{1,2,3,4} and unifies the channels; The RFB-modify module and the LCC module integrate the dynamic activation function Dy-ReLU and respectively extract from the high-level backbone features F3 and F 4 extract semantic features F’ 3 and F’ 4 and extract structural features from low-level backbone features F 1 and F 2 to obtain local features; F’ 1 and F’ 2 to obtain local features; The Trans-Decorator module takes backbone features, local features, and learnable spatial tokens as inputs, and generates global enhanced features, restores spatial attention, and updated tokens through dynamic parameter fusion and spatial attention mechanisms; The RUp module uses spatial attention at the same level to achieve cross-resolution feature alignment, and adopts a cross-layer aggregation strategy to obtain the main output and auxiliary output sequences as the rough prediction map of the camouflage target area; The GLSC loss function first uses the pCE loss function to receive supervision information from the ground truth (points, scribbles); then, it minimizes the difference between the predicted values of similar pixels based on the pixel feature similarity to spread the supervision information globally; further, GLSC improves the diffusion efficiency of sparse annotations by restricting the unidirectional diffusion of annotation information to non-annotated positions; in the second training stage of the two-stage training method, the prediction map generated in the first training stage is used as a pseudo-label and other settings are the same as those in the first training stage. The GLSC loss function of the two-stage training will simultaneously enhance the diffusion of foreground and background annotations, optimize the confidence and sharpness of edge predictions, and obtain the final clear prediction map.

[0006] Furthermore, constructing and training a weakly supervised camouflage target segmentation model includes: S11, dividing the pre-collected image dataset containing camouflage targets into a training set and a test set; S12, constructing a weakly supervised camouflage target segmentation model; S13, using the training set to train the constructed weakly supervised camouflage target segmentation model; S14, using the test set to test the trained weakly supervised camouflage target segmentation model.

[0007] Furthermore, the Trans-Decorator module captures global information from the backbone multi-scale backbone features and generates and updates spatial tokens, which are Dy-ReLU parameters θThe key judgment basis; this module has three inputs: multi-scale backbone features F k , local features F’ k , and spatial tokens Z , randomly initialized by embedding vectors. First, a lightweight cross-attention A f→z is constructed to fuse the backbone features F k and spatial tokens Z . Meanwhile, the weights of this cross-attention are output to participate in the recovery upsampling. Then, a standard Transformer module A z is used to spatially weight the tokens and obtain updated tokens with global priors Z’ ; the updated tokens Z’ are used to calculate the parameter θ = f (z’ m ), to control the threshold of Dy-ReLU for the non-linear mapping of local features F’ k . Here, z’ m is the average value of all global tokens, and f (·) consists of a two-layer MLP with a ReLU activation function. In addition, Z’ is input into a simplified cross-attention A z→f to generate global perception features F” . The Trans-Decorator module outputs global enhanced features F k ”,k ∈{1, 2, 3, 4}, the updated tokens Z’ , and the cross-attention weights H , where Z’ is used as the input for the next level.

[0008] Furthermore, the RUp module uses the attention weights H and the updated tokens Z’ to generate the restored spatial attention H z = softmax ( g ( f ( Z’ )) ,H ), where g (·) represents matrix multiplication, f(·) consists of a two - layer MLP with ReLU activation function; then, the global perception features F k ” are upsampled and dot - multiplied with H z to achieve spatial feature alignment; in addition, the aligned feature channels are randomly shuffled to promote information exchange between channels; Furthermore, the GLSC loss function L glsc , the cross - view loss function L cv and the partial cross - entropy loss function L pce ; the total loss function of the weakly - supervised camouflage object segmentation model is: , where SSIM is the single - scale structural similarity index, is the prediction map of the auxiliary task; M is the total number of pixels, i is the pixel index; α , β and λ are both adaptive weights, , respectively represent the non - gradient and , is a neighborhood window centered at the position a∈ 1,N (coordinates are ([[]] a x ,a y )) with r as the radius, b represents a position other than a within the window, K ab represents P the weighted sum of the pixel feature similarities of m src kernels, K ab and m src can be written as: where represents the color feature (RGB) or position feature (coordinate) vector, σ is a hyperparameter, is the two - norm operation, 1 / ω is the normalization weight; s ∈ {0, 1} represents the valid annotation value, represents the absolute value. ​

[0009] The weakly supervised camouflaged target segmentation method based on long-distance diffusion according to claim 1 is characterized in that the second training stage uses the prediction map generated in the first training stage As pseudo-label , other settings remain the same as in the first training phase, and The specific mapping rules between them are as follows: .

[0010] Beneficial effects: The present invention uses deep learning technology to improve the performance of weakly supervised camouflaged target segmentation. Through the unidirectional gating mechanism of GLSC loss, noise interference is suppressed and the global diffusion efficiency of sparse annotation is improved; through the two-stage training strategy combined with the squeezing effect, the confidence and sharpness of the camouflaged target edge are significantly improved; through the Trans-Decorator module, long-range dependency modeling is achieved at a low computational cost, combined with the RUp module to compensate for the downsampling information loss, and lightweight global modeling is achieved; these features improve the detection ability of camouflaged targets and expand the use scenarios of the present invention; the present invention is a detection model trained on a large-scale data set, with good robustness and generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 This is an overview of the model method and loss group structure of the present invention. The left picture is the LRDNet model method, and the right picture is the loss group structure; Figure 2 It is a structural diagram of the Trans-Decorator module and the RUp module of the present invention; Figure 3 It is a structural diagram of the RFB-modify module and the LCC module of the present invention; Figure 4 It is an analysis diagram of the gating principle of the GLSC of the present invention; Figure 5 Schematic diagram of the two-stage annotated diffusion and extrusion process of the present invention; DETAILED DESCRIPTION

[0012] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0013] Explanation of the symbols and abbreviations used in this patent: Modified Receptive FieldBlock (RFB-modify); Local Context Comparison (LCC); Transformer Decorator (Trans-decorator); Dynamic ReLU (Dy-ReLU); Restoration Upsampling (RUp); Gated Local Saliency Coherence (GLSC); Local Saliency Coherence (LSC); Partial CrossEntropy (pCE).

[0014] See also Figure 1 , Figure 2 and Figure 3 A weakly supervised camouflaged target segmentation method based on long-distance diffusion comprises: inputting an image containing a camouflaged target to be segmented into a constructed and trained weakly supervised camouflaged target segmentation model to segment the camouflaged target; wherein the weakly supervised camouflaged target segmentation model comprises a backbone network, an RFB-modify module and an LCC module, a Trans-decorator module, an RUp module, a GLSC loss function and a two-stage training method; the backbone network extracts multi-scale backbone features, and has four layers, namely F k , k ∈{1,2,3,4}, and perform channel unification, where k Represents the backbone feature hierarchy; integrates the RFB-modify module of dynamic Dy-ReLU and the LCC module to extract local features, including extracting local features from high-level backbone features F 3 and F 4 Extracting semantic features from F’ 3 and F’ 4 , and from the low-level backbone features F 1 and F 2 Extracting structural features F’ 1 and F’ 2 ; Trans-Decorator module is based on the main features F’ k , local features Fk ’,k ∈{1,2,3,4} and learnable spatial tokens As input ( M , d are the number and dimension of tokens respectively), generating global enhanced features through dynamic parameter fusion and spatial attention mechanism F k ” , Cross Attention Weight H and the updated token Z’ ; Rup module realizes the recovery of spatial attention using the same level H z Achieve cross-resolution feature alignment; finally, use a cross-layer aggregation strategy to pool the fourth layer backbone features F 4 With global enhancement features F k ’’ Fusion to get the main output P 0 and auxiliary output sequence P 1 arrive P 4 The prediction results are used to complete the segmentation of the camouflaged target.

[0015] In this embodiment, constructing and training a weakly supervised camouflaged target segmentation model includes: S11, dividing a pre-collected image dataset containing camouflaged targets into a training set and a test set; S12, build a weakly supervised camouflage target segmentation model; S13, using the training set to train the constructed weakly supervised camouflaged target segmentation model; in this embodiment, the loss function for training the constructed weakly supervised camouflaged target segmentation model using the training set is the GLSC loss function L glsc , cross-view loss function L cv And the partial cross entropy loss function L pce The total loss function of the weakly supervised camouflaged target segmentation model is: ,in, SSIM is a single-scale structural similarity index, is the prediction graph of the auxiliary task; M is the total number of pixels, i is the pixel index; α , β and λ are adaptive weights, , Respectively represent the gradient-free and , By location a∈ [ 1,N ](Coordinates are( a x ,a y )) as the center, r is the neighborhood window with radius, b Indicates that the window is a A location other than K ab express P The weighted sum of the pixel feature similarities of the kernels, m src For one-way gating, K ab and m src It can be written as: in, Represents a color feature (RGB) or position feature (coordinate) vector, σ is a hyperparameter, is the two-norm operation, 1 / ω is the normalized weight; s ∈{0, 1} represents a valid annotation value, Indicates absolute value.

[0016] The second training phase uses the prediction graph generated in the first training phase As pseudo-label , other settings remain the same as in the first training phase, and The specific mapping rules between them are as follows: .

[0017] S14, use the test set to test the trained weakly supervised disguised target segmentation model.

[0018] In this embodiment, the backbone network extracts multi-scale backbone features of the image to be segmented containing the camouflaged target, which has four layers, namely F k , k ∈{1,2,3,4}. In general, the backbone network has been trained on the ImageNet dataset and has the ability to segment and divide. The most common networks such as VGG and ResNet can be used, and no specific restrictions are made here.

[0019] In this embodiment, the RFB-modify module and the LCC module integrating the dynamic activation function Dy-ReLU are as follows: Figure 3 As shown, the two are respectively based on the high-level backbone features F 3 and F4 Extracting semantic features from F’ 3 and F’ 4 , and from the low-level backbone features F 1 and F 2 Extracting structural features F’ 1 and F’ 2 , get local features F’ k ; In this embodiment, the Trans-decorator module structure is as follows Figure 2 As shown, the Trans-decorator module includes three inputs: multi-scale backbone features F k , local features F’ k , and space tokens Z , which is randomly initialized by the embedding vector. First, a lightweight cross attention A f→z To fuse the backbone features F k and space tokens Z At the same time, the weight of this cross attention Then, the standard Transformer module is used to restore the upsampling. A z Spatially weight the tokens and obtain updated tokens with global priors Z’ Specifically, the simplified cross attention A f→z and the standard Transformer module A z It can be described as follows: , , where the main feature F k and space tokens Z Split into h Head, that is F k =[ f 1 ···f h ], Z =[z 1 ··· z h ] for multi-head attention. i Splitting of heads With iTokens different. W i Q , W i K and W i V They are i The query, key, and value projection matrices for each header. W O Used to group multiple heads together. Attn(Q, K, V) is the standard attention function. [ · ] 1:h express h The splicing of elements, FFN {·} is a feed-forward network.

[0020] Updated token Z’ To calculate the parameters θ = f ( z’ m ) to control the local features F’ k The threshold of Dy-ReLU for nonlinear mapping. Here, z’ m is the average of all global tokens, and f (·) It consists of two layers of MLP with ReLU activation function (e.g. Figure 2 as shown). Z’ Input to simplified cross attention A z→f To generate global perception features F” : Among them, local features F k ’ Split into h Head, that is Trans-Decorator outputs global enhancement features F k ”,k ∈{1,2,3,4}, updated token Z’ And the cross attention weight H ,in Z’ is used as input to the next level.

[0021] In this embodiment, the RUp module structure is as follows: Figure 2 As shown in the lower right, the Rup module uses attention weights H and update token Z’ To generate the restored spatial attention H z = softmax( g ( f ( Z’ ) ,H )),in g (·) represents matrix multiplication, f (·) consists of two layers of MLP with ReLU activation function. Then, the global perception feature F k ” Upsample and compare H z Point multiplication is performed to achieve spatial feature alignment. In addition, in order to promote information exchange between channels, the aligned feature channels are randomly disrupted, and the upsampling RUp process can be written as: in, C 1×1 is a 1×1 convolutional layer with batch normalization and ReLU activation function, Bi(·) is bilinear interpolation, [ · ] g=c Indicates that c The random channels of the groups are shuffled.

[0022] In this embodiment, the GLSC loss function principle is as follows Figure 4 Specifically, LSC is based on the manifold assumption that pixels with similar features tend to have similar class probabilities. L 1 → 0. Pixel similarity pairs evaluated by gated conditional random fields L 1 The norm is weighted to force the category predictions of similar pixels to remain consistent. The specific calculation process is as follows: in, By location a∈ [ 1,N ](Coordinates are( a x ,a y )) as the center, r is the radius of the neighborhood window. b Indicates that the window is a a location other than . K ab express P The weighted sum of the pixel feature similarities of the kernels, m src For one-way gating, K ab and m src It can be written as: , ,in, Represents a color feature (RGB) or position feature (coordinate) vector, σ is a hyperparameter, is the two-norm operation, 1 / ω is the normalized weight.

[0023] With the help of pCE loss, LSC can diffuse the annotation information to the entire image, such as Figure 4 However, the minimization norm approximation is bidirectional, which means that a → b and b → a . This limits the dissemination of annotation information.

[0024] Therefore, the present invention adopts mask m src , in order to make full use of the annotation information to achieve effective partial one-way approximation. m src The rules are as follows: in, s∈ {0,1} represents a valid annotation value. Indicates absolute value.

[0025] In fact, m src LSC and pCE can be decoupled. In cases i and ii, the initial collaboration between LSC and pCE is chaotic because There are two back-propagation paths, one from (noise), another from s i By removing the The relevant calculation and parameter updating processes have been optimized.

[0026] By adding a gating mechanism to LSC, GLSC optimizes the annotation diffusion efficiency under specific conditions. The final GLSC can be written as: In this embodiment, the squeezing effect produced by the two-stage GLSC is as follows: Figure 5 As shown in Figure 2, two-stage training with GLSC loss achieves active diffusion, which revolves around the effective annotations. Under the influence of foreground annotation diffusion and background annotation diffusion, the uncertain edges are compressed and narrowed, thereby improving the edge prediction quality. As pseudo-label , other settings remain the same as in the first training phase, and The specific mapping rules between them are as follows: .

[0027] The above specific implementation modes are preferred embodiments of the present invention and cannot be used to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solution of the present invention are included in the protection scope of the present invention.

Claims

1. A weakly supervised camouflaged target segmentation method based on long-distance diffusion, inputting an image containing a camouflaged target to be segmented into a constructed and trained weakly supervised camouflaged target segmentation model to segment the camouflaged target, characterized in that: The weakly supervised camouflaged target segmentation model adopts a two-stage training method including a backbone network, an RFB-modify module, an LCC module, a Trans-decorator module, an Rup module and a GLSC loss function; the backbone network generates backbone features for the RFB-modify module and the LCC module to extract local features therefrom, the Trans-decorator module successively receives the backbone features and the local features to update the global feature token and the local features, the Rup module receives the output of the Trans-decorator module and generates the current layer prediction features, and the four layers of prediction features are fused to generate the final prediction map; the backbone network extracts the multi-scale backbone features of the image to be segmented containing the camouflaged target, including four layers of features F k , k ∈{1,2,3,4} and perform channel unification; The RFB-modify module and the LCC module integrate the dynamic activation function Dy-ReLU from the high-level backbone features F 3 and F 4 Extracting semantic features F’ 3 and F’ 4, and from the low-level backbone features F 1 and F 2 Extracting structural features F’ 1 and F’ 2. Obtain local features; The Trans-Decorator module takes backbone features, local features, and learnable spatial tokens as input, generates global enhanced features, restores spatial attention, and updated tokens through dynamic parameter fusion and spatial attention mechanism; The RUp module uses the same-level spatial attention to achieve cross-resolution feature alignment, and adopts a cross-layer aggregation strategy to obtain the main output and auxiliary output sequences as a rough prediction map of the camouflaged target area; The GLSC loss function first uses the pCE loss function to receive supervision information from the true value; then, the difference between the predicted values ​​of similar pixels is minimized according to the pixel feature similarity to diffuse the supervision information globally; further, GLSC improves the diffusion efficiency of sparse annotations by limiting the unidirectional diffusion of annotation information to non-annotated locations; in the two-stage training method, the second training stage converts the prediction map generated in the first training stage into Used as pseudo labels ,The other settings remain the same as the first training stage.,The GLSC loss function of the two-stage training will simultaneously enhance the,diffusion of foreground and background annotations, optimize the confidence and sharpness of edge,prediction, and obtain the final clear prediction image.

2. The weakly supervised camouflaged target segmentation method based on long-distance diffusion according to claim 1 is characterized in that: Building and training a weakly supervised camouflaged object segmentation model involves: S11, dividing a pre-collected image dataset containing camouflaged targets into a training set and a test set; S12, build a weakly supervised camouflage target segmentation model; S13, using the training set to train the constructed weakly supervised camouflaged target segmentation model; S14, use the test set to test the trained weakly supervised disguised target segmentation model.

3. The weakly supervised camouflaged target segmentation method based on long-distance diffusion according to claim 1 is characterized in that: Trans-Decorator module, which captures global information from the backbone multi-scale backbone features and generates and updates spatial tokens, which are Dy-ReLU parameters θ The key judgment basis of this module is: the multi-scale backbone features F k , local features F’ k , and space tokens Z , randomly initialized by the embedding vector, first, a lightweight cross attention A f→z To fuse the backbone features F k and space tokens Z , at the same time, the weight of this cross attention is output to participate in the recovery upsampling, and then, the standard Transformer module is used A z Spatially weight the tokens and obtain updated tokens with global priors Z’ ; Updated token Z’ To calculate the parameters θ = f (z' m ) to control the local features F’ k The threshold of the Dy-ReLU of the nonlinear mapping, here, z’ m is the average of all global tokens, and f (·) consists of two layers of MLP with ReLU activation function, in addition, Z’ Input to simplified cross attention A z→f To generate global perception features F” , the Trans-Decorator module outputs global enhanced features F k ”,k ∈{1,2,3,4}, updated token Z’ , and the cross attention weights H ,in Z’ is used as input to the next level.

4. The weakly supervised camouflaged target segmentation method based on long-distance diffusion according to claim 1 is characterized in that: RUp module, using attention weights H and update token Z’ To generate the restored spatial attention H z =softmax ( g ( f ( Z’ ) , H )),in g (·) represents matrix multiplication, f (·) It consists of two layers of MLP with ReLU activation function; then, the global perception features F k ” Upsample and compare H z Perform point multiplication to achieve spatial feature alignment; In addition, the aligned feature channels are randomly shuffled to promote information exchange between channels.

5. The weakly supervised camouflaged target segmentation method based on long-distance diffusion according to claim 1 is characterized in that: GLSC loss function L glsc , cross-view loss function L cv And the partial cross entropy loss function L pce ; The total loss function of the weakly supervised camouflaged target segmentation model is: ,in, SSIM is a single-scale structural similarity index, is the prediction graph of the auxiliary task; M is the total number of pixels, i is the pixel index; α , β and λ are adaptive weights, , Respectively represent the gradient-free and , By location a∈ [ 1,N ](Coordinates are( a x ,a y )) as the center, r is the neighborhood window with radius, b Indicates that the window is a A location other than K ab express P The weighted sum of the pixel feature similarities of the kernels, m src For one-way gating, K ab and m src It can be written as: in, Represents a color feature (RGB) or position feature (coordinate) vector, σ is a hyperparameter, is the two-norm operation, 1 / ω is the normalized weight; s ∈{0, 1} represents a valid annotation value, Indicates absolute value.

6. The weakly supervised camouflaged target segmentation method based on long-distance diffusion according to claim 1 is characterized in that: The second training phase uses the prediction graph generated in the first training phase As pseudo-label , other settings remain the same as in the first training phase, and The specific mapping rules between them are as follows: .

Images