A Pseudo-Label Optimization Method for Camouflaged Target Detection Based on Point Supervision

Abstract

The application discloses a camouflage target detection method based on point supervision and pseudo label optimization, and specifically comprises the following steps: inputting an image into a backbone network to obtain a corresponding initial prediction map; composing point annotations and boundary box annotations corresponding to the image into a multi-prompt input all things segmentation model to obtain point masks and boundary box masks through segmentation; adopting a semantic entropy multi-mask weighted fusion mechanism to obtain pseudo labels; constructing a global-local perception module, wherein the global-local perception module comprises a global perception module and a local perception module; adopting binary cross entropy loss and contrast loss as a final loss function to train a camouflage target detection model; and the application has the beneficial effects that: (1) the annotation cost is reduced while the quality of the supervision signal is improved; and (2) the sparse point annotation information is fully utilized, and the accuracy of the weakly supervised camouflage target detection is improved.

Description

Technical Field

[0001] This invention relates to the field of camouflaged target detection, specifically to a camouflaged target detection method based on point supervision and pseudo-label optimization. Background Technology

[0002] Camouflaged target detection is a computer vision task aimed at identifying targets that blend seamlessly with the background and whose visual features are not readily apparent. Because camouflaged targets typically exhibit high similarity to their surroundings in terms of color, texture, and shape, their detection is far more challenging than general target detection tasks. Early camouflaged target detection methods relied primarily on manually designed shallow features; however, these handcrafted features have limited expressive power and struggle to handle target concealment in complex backgrounds, resulting in low detection accuracy. With the development of deep learning and the release of large-scale pixel-level labeled datasets, convolutional neural network-based camouflaged target detection methods have made significant progress. However, most existing high-performance camouflaged target detection models rely on fully supervised training, requiring a large amount of manual input of pixel-level masks as training labels. This label acquisition process is extremely time-consuming, with an average annotation cost of up to 60 minutes per image, severely limiting the scalability and practicality of the algorithms.

[0003] Due to the high visual similarity between the target and the background, existing deep learning methods still face the following core challenges in camouflaged target detection: (1) Over-reaction to background regions: some methods generate incorrect activation in the background region, resulting in false detection; (2) Incomplete activation of target regions: only local regions of the target can be identified, and the overall outline is unclear; (3) Inaccurate boundary detection: it is difficult to accurately distinguish between the target and the background edge, resulting in blurred segmentation.

[0004] To reduce the time cost of annotation, some studies have begun to explore weakly supervised camouflage target detection methods. Existing weakly supervised camouflage target detection methods still generally have the following problems: (1) The supervision signals are too sparse, which limits the model's learning ability; (2) There is still a significant gap in accuracy between weakly supervised methods and fully supervised methods; (3) Although existing improved methods have integrated multiple weak supervision signals, they still rely on a large number of points or boxes for manual annotation, which restricts their annotation efficiency and practical application promotion. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a pseudo-label optimization method for detecting camouflaged targets based on point supervision, which improves the detection accuracy of camouflaged objects as much as possible while reducing the annotation time cost and achieving better segmentation results.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a camouflaged target detection method based on point supervision and optimized pseudo-labels, the detection method comprising the following steps:

[0007] Step S1: Input the image into the backbone network for preprocessing to obtain the corresponding initial prediction map;

[0008] Step S2: Based on the image and the initial prediction map from Step S1, obtain multiple prompts, input the multiple prompts into the object segmentation model, and obtain the mask;

[0009] Step S3: The initial prediction map obtained in step S1 and the mask obtained in step S2 are weighted and fused using the semantic entropy multi-mask weighted fusion mechanism to obtain pseudo-labels.

[0010] Step S4: Construct a global-local perception module to obtain global-local interaction enhancement features;

[0011] Step S5: Perform bidirectional cross-layer fusion on the global-local interactive enhancement features obtained in step S4 to obtain fused features, and process the obtained fused features to obtain a prediction map.

[0012] Step S6: Construct a camouflaged target detection model based on the pseudo-labels obtained in step S3 and the prediction map obtained in step S5.

[0013] Furthermore, in step S1, the image is input into the backbone network for preprocessing to obtain the corresponding initial prediction map; specifically:

[0014] Using the point annotations in the image as the center and r as the radius, a circular supervision signal is obtained. This circular supervision signal is then used to supervise the backbone network to obtain the corresponding initial prediction map.

[0015] Furthermore, in step 2, multiple prompts are obtained based on the image and the initial prediction map from step S1. These multiple prompts are then input into the object segmentation model to obtain a mask; specifically:

[0016] Step 21: Based on the maximum bounding matrix in the initial prediction map of step S1, obtain the bounding box annotations; combine the point annotations of the image and the obtained bounding box annotations to form a multi-cue.

[0017] Step 22: Input the multi-cue input into the object segmentation model to obtain point masks and bounding box masks.

[0018] Furthermore, in step S3, a semantic entropy multi-mask weighted fusion mechanism is used to weight and fuse the initial prediction map obtained in step S1 and the mask obtained in step S2 to obtain pseudo-labels; the mask is divided into point mask and boundary mask: specifically:

[0019] Step S31, the initial prediction map and dot mask respectively and bounding box mask The point semantic entropy and bounding box semantic entropy are calculated using the following formulas:

[0020] ;

[0021] in, Represents point semantic entropy, Represents dot product operation. Represents the logarithmic operation. Represents the smallest positive number;

[0022] ;

[0023] in, Represents the semantic entropy of the bounding box;

[0024] Step S32, dot mask and bounding box mask Perform spatial domain decomposition to extract overlapping regions. = ⋂ Non-overlapping areas are designated as exclusive areas. = - And bounding box hints for exclusive areas = - ;

[0025] Step S33, based on the calculated point semantic entropy and bounding box semantic entropy The point mask weights and bounding box mask weights are obtained by assigning weights to non-overlapping regions based on semantic entropy; the formulas are as follows:

[0026] ;

[0027] ;

[0028] in, Represents the point mask weights. Represents the weights of the bounding box mask;

[0029] Step S34, for overlapping areas Non-overlapping areas are designated as exclusive areas. And bounding box hints for exclusive areas According to the dot mask weight and bounding box mask weights The pseudo-labels are obtained by weighted fusion; the formula is as follows:

[0030] ;

[0031] in, This represents a pseudo-tag.

[0032] Furthermore, in step S4, a global-local perception module is constructed to obtain global-local interaction enhancement features; specifically:

[0033] Step S41: Input the image into the backbone network, extract the initial features of the image, and construct a global-local perception module;

[0034] Step S42: Based on the global-local perception module constructed in step S41; the global-local perception module includes a global perception module and a local perception module. The global perception module obtains global perception features, the local perception module obtains local perception features, and global-local interaction enhancement features are obtained based on the global perception features and the local perception features.

[0035] Furthermore, in step S42, the global perception module obtains the global perception features; specifically:

[0036] Step S4201: The initial features of the image are processed by two parallel dilated convolutions with kernel sizes of 3×3 and 5×5 to extract spatial features under different receptive fields. The extracted spatial features are then added to the initial features of the image through a residual structure to obtain residual features.

[0037] Step S4202: Input the residual features into an improved frequency domain self-attention mechanism to obtain the spatial domain residual features, and then transform the spatial domain residual features into frequency domain features through fast Fourier transform to obtain frequency domain query features, frequency domain matching features and frequency domain information features.

[0038] Step S4203: Perform dimensional transformation on the obtained frequency domain query features, frequency domain matching features, and frequency domain information features to obtain dimensionally transformed frequency domain query features, frequency domain matching features, and frequency domain information features. Perform matrix multiplication on the dimensionally transformed frequency domain query features and frequency domain matching features to generate the transposed reference diagram A. f ;

[0039] Step S4204, note the transpose diagram A. f Perform the calculation; the formula is as follows:

[0040] ;

[0041] in, Representative transpose attention diagram A f The real part of a complex number, conj represents the calculation of the conjugate value of the complex number;

[0042] ;

[0043] in, Representative transpose attention diagram A f The imaginary part of a complex number, where j represents the imaginary unit in complex number theory;

[0044] Step S4205: Activate the real part and the imaginary part of the complex number of the transposed attention map using the flexible maximum activation function and then recombine them to obtain the intermediate attention map. Then, perform matrix multiplication on the intermediate attention map and the frequency domain information features after dimension transformation, and obtain the preliminary frequency enhancement features by inverse fast Fourier transform back to the spatial domain features.

[0045] Step S4206: Frequency residual connection yields frequency enhancement features; the formula is as follows:

[0046] ;

[0047] in, Represents frequency enhancement characteristics, Represents an indicator function. Represents the inverse fast Fourier transform. Modify the activation function of the linear unit. Represents Fast Fourier Transform, Represents residual characteristics, Represents matrix multiplication;

[0048] Step S4207: The initial frequency enhancement feature and the frequency enhancement feature are concatenated in the channel dimension, and the channel dimension is transformed by a 1×1 convolution to obtain the frequency domain self-attention enhancement feature;

[0049] Step S4208: The initial features of the image and the frequency domain self-attention enhancement features are added through a residual connection, and the frequency domain enhancement features are obtained through a layer normalization process. The frequency domain enhancement features are then input into a depth-separable convolutional group with three parallel convolutional kernels of 3×3, 5×5 and 7×7 to obtain the orientation enhancement features.

[0050] Step S4209: The frequency domain enhancement features and the direction enhancement features are combined through a residual structure to obtain the global perception features.

[0051] Furthermore, in step S42, the local perception module obtains local perception features; specifically:

[0052] Step S4210: The initial features of the image are fed into the depthwise convolution branches with three parallel convolution kernels of 1×3, 3×1, and 3×3. The depthwise convolution branches extract local features in three different directions, namely, the local features in the horizontal direction. Vertical local features Local features of the same polarity as those in directions other than horizontal and vertical. ;

[0053] Step S4211: The extracted local features from three different directions are concatenated along the channel dimension and activated using the Gaussian error linear unit activation function, and then combined with the initial features of the image. The directional characteristics are obtained by adding them together; the formula is as follows:

[0054] ;

[0055] in, Representing directional characteristics, Conv 1x1 Represents a regular 1x1 convolution. Represents the splicing of channel dimensions;

[0056] Step S4212: The orientation features are processed by a layer normalization and a feedforward neural network, and the processed orientation features are added to a residual structure to obtain local perceptual features.

[0057] Furthermore, in step S42, global-local interaction enhancement features are obtained based on global perception features and local perception features; specifically:

[0058] Step S4213: Divide the global perception features obtained in step S4209 and the local perception features obtained in step S4212 into four equal parts along the channel dimension to obtain four sets of global perception features and four sets of local perception features. The size of the four sets of global perception features and the four sets of local perception features is C / 4×H×W; where C represents the number of channels, H represents the height, and W represents the width.

[0059] Step S4214: After compressing the spatial dimension of the four sets of global perception features by global average pooling, the four sets of global perception features are concatenated to obtain four new sets of global perception features. After compressing the space of the four sets of local perception features by global average pooling, the four sets of local perception features are concatenated to obtain four new sets of local perception features.

[0060] Step S4215: The four new sets of global perception features and the four new sets of local perception features are respectively calculated by the lightweight fully connected network to obtain four sets of normalized fusion weights; wherein the lightweight fully connected network includes two fully connected layers and a modified linear unit activation function.

[0061] Step S4216: Using four sets of normalized fusion weights, the four sets of global perception features and the four sets of local perception features are weighted and fused to obtain four sets of global-local features, and the four sets of global-local features are spliced ​​together in the channel dimension.

[0062] By fusing global and local perceptual features using residual connections, global-local enhanced features are obtained; the formula is as follows:

[0063] ;

[0064] Where i represents the level and m represents the group number. Represents the global-local enhancement features of the i-th layer. Represents the fusion weights of the m-th group in the i-th layer. This represents the global perception feature of the m-th group in the i-th layer. This represents the local perceptual features of the m-th group in the i-th layer. This represents the global perception features of the i-th layer. Represents the local perceptual features of the i-th layer;

[0065] Step S4217: After the global-local enhancement features are processed by the gated convolution mechanism, they are fused with the global-local enhancement features through residual connections to obtain global-local interactive enhancement features. These global-local interactive enhancement features are divided into four layers: the initial global-local interactive enhancement features, the second adjacent layer global-local interactive enhancement features, the third adjacent layer global-local interactive enhancement features, and the deep global-local interactive enhancement features; the formula is as follows:

[0066] ;

[0067] in, Represents global-local interaction enhancement features Representatives criticized normalization. This represents a 3×3 gated convolution.

[0068] Furthermore, in step S5, the global-local interactive enhancement features obtained in step S4 are subjected to bidirectional cross-layer fusion to obtain fused features, and the obtained fused features are processed to obtain a prediction map; specifically:

[0069] Step S51: Apply a 3x3 depthwise separable convolution to the deep global-local interaction enhancement features to obtain the separable features of the deep global-local interaction enhancement features;

[0070] The separable features of the deep global-local interaction enhancement features are generated by passing a set of point convolution groups and a sigmoid activation function to produce a first feature weight map. The first feature weight map is then multiplied with the global-local interaction enhancement features of the third adjacent layer to obtain the first layer semantic enhancement features.

[0071] Global flat pooling is performed on the global-local interaction enhancement features of the third adjacent layer. Then, a second feature weight map is generated through a set of 1x1 convolutions and sigmoid activation functions. The second feature weight map is multiplied with the separable features of the deep global-local interaction enhancement features to obtain the first layer of local spatial refinement features.

[0072] The obtained first-layer semantic enhancement features and first-layer local spatial refinement features are aggregated to obtain the first-layer fusion features;

[0073] Applying a 3x3 depthwise separable convolution to the first layer of fused features yields the separable features of the first layer of fused features;

[0074] The separable features of the first layer fusion features are used to generate a third feature weight map through a set of point convolution groups and a sigmoid activation function. The third feature weight map is then multiplied with the global-local interaction enhancement features of the second adjacent layer to obtain the second layer semantic enhancement features.

[0075] Global flat pooling is performed on the global-local interaction enhancement features of the second adjacent layer. Then, a fourth feature weight map is generated through a set of 1x1 convolutions and sigmoid activation functions. The fourth feature weight map is multiplied with the separable features of the first layer fusion features to obtain the second layer local spatial refinement features.

[0076] The obtained second-layer semantic enhancement features and second-layer local spatial refinement features are aggregated to obtain the second-layer fusion features;

[0077] Applying a 3x3 depthwise separable convolution to the second-layer fused features yields the separable features of the second-layer fused features;

[0078] The separable features of the second layer of fusion features are used to generate the fifth feature weight map through a set of point convolution groups and a sigmoid activation function. The fifth feature weight map is then multiplied with the initial layer global-local interaction enhancement features to obtain the third layer of semantic enhancement features.

[0079] Global flat pooling is performed on the initial global-local interaction enhancement features. Then, a sixth feature weight map is generated through a set of 1x1 convolutions and sigmoid activation functions. The sixth feature weight map is multiplied with the separable features of the second layer fusion features to obtain the third layer local spatial refinement features.

[0080] The third-layer semantic enhancement features and the third-layer local spatial refinement features are aggregated to obtain the final layer of fused features;

[0081] Step S52: The last layer of fused features is processed using 1x1 convolution, batch normalization, and sigmoid activation function to obtain the prediction map.

[0082] Furthermore, in step S6, a camouflage target detection model is constructed based on the pseudo-labels obtained in step S3 and the prediction map obtained in step S5. The binary cross-entropy loss function and the contrastive loss function are used as the final loss functions to train the constructed camouflage target detection model; specifically:

[0083] Step S61: Calculate the partial binary cross-entropy loss and contrast loss of the prediction graph, using the following formulas:

[0084] ;

[0085] in, Represents the binary cross-entropy loss. Representative pseudo-label The true category of a pixel location in an image. The predicted category represents the pixel location in the image from which the predicted map is located;

[0086] ;

[0087] in, Represents the contrast loss, where d represents the number of pixels in the image. The predicted image representing the image. The image represents a predicted image obtained after Gaussian blurring, color jittering, and random rotation.

[0088] Step S62: Train the camouflage target detection model based on the final loss function.

[0089] The beneficial effects of this invention are:

[0090] (1) Based on the advantages and disadvantages of the current weakly supervised camouflage target detection methods, bounding box annotations are obtained through point annotations. Combined with the Segmentation of Everything (SAM) model and the multi-mask weighted fusion mechanism based on semantic entropy, the quality of the supervision signal is improved while reducing the annotation cost.

[0091] (2) This invention not only considers the cross-layer fusion of global-local interaction enhancement features in most camouflage target detection methods, but also considers the global-local interaction enhancement features of global perception features and local perception features, which solves the problems of unstable response and blurred boundaries in detection and improves the accuracy of weakly supervised camouflage target detection. Attached Figure Description

[0092] Figure 1 This is a schematic diagram of the structure of the present invention. Detailed Implementation

[0093] like Figure 1 As shown, a pseudo-label optimization method for camouflaged target detection based on point supervision is proposed, the method comprising the following steps:

[0094] Step S1: Input the image into the backbone network for preprocessing to obtain the corresponding initial prediction map;

[0095] Step S2: Based on the image and the initial prediction map from Step S1, obtain multiple prompts, input the multiple prompts into the object segmentation model, and obtain the mask;

[0096] Step S3: The initial prediction map obtained in step S1 and the mask obtained in step S2 are weighted and fused using the semantic entropy multi-mask weighted fusion mechanism to obtain pseudo-labels.

[0097] Step S4: Construct a global-local perception module to obtain global-local interaction enhancement features;

[0098] Step S5: Perform bidirectional cross-layer fusion on the global-local interactive enhancement features obtained in step S4 to obtain fused features, and process the obtained fused features to obtain a prediction map.

[0099] Step S6: Construct a camouflaged target detection model based on the pseudo-labels obtained in step S3 and the prediction map obtained in step S5.

[0100] Furthermore, in step S1, the image is input into the backbone network for preprocessing to obtain the corresponding initial prediction map; specifically:

[0101] Using the point annotations in the image as the center and r as the radius, a circular supervision signal is obtained. This circular supervision signal is then used to supervise the backbone network to obtain the corresponding initial prediction map.

[0102] Furthermore, in step 2, multiple prompts are obtained based on the image and the initial prediction map from step S1. These multiple prompts are then input into the object segmentation model to obtain a mask; specifically:

[0103] Step 21: Based on the maximum bounding matrix in the initial prediction map of step S1, obtain the bounding box annotations; combine the point annotations of the image and the obtained bounding box annotations to form a multi-cue.

[0104] Step 22: Input the multi-cue input into the object segmentation model to obtain point masks and bounding box masks.

[0105] Furthermore, in step S3, a semantic entropy multi-mask weighted fusion mechanism is used to weight and fuse the initial prediction map obtained in step S1 and the mask obtained in step S2 to obtain pseudo-labels; the mask is divided into point mask and boundary mask: specifically:

[0106] Step S31, the initial prediction map and dot mask respectively and bounding box mask The point semantic entropy and bounding box semantic entropy are calculated using the following formulas:

[0107] ;

[0108] in, Represents point semantic entropy, Represents dot product operation. Represents the logarithmic operation. Represents the smallest positive number;

[0109] ;

[0110] in, Represents the semantic entropy of the bounding box;

[0111] Step S32, dot mask and bounding box mask Perform spatial domain decomposition to extract overlapping regions. = ⋂ Non-overlapping areas are designated as exclusive areas. = - And bounding box hints for exclusive areas = - ;

[0112] Step S33, based on the calculated point semantic entropy and bounding box semantic entropy The point mask weights and bounding box mask weights are obtained by assigning weights to non-overlapping regions based on semantic entropy; the formulas are as follows:

[0113] ;

[0114] ;

[0115] in, Represents the point mask weights. Represents the weights of the bounding box mask;

[0116] Step S34, for overlapping areas Non-overlapping areas are designated as exclusive areas. And bounding box hints for exclusive areas According to the dot mask weight and bounding box mask weights The pseudo-labels are obtained by weighted fusion; the formula is as follows:

[0117] ;

[0118] in, This represents a pseudo-tag.

[0119] Furthermore, in step S4, a global-local perception module is constructed to obtain global-local interaction enhancement features; specifically:

[0120] Step S41: Input the image into the backbone network, extract the initial features of the image, and construct a global-local perception module;

[0121] Step S42: Based on the global-local perception module constructed in step S41; the global-local perception module includes a global perception module and a local perception module. The global perception module obtains global perception features, the local perception module obtains local perception features, and global-local interaction enhancement features are obtained based on the global perception features and the local perception features.

[0122] Furthermore, in step S42, the global perception module obtains the global perception features; specifically:

[0123] Step S4201: The initial features of the image are processed by two parallel dilated convolutions with kernel sizes of 3×3 and 5×5 to extract spatial features under different receptive fields. The extracted spatial features are then added to the initial features of the image through a residual structure to obtain residual features.

[0124] Step S4202: Input the residual features into an improved frequency domain self-attention mechanism to obtain the spatial domain residual features, and then transform the spatial domain residual features into frequency domain features through fast Fourier transform to obtain frequency domain query features, frequency domain matching features and frequency domain information features.

[0125] Step S4203: Perform dimensional transformation on the obtained frequency domain query features, frequency domain matching features, and frequency domain information features to obtain dimensionally transformed frequency domain query features, frequency domain matching features, and frequency domain information features. Perform matrix multiplication on the dimensionally transformed frequency domain query features and frequency domain matching features to generate the transposed reference diagram A. f ;

[0126] Step S4204, note the transpose diagram A. f Perform the calculation; the formula is as follows:

[0127] ;

[0128] in, Representative transpose attention diagram A f The real part of a complex number, conj represents the calculation of the conjugate value of the complex number;

[0129] ;

[0130] in, Representative transpose attention diagram A f The imaginary part of a complex number, where j represents the imaginary unit in complex number theory;

[0131] Step S4205: Activate the real part and the imaginary part of the complex number of the transposed attention map using the flexible maximum activation function and then recombine them to obtain the intermediate attention map. Then, perform matrix multiplication on the intermediate attention map and the frequency domain information features after dimension transformation, and obtain the preliminary frequency enhancement features by inverse fast Fourier transform back to the spatial domain features.

[0132] Step S4206: Frequency residual connection yields frequency enhancement features; the formula is as follows:

[0133] ;

[0134] in, Represents frequency enhancement characteristics, Represents an indicator function. Represents the Fast Inverse Fourier Transform. Modify the linear unit activation function. Represents Fast Fourier Transform, Represents residual characteristics, Represents matrix multiplication;

[0135] Step S4207: The initial frequency enhancement feature and the frequency enhancement feature are concatenated in the channel dimension, and the channel dimension is transformed by a 1×1 convolution to obtain the frequency domain self-attention enhancement feature;

[0136] Step S4208: The initial features of the image and the frequency domain self-attention enhancement features are added through a residual connection, and the frequency domain enhancement features are obtained through a layer normalization process. The frequency domain enhancement features are then input into a depth-separable convolutional group with three parallel convolutional kernels of 3×3, 5×5 and 7×7 to obtain the orientation enhancement features.

[0137] Step S4209: The frequency domain enhancement features and the direction enhancement features are combined through a residual structure to obtain the global perception features.

[0138] Furthermore, in step S42, the local perception module obtains local perception features; specifically:

[0139] Step S4210: The initial features of the image are fed into the depthwise convolution branches with three parallel convolution kernels of 1×3, 3×1, and 3×3. The depthwise convolution branches extract local features in three different directions, namely, the local features in the horizontal direction. Vertical local features Local features of the same polarity as those in directions other than horizontal and vertical. ;

[0140] Step S4211: The extracted local features from three different directions are concatenated along the channel dimension and activated using the Gaussian error linear unit activation function, and then combined with the initial features of the image. The directional characteristics are obtained by adding them together; the formula is as follows:

[0141] ;

[0142] in, Representing directional characteristics, Conv 1x1 Represents a regular 1x1 convolution. Represents the splicing of channel dimensions;

[0143] Step S4212: The orientation features are processed by a layer normalization and a feedforward neural network, and the processed orientation features are added to a residual structure to obtain local perceptual features.

[0144] Furthermore, in step S42, global-local interaction enhancement features are obtained based on global perception features and local perception features; specifically:

[0145] Step S4213: Divide the global perception features obtained in step S4209 and the local perception features obtained in step S4212 into four equal parts along the channel dimension to obtain four sets of global perception features and four sets of local perception features. The size of the four sets of global perception features and the four sets of local perception features is C / 4×H×W; where C represents the number of channels, H represents the height, and W represents the width.

[0146] Step S4214: After compressing the spatial dimension of the four sets of global perception features by global average pooling, the four sets of global perception features are concatenated to obtain four new sets of global perception features. After compressing the space of the four sets of local perception features by global average pooling, the four sets of local perception features are concatenated to obtain four new sets of local perception features.

[0147] Step S4215: The four new sets of global perception features and the four new sets of local perception features are respectively calculated by the lightweight fully connected network to obtain four sets of normalized fusion weights; wherein the lightweight fully connected network includes two fully connected layers and a modified linear unit activation function.

[0148] Step S4216: Using four sets of normalized fusion weights, the four sets of global perception features and the four sets of local perception features are weighted and fused to obtain four sets of global-local features, and the four sets of global-local features are spliced ​​together in the channel dimension.

[0149] By fusing global and local perceptual features using residual connections, global-local enhanced features are obtained; the formula is as follows:

[0150] ;

[0151] Where i represents the level and m represents the group number. Represents the global-local enhancement features of the i-th layer. Represents the fusion weights of the m-th group in the i-th layer. This represents the global perception feature of the m-th group in the i-th layer. This represents the local perceptual features of the m-th group in the i-th layer. This represents the global perception features of the i-th layer. Represents the local perceptual features of the i-th layer;

[0152] Step S4217: After the global-local enhancement features are processed by the gated convolution mechanism, they are fused with the global-local enhancement features through residual connections to obtain global-local interactive enhancement features. These global-local interactive enhancement features are divided into four layers: the initial global-local interactive enhancement features, the second adjacent layer global-local interactive enhancement features, the third adjacent layer global-local interactive enhancement features, and the deep global-local interactive enhancement features; the formula is as follows:

[0153] ;

[0154] in, Represents global-local interaction enhancement features Representatives criticized normalization. This represents a 3×3 gated convolution.

[0155] Furthermore, in step S5, the global-local interactive enhancement features obtained in step S4 are subjected to bidirectional cross-layer fusion to obtain fused features, and the obtained fused features are processed to obtain a prediction map; specifically:

[0156] Step S51: Apply a 3x3 depthwise separable convolution to the deep global-local interaction enhancement features to obtain the separable features of the deep global-local interaction enhancement features;

[0157] The separable features of the deep global-local interaction enhancement features are generated by passing a set of point convolution groups and a sigmoid activation function to produce a first feature weight map. The first feature weight map is then multiplied with the global-local interaction enhancement features of the third adjacent layer to obtain the first layer semantic enhancement features.

[0158] Global flat pooling is performed on the global-local interaction enhancement features of the third adjacent layer. Then, a second feature weight map is generated through a set of 1x1 convolutions and sigmoid activation functions. The second feature weight map is multiplied with the separable features of the deep global-local interaction enhancement features to obtain the first layer of local spatial refinement features.

[0159] The obtained first-layer semantic enhancement features and first-layer local spatial refinement features are aggregated to obtain the first-layer fusion features;

[0160] Applying a 3x3 depthwise separable convolution to the first layer of fused features yields the separable features of the first layer of fused features;

[0161] The separable features of the first layer fusion features are used to generate a third feature weight map through a set of point convolution groups and a sigmoid activation function. The third feature weight map is then multiplied with the global-local interaction enhancement features of the second adjacent layer to obtain the second layer semantic enhancement features.

[0162] Global flat pooling is performed on the global-local interaction enhancement features of the second adjacent layer. Then, a fourth feature weight map is generated through a set of 1x1 convolutions and sigmoid activation functions. The fourth feature weight map is multiplied with the separable features of the first layer fusion features to obtain the second layer local spatial refinement features.

[0163] The obtained second-layer semantic enhancement features and second-layer local spatial refinement features are aggregated to obtain the second-layer fusion features;

[0164] Applying a 3x3 depthwise separable convolution to the second-layer fused features yields the separable features of the second-layer fused features;

[0165] The separable features of the second layer of fusion features are used to generate the fifth feature weight map through a set of point convolution groups and a sigmoid activation function. The fifth feature weight map is then multiplied with the initial layer global-local interaction enhancement features to obtain the third layer of semantic enhancement features.

[0166] Global flat pooling is performed on the initial global-local interaction enhancement features. Then, a sixth feature weight map is generated through a set of 1x1 convolutions and sigmoid activation functions. The sixth feature weight map is multiplied with the separable features of the second layer fusion features to obtain the third layer local spatial refinement features.

[0167] The third-layer semantic enhancement features and the third-layer local spatial refinement features are aggregated to obtain the final layer of fused features;

[0168] Step S52: The last layer of fused features is processed using 1x1 convolution, batch normalization, and sigmoid activation function to obtain the prediction map.

[0169] Furthermore, in step S6, a camouflage target detection model is constructed based on the pseudo-labels obtained in step S3 and the prediction map obtained in step S5. The binary cross-entropy loss function and the contrastive loss function are used as the final loss functions to train the constructed camouflage target detection model; specifically:

[0170] Step S61: Calculate the partial binary cross-entropy loss and contrast loss of the prediction graph, using the following formulas:

[0171] ;

[0172] in, Represents the binary cross-entropy loss. Representative pseudo-label The true category of a pixel location in an image. The predicted category represents the pixel location in the image from which the predicted map is located;

[0173] ;

[0174] in, Represents the contrast loss, where d represents the number of pixels in the image. The predicted image representing the image. The image represents a predicted image obtained after Gaussian blurring, color jittering, and random rotation.

[0175] Step S62: Train the camouflage target detection model based on the final loss function.

[0176] The backbone network in step S1 is PVTV2-B4.

[0177] Among them, the object segmentation model in step S2 is SAM.

[0178] In step S4, the pseudo-label serves as a monitoring signal.

[0179] In step S4203, transpose attention diagram A f Since it is a complex number, the softmax activation function cannot be used directly for activation. The transpose needs to be considered (see Figure A). f Calculate the real and imaginary parts of a complex number separately.

[0180] In step S4217, a gated convolution mechanism is further introduced to suppress redundant information in the global-local interaction enhancement features and enhance the expression of key regions.

[0181] The method of this invention (a pseudo-label optimization method for camouflaged target detection based on point supervision) and other methods were compared and evaluated on the concealed target detection dataset CAMO, the camouflaged target detection dataset COD10K, and the camouflaged object detection dataset NC4K. Tables 1, 2, and 3 show the segmentation results of the method of this invention on the concealed target detection dataset CAMO, the camouflaged target detection dataset COD10K, and the camouflaged object detection dataset NC4K, respectively. Evaluation metrics include mean absolute error (MAE), structural metric Sm, enhancement matching metric Em, and segmentation accuracy Fw (weighted F metric).

[0182] Experimental results show that the segmentation accuracy (Fw) of the proposed method on the concealed target detection dataset CAMO is 0.057, 0.839, 0.907, and 0.79, respectively. On the camouflaged target detection dataset COD10K, the accuracy (Fw) is 0.028, 0.841, 0.914, and 0.752, respectively. On the camouflaged object detection dataset NC4K, the accuracy (Fw) is 0.04, 0.858, 0.921, and 0.811, respectively. This demonstrates the superior segmentation accuracy of the proposed method, fully validating its effectiveness in weakly supervised target localization tasks.

[0183] Table 1. Performance comparison of the proposed method and other methods on the concealed target detection dataset CAMO.

[0184]

[0185] Table 2. Performance comparison of the proposed method and other methods on the camouflage target detection dataset COD10K.

[0186]

[0187] Table 3. Performance comparison of the proposed method and other methods on the camouflage object detection dataset NC4K.

[0188] .

Claims

1. A camouflage target detection method based on point supervision and pseudo label optimization, characterized in that: The method comprises the following steps: Step S1, inputting an image into a backbone network for preprocessing to obtain a corresponding initial prediction map; Specifically: The point annotation of the image is taken as a circle center, and r is taken as a radius to obtain a circular supervision signal, and the circular supervision signal is used to supervise the backbone network to obtain a corresponding initial prediction map; Step S2, obtaining multiple hints based on the image and the initial prediction map of step S1, inputting the multiple hints into a universal segmentation model to obtain a mask; Specifically: Step 21, obtaining a bounding box annotation based on a maximum circumscribed matrix in the initial prediction map of step S1; and combining the point annotation of the image and the obtained bounding box annotation to form multiple hints; Step 22, inputting the multiple hints into the universal segmentation model to obtain a point mask and a bounding box mask through segmentation; Step S3, performing weighted fusion on the initial prediction map obtained in step S1 and the mask obtained in step S2 by using a semantic entropy multi-mask weighted fusion mechanism to obtain a pseudo label; wherein the mask is divided into a point mask and a bounding box mask; and specifically: Step S31, calculating the initial prediction map respectively with the point mask and the bounding box mask to obtain the point semantic entropy and the bounding box semantic entropy, and the formula is as follows: ； wherein, representative point semantic entropy, representative point multiplication, representative logarithm operation, representative minimum positive number; ； wherein, represents the bounding box semantic entropy; Step S32, spatial domain decomposition of the point mask and the bounding box mask into overlapping regions = ∩ and non-overlapping regions = - and the bounding box mask = - ; Step S33, according to the calculated point semantic entropy and the bounding box semantic entropy , the semantic entropy-based weight distribution is performed on the non-overlapping area to obtain the point mask weight and the bounding box mask weight; the formula is as follows: ； ； wherein, represent a point mask weight, represent a bounding box mask weight; Step S34, the point prompt exclusive region of the overlapping region , the point prompt exclusive region of the non-overlapping region , and the bounding box prompt exclusive region , the pseudo label is obtained by weighted fusion according to the point mask weight and the bounding box mask weight ; the formula is as follows: ； wherein represents a dummy label; Step S4, constructing a global-local perception module to obtain global-local interaction enhanced features; Step S5, performing bidirectional cross-layer fusion on the global-local interaction enhanced features obtained in step S4 to obtain fused features, and processing the fused features to obtain a prediction map; Step S6, constructing a pseudo target detection model based on the pseudo label obtained in step S3 and the prediction map obtained in step S5.

2. The camouflage target detection method based on pseudo-label optimization under point supervision according to claim 1, characterized in that: The global-local perception module is constructed in step S4 to obtain global-local interaction enhanced features; Specifically: Step S41, inputting an image into a backbone network to extract initial features of the image, and constructing a global-local perception module; Step S42, based on the global-local perception module constructed in step S41; the global-local perception module comprises a global perception module and a local perception module, the global perception module obtains global perception features, the local perception module obtains local perception features, and the global-local interaction enhanced features are obtained based on the global perception features and the local perception features.

3. The camouflage target detection method based on pseudo-label optimization supervised by points according to claim 2, characterized in that: In step S42, the global perception module obtains global perception features; Specifically: Step S4201, the initial features of the image are input into two parallel hole convolutions, the convolution kernel sizes of the hole convolutions are 3*3 and 5*5 respectively, spatial features under different receptive fields are extracted, and the extracted spatial features are added to the initial features of the image through a residual structure to obtain residual features; Step S4202, the residual features are input into an improved frequency domain self-attention mechanism to obtain spatial domain residual features, and the spatial domain residual features are converted into frequency domain features through fast Fourier transform to obtain frequency domain query features, frequency domain matching features and frequency domain information features; In step S4203, the obtained frequency domain query feature, frequency domain matching feature and frequency domain information feature are subjected to dimension transformation to obtain dimension-transformed frequency domain query feature, dimension-transformed frequency domain matching feature and dimension-transformed frequency domain information feature. The dimension-transformed frequency domain query feature and the dimension-transformed frequency domain matching feature are subjected to matrix multiplication to generate a transposed attention graph A f ; Step S4204, performing transpose on the attention map A f Operation is performed; the formula is as follows: ； wherein representing the transposed attention map A f the real part of a complex number, conj represents the calculation of the conjugate value of a complex number; ； wherein representing the transposed attention map A f the imaginary part of a complex number, j represents the imaginary unit in complex number theory; Step S4205, the real part of the transposed attention map complex number and the imaginary part of the transposed attention map complex number are respectively activated using a flexible maximum value activation function, and then recombined to obtain an intermediate attention map, and then the intermediate attention map is multiplied by the frequency domain information features after dimension transformation to obtain preliminary frequency enhanced features; Step S4206, frequency residual connection is performed to obtain frequency enhanced features; the formula is as follows: ； wherein, representing frequency enhancement features, representing indicator functions, representing fast inverse Fourier transforms, rectified linear unit activation functions, representing fast Fourier transforms, representing residual features, representing matrix multiplications; Step S4207, the initial frequency enhancement feature and the frequency enhancement feature are spliced in the channel dimension, and the frequency domain self-attention enhancement feature is obtained by transforming the channel dimension through a 1x1 convolution; Step S4208, the initial feature of the image and the frequency domain self-attention enhancement feature are added through a residual connection, and the frequency domain enhancement feature is obtained by processing through a layer normalization; the frequency domain enhancement feature is input into a deep separable convolution group with parallel 3x3, 5x5 and 7x7 convolution kernels to obtain the direction enhancement feature; Step S4209, the frequency domain enhancement feature and the direction enhancement feature are fused through a residual structure to obtain the global perception feature.

4. The camouflage target detection method based on pseudo-label optimization under point supervision according to claim 3, characterized in that: In step S42, the local perception module obtains the local perception feature; Specifically: Step S4210, the initial features of the image are input into the deep convolution branches of three convolution kernels of 1x3, 3x1 and 3x3, and the deep convolution branches extract local features in three different directions, which are horizontal direction local features , vertical direction local features and local features in other directions except the horizontal direction and the vertical direction ; Step S4211: The extracted local features from three different directions are concatenated along the channel dimension and activated using the Gaussian error linear unit activation function, and then combined with the initial features of the image. The directional characteristics are obtained by adding them together; the formula is as follows: ； wherein, represents a direction feature, Conv 1x1 represents a common 1x1 convolution, represents a channel dimension concatenation; Step S4212, the direction feature is processed through a layer normalization and a feedforward neural network, and is added to the processed direction feature through a residual structure to obtain the local perception feature.

5. The camouflage target detection method based on pseudo-label optimization supervised by points according to claim 4, characterized in that: In step S42, the global-local interaction enhancement feature is obtained based on the global perception feature and the local perception feature; Specifically: Step S4213, the global perception feature obtained in step S4209 and the local perception feature obtained in step S4212 are respectively averaged and cut into four parts along the channel dimension, to obtain four groups of global perception features and four groups of local perception features, and the size of the four groups of global perception features and the four groups of local perception features is C / 4xHxW; wherein, C represents the number of channels, H represents the height, and W represents the width; Step S4214, four groups of global perception features are compressed in space dimension through global average pooling respectively, and four groups of global perception features are spliced to obtain new four groups of global perception features; four groups of local perception features are compressed in space through global average pooling respectively, and four groups of local perception features are spliced to obtain new four groups of local perception features; Step S4215, four groups of normalized fusion weights are calculated from the new four groups of global perception features and the new four groups of local perception features through a lightweight fully connected network; wherein, the lightweight fully connected network includes two fully connected layers and a rectified linear unit activation function; Step S4216, four groups of global-local features are obtained by weighting and fusing four groups of global perception features and four groups of local perception features using four groups of normalized fusion weights, and four groups of global-local features are spliced in the channel dimension; The global-local enhancement feature is obtained by fusing the global perception feature and the local perception feature through a residual connection; the formula is as follows: ； wherein i represents a level, m represents a group number, represents the i-th level global-local enhanced feature, represents the i-th level m-th group fusion weight, represents the i-th level m-th group global-aware feature, represents the i-th level m-th group local-aware feature, represents the i-th level global-aware feature, represents the i-th level local-aware feature; Step S4217, the global-local enhancement feature is input into a gated convolution mechanism for processing, and is fused with the global-local enhancement feature through a residual connection to obtain the global-local interaction enhancement feature, wherein, the global-local interaction enhancement feature is divided into four layers, which are the initial layer global-local interaction enhancement feature, the second adjacent layer global-local interaction enhancement feature, the third adjacent layer global-local interaction enhancement feature and the deep layer global-local interaction enhancement feature; the formula is as follows: ； wherein, represents global-local interaction enhanced features represents batch normalization, represents 3x3 gated convolution.

6. The camouflage target detection method based on pseudo-label optimization under point supervision according to claim 5, characterized in that: In step S5, the global-local interaction enhancement feature obtained in step S4 is fused in a bidirectional cross-layer manner to obtain a fusion feature, and the obtained fusion feature is processed to obtain a prediction map; specifically: Step S51, a 3x3 depth separable convolution process is applied to the deep global-local interaction enhanced feature to obtain separable features of the deep global-local interaction enhanced feature; The separable features of the deep global-local interaction enhanced feature are generated into a first feature weight map through a group of point convolution groups and an S-shaped activation function, and the first feature weight map is multiplied with the third adjacent layer global-local interaction enhanced feature to obtain the first layer semantic enhanced feature; The third adjacent layer global-local interaction enhanced feature is globally pooled, and then a second feature weight map is generated through a group of 1x1 convolution and S-shaped activation function, and the second feature weight map is multiplied with the separable features of the deep global-local interaction enhanced feature to obtain the first layer local spatial refinement feature; The first layer semantic enhanced feature and the first layer local spatial refinement feature are aggregated to obtain the first layer fusion feature; A 3x3 depth separable convolution process is applied to the first layer fusion feature to obtain separable features of the first layer fusion feature; The separable features of the first layer fusion feature are generated into a third feature weight map through a group of point convolution groups and an S-shaped activation function, and the third feature weight map is multiplied with the second adjacent layer global-local interaction enhanced feature to obtain the second layer semantic enhanced feature; The second adjacent layer global-local interaction enhanced feature is globally pooled, and then a fourth feature weight map is generated through a group of 1x1 convolution and S-shaped activation function, and the fourth feature weight map is multiplied with the separable features of the first layer fusion feature to obtain the second layer local spatial refinement feature; The second layer semantic enhanced feature and the second layer local spatial refinement feature are aggregated to obtain the second layer fusion feature; A 3x3 depth separable convolution process is applied to the second layer fusion feature to obtain separable features of the second layer fusion feature; The separable features of the second layer fusion feature are generated into a fifth feature weight map through a group of point convolution groups and an S-shaped activation function, and the fifth feature weight map is multiplied with the initial layer global-local interaction enhanced feature to obtain the third layer semantic enhanced feature; The initial layer global-local interaction enhanced feature is globally pooled, and then a sixth feature weight map is generated through a group of 1x1 convolution and S-shaped activation function, and the sixth feature weight map is multiplied with the separable features of the second layer fusion feature to obtain the third layer local spatial refinement feature; The third layer semantic enhanced feature and the third layer local spatial refinement feature are aggregated to obtain the last layer fusion feature; Step S52, a 1x1 convolution, a batch normalization and an S-shaped activation function are used to process the last layer fusion feature to obtain a prediction map.

7. The camouflage target detection method based on pseudo-label optimization supervised by points according to claim 6, characterized in that: In step S6, a pseudo target detection model is constructed based on the pseudo label obtained in step S3 and the prediction map obtained in step S5, a binary cross entropy loss function and a contrast loss function are used as the final loss function to train the constructed pseudo target detection model; specifically: Step S61, the partial binary cross entropy loss and the contrast loss of the prediction map are calculated, and the formula is as follows: ； wherein, represents binary cross-entropy loss, represents pseudo labels true class of a pixel position in an image, represents predicted class of a pixel position in an image by a prediction map; ； wherein, represents a contrast loss, d represents a pixel point of an image, represents a prediction map of an image, represents a prediction map obtained by processing an image through Gaussian blur, color jitter, and random rotation. Step S62, the pseudo target detection model is trained based on the final loss function.

Images