A method and system for detecting salient and camouflaged objects based on contrastive distillation learning

By comparing the Distillation Learning Model (CDP) with the detection of salient and camouflaged objects, this approach addresses the limitations of existing methods in real-world applications, achieving unified detection of salientity and camouflage, improving detection performance and consistency, and reducing manual annotation costs.

CN118447374BActive Publication Date: 2026-11-17CHANGZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410349359.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-26
Publication Date
2026-11-17
Estimated Expiration
2044-03-26

AI Technical Summary

Technical Problem

Existing methods for salient object detection (SOD) and camouflaged object detection (COD) are not yet mature in joint learning, especially in real-world applications where they have limitations. Furthermore, existing contrastive learning models lack sufficient experimental validation and cannot effectively unify the detection of salient and camouflaged objects.

Method used

We employ a contrastive distillation learning model (CDP) to distinguish between foreground and background. We design a framework that includes foreground semantic acquisition, background semantic acquisition, and foreground-background contrastive learning. We utilize an improved ResNet50 network to extract multi-scale features and train the model using a joint contrastive loss and cross-entropy loss function to achieve unified detection of salience and camouflage.

Benefits of technology

It achieves unified detection of salient and camouflaged objects under a task-agnostic framework, improving the completeness and consistency of detection, adapting to real-world scenarios, reducing the labor demand for large-scale manual annotation, and achieving state-of-the-art performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118447374B_ABST
    Figure CN118447374B_ABST
Patent Text Reader

Abstract

This invention relates to the field of image processing technology, and more particularly to a method and system for detecting salient and camouflaged objects based on contrastive distillation learning. The method includes extracting deep features from an input image; designing a contrastive distillation learning model comprising foreground semantic acquisition, background semantic acquisition, and foreground-background contrastive learning; mining features implicit in the deep backbone; and, under ground truth supervision, inferring the foreground image from multi-scale features to obtain a foreground semantic vector; extracting features from intermediate layers of the network and combining them with the inverted ground truth image to obtain a background semantic vector; and training the contrastive distillation learning model using a loss function. This invention utilizes a contrastive distillation learning model to distinguish between foreground and background, and is suitable for both salient and camouflaged scenes, achieving task independence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for detecting salient and camouflaged objects based on contrastive distillation learning. Background Technology

[0002] Satious object detection (SOD) is used to identify the most conspicuous regions or objects in an image; while camouflaged object detection (COD) is used to find objects that are usually hidden in the environment. For example, salient perception and camouflaged perception are needed in autonomous driving; the same applies to remote sensing.

[0003] Early SOD and COD methods relied on handcrafted features to detect targets, but their performance could not be improved due to the limitations of handcrafted features. With the development of deep learning, the performance of SOD and COD has been greatly improved thanks to its powerful feature representation capabilities. For example, DUTS and COD10 K, research on deep supervised SOD and COD has been well developed, but joint learning of SOD and COD is still in its early stages.

[0004] Contrastive learning learns general and robust feature representations by comparing similar and dissimilar pairs, demonstrating its power in many computer vision tasks. Examples include Tianhong Li et al.'s Targeted supervised contrastive learning for long-tailed recognition and Sha Meng et al.'s Tracking without label: Unsupervised multiple object tracking via contrastive similarity learning. However, these models have not been validated on sufficiently broad datasets or considered real-world applications. Both have designed targeted supervised contrastive learning frameworks to enhance the uniformity of feature distribution in image recognition under different data conditions and to achieve multi-object tracking by combining self-contrast, cross-contrast, and blurred contrast learning. However, their models all have limitations in experimental validation, lacking validation on sufficiently broad datasets, having limited application in object detection, and failing to consider real-world scenarios.

[0005] Liu Yi et al.'s method for detecting salient and camouflaged objects based on contextual information performs unsupervised learning by combining contextual information. By combining interval context and global contextual relationship information, the foreground map and pseudo-mask inferred by the interval layer and global context network are supervised by a joint error function. Although this model retains the semantic information of the context and guides and generates the prediction map, it also retains noise at different scales, resulting in some loss of detail and granularity, such as blurred edges and segmentation into background objects. Summary of the Invention

[0006] To address the shortcomings of existing methods, this invention utilizes a contrastive distillation paradigm (CDP) to distinguish between foreground and background. It is suitable for both salient and camouflaged scenes, achieving task independence. That is, it can detect target objects without determining whether a given scene requires detection of salient or camouflaged targets. This unifies the methods for detecting salient and camouflaged objects and is more suitable for real-world scenarios.

[0007] The technical solution adopted in this invention is: a method for detecting salient and camouflaged objects based on contrastive distillation learning, comprising the following steps:

[0008] Step 1: Extract depth features from the input image;

[0009] In a preferred embodiment of the present invention, a ResNet50 network with the fully connected layers removed is used to extract depth features from the input image.

[0010] Step 2: Design a contrastive distillation learning model that includes foreground semantic acquisition, background semantic acquisition, and foreground-background contrastive learning; mine features hidden in the deep backbone, and under ground truth supervision, infer the foreground map by jointly using multi-scale features to obtain the foreground semantic vector; extract features from the intermediate layers of the network and the inverted ground truth. Figure 1 The background semantic vector is obtained; a contrastive distillation learning model is trained using a loss function.

[0011] In a preferred embodiment of the present invention, foreground semantic acquisition includes:

[0012] Step 21: Perform a convolution operation on the depth features to obtain the convolutional features;

[0013] Step 22: By fusing features from different layers, two fused multi-scale features are obtained respectively;

[0014] Step 23: Calculate foreground semantic features using the fused multi-scale features and deep features;

[0015] In a preferred embodiment of the present invention, the formula for the foreground semantic features is:

[0016]

[0017] Here, concat(·) represents a cascading operation along the channel dimension, bi(·) represents a bilinear interpolation upsampling operation, down(·) represents a downsampling operation, and sigmoid(·) is an activation function operation. To fuse multi-scale features F0 and F2; This is a multi-scale feature fusion feature set from F1 and F3.

[0018] Step 24: Calculate the foreground semantic vector using foreground semantic features.

[0019] In a preferred embodiment of the present invention, background semantic acquisition includes: under truth-monitoring, obtaining the background semantic vector v b It is obtained by downsampling the background mask (1-GT) and multiplying it element-wise with the core feature E2 of the encoder intermediate layer, and then passing it through a fully connected layer.

[0020] In a preferred embodiment of the present invention, the loss function is a joint contrastive loss and a cross-entropy loss function, and the joint error is calculated.

[0021] In a preferred embodiment of the present invention, the weighted average F-measure value and MAE value are used to evaluate the distillation learning model.

[0022] As a preferred embodiment of the present invention, a system for detecting salient and camouflaged objects based on contrastive distillation learning includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the method for detecting salient and camouflaged objects based on contrastive distillation learning.

[0023] As a preferred embodiment of the present invention, a computer-readable medium storing computer program code implements a method for detecting salient and camouflaged objects based on contrastive distillation learning when executed by a processor.

[0024] The beneficial effects of this invention are:

[0025] 1. This invention is the first to conduct a unified study of SOD and COD in a task-agnostic framework, and achieves significant results in binary segmentation tasks by comparing distillation models.

[0026] 2. The CDP model unifies SOD and COD, solving the problem of the huge labor cost of large-scale manual annotation;

[0027] 3. Experiments on publicly available SOD and COD benchmarks demonstrate that the task-agnostic framework of this invention achieves state-of-the-art performance in supervised settings compared to existing methods. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method for detecting salient and camouflaged objects based on contrastive distillation learning according to the present invention;

[0029] Figure 2 This is a diagram of the contrastive distillation-based learning network framework of the present invention;

[0030] Figure 3 This is a qualitative visual comparison chart of the present invention with other existing methods. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments. The drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0032] like Figure 1 As shown, methods for detecting salient and camouflaged objects based on contrastive distillation learning include:

[0033] Calculating the contrast between target inference and background, contrastive learning is an important paradigm in supervised learning networks; in order to identify targets from the surrounding environment, this invention adopts a contrastive distillation learning model (CDP), which can help the network to more effectively distinguish between foreground and background, and is suitable for both salient and camouflaged scenes; the CDP model is implemented by three parts: foreground semantic vector, background semantic vector, and foreground-background contrastive learning.

[0034] To design a simple and effective network for decoding image semantics for further reasoning, the overall network architecture is as follows: Figure 2 As shown, features hidden in the deep backbone were mined. Under ground truth supervision, foreground maps were inferred from multi-scale features, thereby obtaining foreground semantic vectors. Furthermore, features extracted from the intermediate layers of the network and their inverted ground truth values ​​were analyzed. Figure 1 The background semantic vector is obtained, and the network model is trained by combining the contrastive loss and cross-entropy loss functions inside the contrastive distillation learning module (CDP).

[0035] Step 1: Extract depth features from the input image;

[0036] A pre-trained improved ResNet50 network is used to extract multi-level deep features E from the input image I. i , i∈[0,1,2,3,4]; where, the improved ResNet50 network is a network with the last fully connected layer of the ResNet50 network removed.

[0037] The input image is cropped to 320×320, and features from five network layers at five scales (Conv1_3, Conv2_3, Conv3_3, Conv4_3, and Conv5_3) are extracted. This means that the input image is assumed to be... The image is used to extract multi-level deep features through an improved ResNet-50, denoted as... i∈[0,1,2,3,4],C i ∈[64,256,512,1024,2048].

[0038] Step 2: Design a contrastive distillation learning model, including foreground semantic acquisition, background semantic acquisition, and foreground-background contrastive learning; among which,

[0039] Foreground semantic acquisition includes:

[0040] Step 21: Transfer multi-level deep features E i Performing a 1×1 convolution operation yields the feature F. i ;Right now

[0041] F i =conv(E i ), i = 0, 1, 2, 3, 4, (1)

[0042] Here, conv(·) represents a convolutional network.

[0043] Step 22: Fuse features F0 and F2 from different layers to obtain the fused multi-scale features. By combining the interval layer features F1 and F3, the fused multi-scale features are obtained. The formula is:

[0044]

[0045] Here, concat(·) represents a cascading operation along the channel dimension, and bi(·) represents a bilinear interpolation sampling operation.

[0046] Step 23: Joint integration and fusion of multi-scale features And the deep feature F4, to calculate the final foreground semantic feature T; the formula is:

[0047]

[0048] Here, concat(·) represents a cascade operation along the channel dimension, bi(·) represents an upsampling operation using bilinear interpolation, down(·) represents a downsampling operation, and Sigmoid(·) is an activation function operation.

[0049] Step 24, Foreground semantic vector v fThe formula is obtained through learning the foreground semantic features T and the fully connected layer:

[0050]

[0051] Background semantic acquisition includes: under truth-value supervision, the background semantic vector v b The background mask (1-GT) is downsampled and then element-wise multiplied with the encoder's intermediate backbone features E2, followed by a fully connected layer. The intermediate backbone features are the ResNet50 third-layer features E2 of the input image, and the formula is:

[0052]

[0053] Foreground-background contrastive learning includes: given a batch of n samples, denoted as I {1:n} , to generate n foreground semantic vectors With background semantic vector Feed contrast loss function L NEG Training will be conducted in the following manner.

[0054] The contrastive loss function is used to train the contrastive distillation learning model;

[0055] To train the contrastive distillation learning model, foreground and background semantics are fed into the contrastive learning loss L. NEG The loss function L learned by comparison with distillation NEG and cross-entropy loss function L B To jointly train a unified salient object and camouflaged object detection network; the joint error function is:

[0056]

[0057] L NEG The comparison loss function formula is as follows:

[0058]

[0059] L B The cross-entropy loss function is:

[0060] L B (T,GT)=-Σ[GT m log(T m )+(1-GT m log(1-T) m (8)

[0061] Where cos(·) represents cosine similarity, i and j represent vector indices, and m is the pixel index.

[0062] Simulation experiment:

[0063] All experiments were implemented on the PyTorch framework. To avoid overfitting caused by training from scratch, this invention initializes the improved ResNet50 and uses the SGD optimizer to train the model with an initial learning rate of 10. -3 The training datasets include: the SOD training dataset, the DUTS-TR training subset, the MSRA-B training subset, and the COD training dataset (CAMO and COD 10K), with data augmentation strategies such as horizontal flipping employed.

[0064] This invention, along with the FOUND method and IGC (a context-informed method for detecting salient and camouflaged objects), was subjected to unified experiments on the public image SOD and COD datasets for salient and camouflaged object detection. Some results were compared intuitively. Figure 3 As shown, the original image represents the image used as experimental input in the database, and the ground truth image represents the manually calibrated binary image; the prediction results of this invention and the FOUND and IGC methods are given below.

[0065] from Figure 3 As can be seen, compared with the FOUND method and IGC method, the present invention has better integrity in detecting both salient and camouflaged targets, more obvious background suppression effect, and better performance in detecting salient and camouflaged targets in real-world scenarios.

[0066] The method of this invention, along with the ITSD and FOUND models, was used in a unified experiment on the publicly available SOD and COD datasets for salient and camouflaged target detection. The evaluation metrics were weighted average F-measure and MAE values. The results are shown in Table 1.

[0067] Table 1 compares the method of the present invention with existing methods.

[0068]

[0069] As can be seen from Table 1, compared with ITSD, FOUND and IGC, the present invention has a higher weighted average F-measure value and a lower MAE value, which shows that the present invention has better integrity and consistency in the detection of both salient targets and camouflaged targets, fully demonstrating the effectiveness and superiority of the method of the present invention.

[0070] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for detecting salient and camouflaged objects based on contrastive distillation learning, characterized in that, Includes the following steps: Step 1: Extract depth features from the input image; Step 2: Design a contrastive distillation learning model that includes foreground semantic acquisition, background semantic acquisition, and foreground-background contrastive learning; mine the features hidden in the deep backbone, and under ground truth supervision, use multi-scale features to infer the foreground map and obtain the foreground semantic vector; extract features from the intermediate layers of the network and obtain the background semantic vector together with the inverted ground truth map; train the contrastive distillation learning model using a loss function. Foreground semantic acquisition includes: Step 21: Perform a convolution operation on the depth features to obtain the convolutional features; Step 22: By fusing features from different layers, two fused multi-scale features are obtained respectively; Step 23: Calculate foreground semantic features using the fused multi-scale features and deep features; The formula for foreground semantic features is: ; in, This indicates a cascading operation along the channel dimension. This indicates the use of bilinear interpolation upsampling operation. This indicates a downsampling operation. For activation function operations, For fusion features , Multiscale features; For fusion features , Multiscale features; Step 24: Calculate the foreground semantic vector using foreground semantic features; Background semantic acquisition includes: under truth-value supervision, the background semantic vector. From background mask After downsampling, the main features of the encoder intermediate layer are compared with those of the encoder. The result is obtained by performing element-wise multiplication and then passing it through a fully connected layer.

2. The method for detecting salient and camouflaged objects based on contrastive distillation learning according to claim 1, characterized in that, The loss function is a combination of contrastive loss and cross-entropy loss, which calculates the joint error.

3. The method for detecting salient and camouflaged objects based on contrastive distillation learning according to claim 1, characterized in that, Deep features are extracted from the input image using a ResNet50 network with the fully connected layers removed.

4. The method for detecting salient and camouflaged objects based on contrastive distillation learning according to claim 1, characterized in that, The distillation learning model was evaluated by comparing the weighted average F-measure value and the MAE value.

5. A system for detecting salient and camouflaged objects based on contrastive distillation learning, characterized in that, include: Memory is used to store instructions that can be executed by the processor; A processor for executing instructions to implement the method for detecting salient and camouflaged objects based on contrastive distillation learning as described in any one of claims 1-4.

6. A computer-readable medium storing computer program code, characterized in that, The computer program code, when executed by a processor, implements the method for detecting salient and camouflaged objects based on contrastive distillation learning as described in any one of claims 1-4.