A camouflage object detection method based on edge guidance network

Abstract

The application discloses a camouflage object detection method based on an edge guidance network and belongs to the technical field of image processing. Most existing deep learning methods often cannot accurately identify the complete and fine structure of a camouflage object. The application creatively proposes a method based on an edge guidance network, wherein the edge guidance network comprises a backbone module, an edge perception module, an edge guidance feature module and a context aggregation module, the representation learning ability of object-related edge semantic information is enhanced, and a complete and fine mask graph and contour graph of the camouflage object are output. Experiments on three data sets show that the four evaluation indexes of the application are superior to all the most advanced camouflage object detection methods.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method for detecting camouflaged objects based on edge-guided networks. Background Technology

[0002] A camouflage scenario occurs when a foreground subject (relatively prominent) attempts to "hide" in the background to deceive the observer's vision. Identifying and segmenting camouflaged objects (foreground subjects) within a camouflage scenario is known as camouflage object detection. Due to the prevalence of camouflage scenarios, camouflage object detection has a wealth of downstream tasks and applications, such as medical image segmentation, industrial defect detection, locust infestation detection, and creative image synthesis, attracting widespread attention and research interest from the computer vision community.

[0003] Early methods for detecting camouflaged objects relied on traditional visual features such as color, texture, momentum, and gradient. Extracting and using only one visual feature rarely yielded satisfactory results, so some methods attempted to enhance performance by combining these features. However, due to the limitations of manual feature extraction, these methods often failed in more complex scenes or real-world applications.

[0004] With the rise of deep learning technology, convolutional neural network-based methods have automatically learned deep features from a large number of training images that are more comprehensive, general, and effective than traditional features. These detection methods can be roughly divided into three categories: one is biomimetic methods, which mimic the behavior of predators in nature or human visual psychological patterns to design networks; another is designing targeted network modules / architectures to effectively explore features that can be used to identify camouflaged objects; and the third is incorporating some auxiliary tasks into joint learning / multi-task learning frameworks, such as classification tasks, edge extraction, salient object detection, and camouflaged object ranking. These methods can extract valuable additional clues from shared features, thereby significantly enhancing the feature representation of camouflaged object detection.

[0005] In joint learning / multi-task learning frameworks, existing detection methods that employ edge extraction as an auxiliary task are interactive graph learning models based on graph neural networks. This inevitably increases the model's complexity and consumes more computational resources. Furthermore, although it introduces boundary cues, it still loses some boundary-related details and introduces noticeable background noise, thus weakening the performance of detecting camouflaged objects. Summary of the Invention

[0006] The purpose of this invention is to address the problem in existing methods for detecting camouflaged objects that, when the camouflaged object is highly integrated with the environment, especially when the edges are interrupted, only rough or incomplete object boundaries are provided. Therefore, this invention designs a camouflaged object detection method based on an edge-guided network.

[0007] The Boundary-Guided Network (BGNet) consists of a backbone network (BN), an edge-aware module (EAM), an edge-guidance feature module (EFM), and a context aggregation module (CAM).

[0008] The method of the present invention includes the following steps.

[0009] Step 1: Feature extraction of the image to be detected. A backbone network (BN) is used to extract multi-level features from the image to be detected, including low-level features containing local edge details and high-level features containing global localization information.

[0010] Step 2: Mining edge information of camouflaged objects. Using the Edge Awareness (EAM) module, multi-level edge semantic information related to camouflaged objects is mined from the extracted low-level and high-level features.

[0011] Step 3: Edge Information-Guided Feature Extraction. Based on the edge semantic information of the camouflaged objects mined in Step 2, the Edge-Guided Feature Module (EFM) is used to extract features from the backbone network, enhancing the model's ability to represent and learn boundary features.

[0012] Step 4: Contextual information aggregation and output. Using the CAM context aggregation module, multi-level features and edge information are aggregated to generate a mask and contour map of the camouflaged object.

[0013] Advantages and benefits of this invention: By introducing edge cues and mining semantic information about object edges, this invention enhances the model's representation capabilities and outputs more accurate predictions of camouflaged objects. Experiments show that this invention outperforms current camouflaged target detection methods on all four evaluation metrics across three publicly available datasets. Attached Figure Description

[0014] Figure 1 This is a flowchart of the BGNet camouflage object detection method of the present invention.

[0015] Figure 2 This is a schematic diagram of the specific implementation framework of the EAM edge perception module in this invention.

[0016] Figure 3 This is a schematic diagram of the specific implementation framework of the EFM edge guidance feature module in this invention.

[0017] Figure 4 This is a schematic diagram of the specific implementation framework of the CAM context aggregation module in this invention.

[0018] Figure 5 The figure shows the experimental results of this invention and 18 comparative methods on 3 datasets and 4 indicators.

[0019] Figure 6 Examples of comparison results between this invention and the best current deep learning methods are shown in the figures (a) Input image, b) Ground truth image, c) Ours method, and dk) JCSOD, C2FNet, LSR, UGTR, R-MGL, S-MGL, PFNet, and SINet (8 comparison methods). Detailed Implementation

[0020] To further illustrate the technical solution of the present invention, it will now be described in detail with reference to the accompanying drawings.

[0021] refer to Figure 1 The process shown is implemented in the following steps.

[0022] a. Use the Res2Net_50 backbone network to extract features from the input image, obtaining a set of multi-level features. .

[0023] b. Use an Edge-Aware Module (EAM) to extract features from low-level features containing local edge details. High-level features containing global positioning information Mining edge semantic information related to objects .

[0024] c. Next, four Edge-guidance Feature Modules (EFMs) are used to extract the discovered edge cues. It is integrated into the feature extraction of the backbone network to enhance its ability to represent and learn boundary features.

[0025] d. Finally, three Context Aggregation Modules (CAMs) are used to aggregate the fused multi-level features in a top-down manner and output the outline map of the camouflaged object.

[0026] The above EAM module reference Figure 2 As shown, the specific implementation steps are as follows.

[0027] a. Use two 1x1 convolutional layers to process the input features. , After transformation, they become 64-dimensional units. The dimension is 256. .

[0028] b. and upsampling The edges are then concatenated, followed by two 3x3 convolutional layers, one 1x1 convolutional layer, and a sigmoid activation function to obtain the mined edge information. .

[0029] The above EFM module reference Figure 3 As shown, the specific implementation steps are as follows.

[0030] a. Input features and downsampling (to match input features) Edge information (size) And perform element-wise multiplication on it.

[0031] b. Simultaneously, input features There is also a skip connection, which is added to the output of step a.

[0032] c. The summed result is then fed into a 3x3 convolutional layer to obtain the initial fused features. .

[0033] d. Introduce local attention to mine key feature channels. Preferably, use the algorithm proposed in the CVPR 2020 paper "ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks". Specifically, use a GAP global average pooling operation to... After processing, a one-dimensional convolution with kernel k and a sigmoid function are used to obtain the attention, i.e., the weights, for the corresponding channel dimensions.

[0034] e. In the above one-dimensional convolution, the kernel size k is set to adaptive, and the calculation formula is: , where represents the nearest odd number, and C is . The number of channels.

[0035] f. Combine attention values ​​with fusion features Element-wise multiplication yields the final output of this module. .

[0036] The above CAM module reference Figure 4 As shown, the specific implementation steps are as follows.

[0037] a. When i=2,3,4, perform the following operation: EFM output The output of the adjacent high-level CAM The values ​​are input into the CAM together. When i=4, since there is no corresponding high-level CAM output, the corresponding EFM output is used. replace.

[0038] b. Take the smaller one. First, perform upsampling preprocessing to make it consistent with... The sizes are matched, and the two are concatenated and fed together into a 1x1 convolutional layer to obtain the initial aggregated features. .

[0039] c. Next, The feature map is divided into four parts along the channel dimension, respectively. .

[0040] d. Perform cross-scale interactive learning on these feature maps, specifically by using the form... The method extracts multi-scale contextual features of neighboring branches, where For a 3x3 dilated convolution, the dilation rate is... =j, meaning that the values ​​are equal to the branch index j. Where, when At that time, only .when At that time, only .

[0041] e. That is, four Concatenate the data, pass it through a 1x1 convolution, and then connect it with a skip connection. Add them together, then perform a 3x3 convolution to obtain the output of this module.

[0042] f. In the future After another 1x1 convolution, the channel dimension is reduced to 1, thus obtaining the prediction result of the disguised object. .

[0043] During the training phase, two types of supervisory information are used to camouflage object masks. Edges of camouflaged objects .

[0044] Among them, supervised learning using the mask information of camouflaged objects, referred to as mask supervision, adopts a weighted binary cross-entropy loss function. and weighted IOU loss Supervised learning using the edges of camouflaged objects is called edge supervision, and it employs the Dice loss function. .

[0045] Mask supervision is applied to the outputs of the three CAM components. .

[0046] The total loss function is defined by the following formula:

[0047] .

[0048] in, This is a trade-off coefficient, and preferably, in this invention, it is set to 3. It predicts the edges of camouflaged objects.

[0049] The training process of the model is as follows: Res2Net-50 pre-trained on ImageNet is selected as the backbone network.

[0050] All images to be detected are transformed to the same 416x416 size and input into the method of this invention, and random horizontal flipping is used as a means of data augmentation.

[0051] During training, the batch size was 16, the optimizer was Adam, and the learning rate was adjusted using the Poly policy. The initial learning rate was 0.0001, the power was 0.9, and the training lasted for 25 epochs.

[0052] The beneficial effects of this invention are further illustrated by the following simulation experiments: Experimental datasets and simulation conditions: The CAMO dataset contains 1250 camouflage images with 8 object classifications, of which 1000 images are used for training and 250 images are used for testing; the COD10K dataset contains 5066 camouflage images with 5 superclasses and 69 subclasses, of which 3040 images are used for training and 2026 images are used for testing; the NC4K dataset contains 4121 images, which were downloaded from the Internet.

[0053] Developed based on the PyTorch deep learning framework and running on the NVIDIA Tesla P40 GPU platform.

[0054] Evaluation criteria for camouflaged object detection performance: using Mean Absolute Error (MAE) and Weighted F-measure. Structural metrics Adaptive E-metric .

[0055] Figure 5 This paper presents experimental results comparing the present invention with 18 existing methods on the aforementioned three datasets and four metrics. Compared to the second comparative method, JCSOD, the present invention's method... The indicator increased by 1.72%. It increased by 1.48%. It increased by 3.5%.

[0056] Figure 6 Examples of comparison results between this invention and current deep learning methods are shown in the images (a) Input image, (b) Ground truth image, (c) Our method, and (dk) JCSOD, C2FNet, LSR, UGTR, R-MGL, S-MGL, PFNet, and SINet, a total of 8 comparison methods). It can be seen that this invention can provide more accurate predictions of camouflaged objects, and preserves more refined and complete object structure and edge details.

Claims

1. A method for detecting camouflaged objects based on the BGNet model, characterized in that... The BGNet model comprises a backbone network, an edge-aware module, an edge-guided feature module, and a context aggregation module. This method can mine multi-level semantic edge information of camouflaged objects and use this edge information to guide the model's representation learning process, ultimately generating a mask map and contour map of the camouflaged object. The method includes the following steps: Step 1: Use a backbone network to extract features from the image to be detected. This network contains five convolutional blocks connected in sequence to extract multi-level information from the input image. Step 2: Using the edge perception module, extract object-related edge information from the extracted features. Fuse and process the feature maps generated by the second and fifth convolutional blocks in Step 1 to generate a disguised object boundary map: The feature map output by the second convolutional block contains low-level local edge detail features; the feature map output by the fifth convolutional block contains high-level global edge localization information. Step 3: Utilize the edge-guided feature module to guide the feature extraction of the backbone network with edge information. Specifically, the features extracted by the backbone network are fused with the boundary map generated by the edge perception module. Key feature channels are mined based on local attention, and the optimized feature map under edge guidance is output. Step 4: Using the context aggregation module, multi-level features and edge information are aggregated to generate a mask and contour map of the disguised object. Specifically, the outputs of adjacent edge-guided feature modules are integrated to extract multi-scale context features of neighboring branches, and finally, a mask and contour map of the disguised object are obtained.

Images