Progressive enhancement network for camouflage object detection

Abstract

The application provides a kind of progressive enhancement network for camouflage object detection, including object positioning module, group attention module and context feature recovery module are respectively used for accurately locating camouflaged object, refine texture and restore boundary in a progressive enhancement manner;Res2Net-50 is used as the main stem to extract its multi-level features from the input image, then use several object positioning modules to locate the potential camouflage object, then input the image of the initial position to the group attention module to refine the details, finally use several context feature recovery modules to restore the complete object to obtain clear boundaries.A new bionic framework is proposed, which greatly improves the performance of camouflage target detection in a progressive enhancement manner to locate objects, enhance texture features and restore boundaries.It can effectively extract global information and local features, so as to accurately determine the position of the camouflage object;The framework significantly improves the existing camouflage object segmentation performance.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of camouflage object detection, and particularly relates to a progressive enhancement network for camouflage object detection. BACKGROUND

[0002] Camouflage is a ubiquitous biological phenomenon in nature, which can help some organisms hide in the surrounding environment to protect themselves from predators. In practice, camouflage objects usually hide themselves by imitating the appearance, color or pattern of the environment and destructive coloring, so that they are difficult to be found. Based on this strategy, humans began to study bionic camouflage objects according to their own ideas to achieve the purpose of camouflage. For example, in military operations, soldiers achieve invisibility by wearing camouflage clothes made of special materials, hunters use camouflage sounds to lure animals in the forest, and farmers build scarecrows in the field to drive away birds. Recently, camouflage object detection (COD) has attracted increasing research interest in the computer vision community and has developed many potential applications in different fields, such as medical diagnosis (e.g., polyp segmentation, lung infection segmentation), agriculture (e.g., locust detection to prevent invasion), industry (surface defect detection), security and surveillance (e.g., search and rescue work), animal protection (e.g., species discovery), and art (e.g., leisure art). However, COD is a highly challenging task because the camouflage objects make themselves “perfectly” blend into the surrounding environment by materials, colors or lighting. It is very difficult to find due to low contrast, diverse manifestations, similar object size to the surrounding environment.

[0003] To solve this problem, many state-of-the-art methods have been proposed, and existing COD algorithms can be roughly divided into traditional methods and deep learning-based methods. Early traditional camouflage object detectors try to rely on artificial visual features (such as color, 3D convexity and appearance texture) to extract discriminative hand-crafted low-level features. In recent years, benefiting from the rapid development of deep learning and the fine open labeled COD dataset, a large number of deep neural network-based camouflage object detection algorithms have been proposed, which can extract low-level texture features and high-level semantic information at the same time, thus making great progress in all COD benchmarks. Although good performance achievements have been made, there is still much room for improvement. First, the idea of most existing methods is to use RFBs or ASPP modules in a single-stage manner to extract context, and then use some simple fusion methods to decode the segmentation results. This method does not take into account the positioning of the camouflage object, so it often appears inaccurate when segmenting the object. Second, although there are some two-stage methods, they usually only consider predicting the camouflage object from coarse to fine without considering object positioning, texture enhancement and boundary recovery. SUMMARY

[0004] The technical problem solved by the present application is to provide a progressive enhancement network for camouflage object detection to solve at least one of the above problems existing in the prior art.

[0005] To solve the above problems, one or more embodiments of the present application provide a progressive enhancement network for camouflage object detection, which comprises an object positioning module, a group attention module and a context feature recovery module, and the object positioning module, the group attention module and the context feature recovery module are respectively used for accurately positioning the camouflaged object, refining the texture and restoring the boundary in a progressive enhancement manner; for an input RGB image I with a size of HxW, Res2Net-50 is used as the backbone to extract f2, f3, f4 and f5 layer features from the input image, then a plurality of object positioning modules are used to locate the potential camouflage object, then the image at the initial position is input to the group attention module to refine the details, and finally a plurality of context feature recovery modules are used to restore the complete object to obtain clear boundaries.

[0006] Based on the above technical solutions of the present application, the following improvements can be made:

[0007] Optionally, the object positioning module comprises a global module and a local module, the global module is implemented in a non-local manner to capture long-range dependencies, thereby enhancing the context semantic representation from a global perspective; the local module is used to extract local information by using a plurality of convolution layers; the global module and the local module explore the potential object region in a complementary manner.

[0008] Optionally, in the global module, first, a receptive field module structure is used to expand the receptive field, then four 1x1 convolution layers are used to obtain four feature mappings B, C, D and E, wherein {B, C, D, E}∈R C×H×W , and then it is reshaped into R C×N , the transpose of B is multiplied by the C matrix, and a softmax layer is executed to calculate the global spatial attention map; a matrix multiplication operation is used to obtain the semantic enhanced global feature f g ∈R C×H×W , wherein R C×H×W represents a real number set with C channels, H height and W width, and R C×N represents a real number set with N columns and C channels; the process is described as follows:

[0009]

[0010]

[0011] wherein gsa ij represents the influence of the j position on the i position; η is initialized to 0, and more weights are gradually learned in the subsequent process, f iis the i-th layer feature.

[0012] Optionally, in the local module, a set of 1x1 convolutional layers, a BatchNorm2d function and a ReLU function are used to obtain local features, and then element-wise addition operation is performed on the global features and the local features to obtain aggregated features; the process is described as follows:

[0013] f l = ReLU(1x1Conv(ReLU(BN(1x1Conv(E)))))) (3)

[0014] f a = ReLU(3x3Conv(f g +f l )) (4)

[0015] In the formula, 1x1 represents a 1x1 convolutional layer, 3x3 represents a 3x3 convolutional layer, f a is the attention feature, f l is the local feature, and f g is the global feature. Optionally, the group attention module includes four convolutional blocks and three group attention modules, the first uses the overall attention module to merge the rough prediction and the feature map to highlight the entire object region, and then uses three convolutional blocks to obtain the bottom-up features, the three convolutional blocks include f3', f4' and f5', and then combines the f2 feature to construct a pyramid network, in each layer of the pyramid network from top to bottom path, the group attention module is used to mine multi-scale features; the re-grouped features f∈R C×H×W are obtained in the following way:

[0016]

[0017] Then the re-grouped features are respectively fed into the spatial attention and the channel attention to enhance the texture features, in the spatial attention, a 3x3 deconvolutional layer with one output channel is used to compress the spatial information, and then a sigmoid function is used to normalize the interval (0, 1), which is expressed as:

[0018] SA = σ(Deconv(f)) (6)

[0019] Where σ represents the sigmoid function, Deconv refers to the 3x3 deconvolutional layer, and SA represents the spatial attention feature.

[0020] In the channel attention, a global average pooling and two 1x1 convolutional layers are used to reduce the dimension, and then a ReLU and a sigmoid function are applied to obtain the channel attention:

[0021] CA = σ (1 x 1Deconv (ReLU (1 x 1Conv (GAP (f)))) ) (7)

[0022] where 1 x 1Deconv denotes a 1 x 1 deconvolution layer, GAP is a global average pooling;

[0023] Finally, the spatial attention feature and the channel attention feature are added to obtain a weight feature f', which is used as an input attention map of the next group attention module:

[0024] f' = UP (f · (SA + CA)) (8)

[0025] where UP represents an up-sampling operation, and CA is a channel attention.

[0026] Optionally, the context feature recovery module includes three branches, namely a bottom feature, a global feature, and a high-level feature. A 3 x 3 convolution layer is added to each of the three branches, and then a splicing operation is used to guide the high-level feature and the bottom feature using the global feature. A connection operation and a convolution layer are used to obtain the next global feature, and then a global average pooling is used to obtain a pooled feature and Finally, a connection operation and a convolution layer are used to obtain the next global feature f' global , the process is represented as:

[0027]

[0028]

[0029]

[0030] Optionally, element-wise addition operations are used to increase the missing context features in the bottom feature and the high-level feature, and then another group of 3 x 3 convolution layers are added to each of the two branches to obtain enhanced features f' low and f' high , and finally a connection operation and a 3 x 3 convolution layer are applied to fuse these context features, and the process is represented as:

[0031]

[0032]

[0033] f' = 3 x 3Conv (Concat (f' low , f' high , f' global )) (14).

[0034] Optionally, the high-level features are fused with the bottom-level features, and an edge map is used to supervise them; a residual channel attention block is executed in the high-level features as guidance information, and is fused with the bottom-level features, followed by two 3x3 convolution layers, a normalization layer and a ReLU layer to obtain the fused edge map f edge This process is expressed as:

[0035] f edge = CBR(CBR(RCBR(f'5) + f'2)) (15)

[0036] Where RCBR is a residual channel attention block, and CBR represents a combination of a 3x3 convolution layer, BatchNorm2d and ReLU.

[0037] Optionally, the loss function of the progressive enhancement network combines weighted binary cross-entropy loss, cross-joint loss and consistency enhancement loss, calculates the difference between the center pixel and its surrounding environment, pays more attention to hard pixels to enhance the model generalization ability, and considers the relationship between pixels and highlights the entire camouflage area.

[0038] The loss function is defined as:

[0039]

[0040] Where λ is a hyperparameter, set to 1, is the weighted binary cross-entropy loss, is the IoU loss, and L CEL is the consistency enhancement loss.

[0041] Optionally, the dice loss loss function is used to handle the strong imbalance between positive and negative samples; and the total loss function is expressed as:

[0042] L total = L(P location , GT) + L(P restoration , GT) + L dice (P edge , Edge) (17)

[0043] Where P location is the predicted position map, P restoration is the predicted recovery map, GT is the labeled Ground Truth map, L dice is the dice loss loss function, P edge is the predicted edge map, and Edge is the labeled edge map.

[0044] The beneficial effects of the present application are that the present application provides a progressive enhancement network for camouflage object detection, proposes a new bionic framework, a progressive enhancement network (PENet) called COD, which greatly provides the performance of camouflage target detection in a progressive enhancement manner to locate objects, enhance texture features, and restore boundaries. An object localization module (OLM) is proposed to infer the initial position of the camouflage object, which can effectively extract global information and local features, so as to accurately determine the position of the camouflage object. A group attention module (GAM) is also designed to refine the texture, and a context feature restoration module (CFRM) is designed to restore clear boundaries. The framework significantly improves the existing camouflage object segmentation performance. It is composed of three key modules, namely the object localization module, the group attention module and the context feature restoration module, which are respectively used for accurately locating the camouflage object, refining the texture and restoring the boundary in a progressive enhancement manner. Specifically, the object localization module is composed of a global attention part and a local attention part, which simulates the human detection process by first locating the target globally. The group attention module is designed as a group attention module, which contains channel attention and spatial attention, to focus on different scales of detailed functional context feature restoration module uses attention guided fusion of texture features between different layers to realize boundary reduction. A large number of experiments on four benchmark datasets show that our PENet realizes the most advanced performance of camouflage object detection. The qualitative and quantitative results prove the effectiveness of our method. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A schematic diagram of a progressive enhancement network for camouflage object detection according to an embodiment of the present application.

[0046] Figure 2 A schematic diagram of a progressive enhancement network for camouflage object detection according to an embodiment of the present application.

[0047] Figure 3 A schematic diagram of an object module of a progressive enhancement network for camouflage object detection according to an embodiment of the present application.

[0048] Figure 4 A schematic diagram of a group attention module of a progressive enhancement network for camouflage object detection according to an embodiment of the present application.

[0049] Figure 5 A schematic diagram of a context feature restoration module of a progressive enhancement network for camouflage object detection according to an embodiment of the present application.

[0050] Figure 6 A schematic diagram of a qualitative comparison of a progressive enhancement network for camouflage object detection according to an embodiment of the present application and an advanced method.

[0051] Figure 7 A feature diagram of a visual localization, refinement and recovery of a progressive enhancement network for camouflage object detection according to an embodiment of the present application.

[0052] Figure 8 Detailed comparison results of a progressive enhancement network for camouflage object detection according to an embodiment of the present application with other 37 state-of-the-art methods on four benchmark datasets.

[0053] Figure 9 Ablation analysis of three datasets of a progressive enhancement network for camouflage object detection according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the objects, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to specific embodiments and drawings.

[0055] It should be noted that, unless otherwise defined, technical terms or scientific terms used in one or more embodiments of the present application should be understood as their common meanings to those skilled in the art to which the present disclosure pertains. The terms "first", "second" and similar terms used in one or more embodiments of the present application do not represent any order, number or importance, but are only used to distinguish different components. The terms "include" or "contain" and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right" and the like are only used to represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships may also change accordingly.

[0056] Reference Figures 1-9 One or more embodiments of the present application provide a progressive enhancement network for camouflage object detection, which includes an object localization module, a group attention module and a context feature recovery module, and the object localization module, the group attention module and the context feature recovery module are respectively used to accurately locate the camouflaged object, refine the texture and recover the boundary in a progressive enhancement manner; for an input RGB image I with a size of HxW, Res2Net-50 is used as the backbone to extract multi-level features from the input image, then a plurality of object localization modules are used to locate potential camouflage objects, the image at the initial position is input to the group attention module to refine the details, and finally a plurality of context feature recovery modules are used to recover the complete object to obtain clear boundaries.

[0057] Figure 2The overall framework of the progressive enhancement network (PENet) for camouflage object detection is shown, specifically, for an input RGB image I of size HxW, we employ Res2Net-50 as the backbone to extract its multi-level features from the input image, denoted as f i (i = 1, 2,..., 5) Then, we use three object localization modules (OLMs) to localize potential camouflaged objects. Next, we input the image at the initial location to another three convolutional blocks and a group attention pyramid network to refine the details. Finally, to obtain clear boundaries, we utilize three context feature restoration modules (CFRMs) to restore the complete objects.

[0058] As Figure 3 shown, the object localization module includes a global module and a local module, the global module is implemented in a non-local manner to capture long-range dependencies, thereby enhancing the context semantic representation from a global perspective; the local module is to extract local information by using a number of convolutional layers; the global module and the local module explore the potential object region in a complementary manner.

[0059] In the global module, first, the receptive field is expanded by using a receptive field module structure, and then four feature maps B, C, D and E are obtained through four 1x1 convolutional layers, where {B, C, D, E} ∈ R C×H×W Then, it is reshaped into R C×N The transpose of B is multiplied by the C matrix, and a soft magnetic layer is executed to calculate the global spatial attention map; the matrix multiplication operation is used to obtain the semantic enhanced global feature f g ∈ R C×H×W ,R C×H×W represents a real number set with C channels, H height and W width, R C×N represents an N-column real number set with C channels; the process is described as follows:

[0060]

[0061]

[0062] where gsa ij represents the influence of the j position on the i position; η is initialized to 0, and more weights are learned subsequently, f i is the i-th layer feature.

[0063] In the local module, a set of 1x1 convolutional layers, BatchNorm2d functions and ReLU functions are used to obtain local features, and then the global features and the local features are subjected to element addition operation to obtain aggregated features; the process is described as follows:

[0064] f l=ReLU(1×1Conv(ReLU(BN(1×1Conv(E))))) (3)

[0065] f a =ReLU(3×3Conv(f g +f l (4)

[0066] In the formula, 1×1 represents a 1×1 convolutional layer, 3×3 represents a 3×3 convolutional layer, and f a For attention features, f l For local features, f g These are global features. For example... Figure 2 As shown, the group attention module includes four convolutional blocks and three group attention modules. We first use a global attention module (HA) to merge coarse predictions and feature maps to highlight the entire object region. Then, we use three convolutional blocks (f3′, f4′, and f5′) to obtain bottom-up features. We then combine the features from f2 to construct a pyramid network. In each layer of the pyramid network's top-down path, group attention modules are used to mine multi-scale features. Figure 4 As shown, we first use the Receive Field Block (RFB) to expand the receive field, then... and The regrouping features f∈R are obtained in the following way. C×H×W :

[0067]

[0068] The regrouped features are then fed into spatial attention and channel attention respectively to enhance texture features. In spatial attention, a 3×3 deconvolutional layer with one output channel is used to compress spatial information, and then the sigmoid function is used to normalize the interval (0,1), expressed as:

[0069] SA=σ(Deconv(f)) (6)

[0070] Where σ represents the sigmoid function, Deconv refers to a 3×3 deconvolution layer, and SA represents spatial attention features;

[0071] In channel attention, global average pooling and two 1×1 convolutional layers are used to reduce dimensionality, and then ReLU and sigmoid functions are applied to obtain channel attention:

[0072] CA=σ(1×1Conv(ReLU(1×1Conv(GAP(f))))) (7)

[0073] where 1x1Conv denotes a 1x1 convolution layer, and GAP is a global average pooling;

[0074] Finally, the spatial attention feature and the channel attention feature are added to obtain the weight feature f', which is used as the input attention map of the next group attention module:

[0075] f'= UP(f-(SA+CA)) (8)

[0076] where UP represents an up-sampling operation, and CA is a channel attention.

[0077] The proposed group attention module is used to refine the texture features at different levels, and these features are used to obtain the texture details of the camouflaged object using the group attention module. As known, the context feature contains rich semantic information, and the fusion of the context feature is crucial for restoring the complete camouflaged object. Therefore, we propose a context feature recovery module (CFRM) to aggregate rich context features to improve the performance of the COD.

[0078] Specifically, as shown in Figure 5 , the context feature recovery module includes three branches, namely the bottom feature, the global feature, and the high-level feature. A 3x3 convolution layer is added to each of the three branches, and then the global feature is used to guide the high-level feature and the bottom feature through a concatenation operation. A connection operation and a convolution layer are then used to obtain the next global feature, and then the bottom feature and the high-level feature are used to obtain the pooled features and by global average pooling. Finally, a connection operation and a convolution layer are used to obtain the next global feature f g ′ lobal , and the process is represented as:

[0079]

[0080]

[0081]

[0082] Meanwhile, in the bottom feature and the high-level feature, an element-wise addition operation is used to increase the missing context feature, and then another 3x3 convolution layer is added to each of the two branches to obtain the enhanced features f l ′ ow and f h ′ igh , respectively. Finally, a connection operation and a 3x3 convolution layer are applied to fuse these context features, and the process is represented as:

[0083]

[0084]

[0085] f′ = 3 × 3 Conv(Concat(f′ low ,f′ high ,f′ global )) (14).

[0086] The high-level features are fused with the low-level features and supervised by the edge map; a residual channel attention block is performed in the high-level features as guidance information, and fused with the low-level features, followed by two 3 × 3 convolution layers, a normalization layer and a ReLU layer to obtain the fused edge map f edge , which is expressed as:

[0087] f edge = CBR(CBR(RCBR(f′5) + f′2)) (15)

[0088] where RCBR is a residual channel attention block, and CBR represents a combination of a 3 × 3 convolution layer, a BatchNorm2d and a ReLU.

[0089] The loss function of the progressive enhancement network combines a weighted binary cross-entropy loss and a cross-joint loss and a consistency enhancement loss, calculates the difference between the center pixel and its surrounding environment, pays more attention to hard pixels to enhance the model generalization ability, and considers the relationship between pixels and highlights the entire camouflage area;

[0090] The loss function is defined as:

[0091]

[0092] where λ is a hyperparameter, set to 1, is a weighted binary cross-entropy loss, is an IoU loss, and L CEL is a consistency enhancement loss.

[0093] The dice loss loss function is used to handle the strong imbalance between positive and negative samples; and the total loss function is expressed as:

[0094] L total = L(P location , GT) + L(P restoration , GT) + L dice (P edge , Edge) (17)

[0095] where P location is a predicted position map, P restoration is a predicted recovery map, GT is a labeled Ground Truth map, and Ldice is the dice loss function, P edge is the predicted edge map, and Edge is the annotated edge map.

[0096] Experimental setup

[0097] Datasets. We evaluate our method using four widely used COD benchmark datasets, including: CHAMELEON, CAMO, COD10K, and NC4K. CHAMELEON contains 76 images collected by Google search engine with the keyword “camouflaged animals”. CAMO has 1,250 camouflage images (split into 1K for training and 0.25K) covering 8 classes. COD10K includes 5,066 camouflage images (3,040 for training and 2,026 for testing) covering 5 superclasses and 69 subclasses. NC4K is another large COD test dataset, including 4,121 images from the Internet. It contains more samples than previous evaluation benchmarks, and it reveals the general ability of COD models well. Following previous work, we combine the train sets of CAMO and COD10K (4,040 images) as the training set and evaluate the rest of the training set.

[0098] Evaluation metrics. Generally, we use four popular standard metrics to evaluate the performance of our method: structural measure (Sα), E-measure (Eφ), weighted F-measure and mean absolute error. Structural measure (Sα) is adopted to calculate the structural similarity between object perception and region perception. E-measure focuses on evaluating the overall and local precision of camouflage target detection, which is related to human visual perception mechanisms. Weighted F-measure is a comprehensive reliable measure of weighted precision and weighted recall. Mean absolute error (M) evaluates the element-wise difference between normalized predictions and ground truth.

[0099] Implementation details. We implement our network in PyTorch and use Res2Net-50 as the backbone, which is pre-trained on ImageNet, and other newly added layers are randomly initialized. Both input images and Ground Truth maps are resized to 352x352 for training and testing. During training, we optimize the network using Adam with a learning rate of 5e-5, divided by 10 every 50 epochs, and apply various augmentation strategies (i.e., horizontal flipping, boundary cropping, and random rotation) to avoid overfitting. Training is performed on a PC equipped with Intel(R) Xeon(R) Gold 6,240, 2.60GHz CPU and NVIDIA Tesla V100 GPU (with 32GB memory), and the whole training takes about 5 hours with a batch size of 36.

[0100] Comparison with SOTA methods:

[0101] To demonstrate the effectiveness of our proposed CINet, we compare it with 37 state-of-the-art COD methods, including NLDF, PiCANet, BASNet, CPD, PoolNet, EGNet, F3Net, SCRN, CSNet, SSAL, UCNet, MINet, ITSD, PraNet, ANet SRM, SINet, R-MGL, TINet, UGTR, PFNet, SLSR, BgNet, MirrorNet, UJSC*, SegMaR, BSA-Net, FBNet, MAUNet, BGNet, ERRNet, CubeNet, D2C-Net, UR-SINet-v2, ZoomNet, C2F-Net, C2F-Net, and SINet-v2. To make a fair comparison, the results of these methods are taken from ZoomNet and BgNet.

[0102] Quantitative evaluation. Figure 8 Detailed comparison results of PENet with other 37 state-of-the-art methods on four benchmark datasets are reported. Among them, the quantitative comparison on four benchmark datasets using four widely used evaluation indicators (i.e., long Sα, M, Eφ) shows that the higher (or lower) is better. The best result is marked with bold font. “—”: indicates not available, *: uses more datasets. Our method outperforms other four benchmark evaluation indicators on four benchmark datasets, and has a greater profit margin than other peers. It can be seen that our proposed method is consistently significantly better than all other previous methods on all four standard indicators, and has a great advantage. For example, compared with the state-of-the-art SegMaR, although it uses an iterative refinement strategy for training and is much better than other methods, our method is still better than it and has a significant improvement. Specifically, compared with the simultaneous localization, segmentation, and ranking network, our PENet improves the long Sα, The improvements are 7.6%, 3.0%, 17.9% and 3.3%. It is worth mentioning that the results of our method are competitive with C2F-Net which uses a context-aware cross-level fusion strategy, and it improves M by 3.9%, 2.3% and 6.8% on CAMO, CHAMELEON and COD10K datasets, respectively. Moreover, our method outperforms the UJSC* method which even introduces extra SOD data for training on all four metrics of the four datasets. Experimental results show that our proposed method is very competitive and the introduction of the concept of "locate-refine-restore" in the COD task indeed improves the detection accuracy.

[0103] Qualitative evaluation. We provide several typical examples in Figure 6 which intuitively show the qualitative results of our PENet and other state-of-the-art methods. It can be seen that most of the compared methods tend to detect some irrelevant environments or ignore certain areas of the camouflaged objects (e.g., rows 2, 4 and 6). In contrast, the detection results of our PENet are more accurate and closer to the ground truth, including large camouflaged objects (e.g., rows 1, 3 and 8), small camouflaged objects (e.g., rows 5 and 6) and low-contrast camouflaged objects (e.g., rows 4, 8 and 9). These results intuitively demonstrate the superior performance of the proposed method. Most importantly, our method can achieve these results without any post-processing, such as a fully connected conditional random field (CRF) to further improve the final prediction.

[0104] Ablation experiments

[0105] To verify the effectiveness of each key module, we design a series of control experiments to verify the importance of the three modules, namely the object location module (OLM), the group attention module (GAM) and the context feature restoration module (CFRM), and the results will be shown in Figure 9 the ablation analysis of the three datasets. "B" means removing all OLM, GAM and CRAM, and they only use pre-trained models through simple concatenation. ↑ (or ↓) means the higher (or lower) the better. The best result is marked in bold. "w / o R" means without RCAB. "w / o E" means without edge supervision. Obviously, it can be observed that each proposed module makes an important contribution to the performance of COD.

[0106] Effectiveness of OLM. In Figure 9 , experiment (a) means that we remove all OLM, GAM and CRAM from the network, and then simply concatenate the last four layers of the pre-trained model. Compared with the basic model (a), we can see that (b) improves M by 3.9%, 2.3% and 6.8% on the CAMO, CHAMELEON, COD10K and NC4K datasets, respectively. In this respect, it outperforms (a) 1.3%, 3.0%, 2.6%, and 3.4%, respectively. This indicates that OLM is effective for target localization and plays an important role in achieving high performance in camouflaged target detection tasks.

[0107] The effectiveness of GAM. From Figure 9 As can be seen in (c) and (a) or (b) and (e), our proposed GAM further improves the metric results. Specifically, compared to (c) and (a), on the CAMO, CHAMELEON, COD10K, and NC4K datasets, E φ The performance improvements were 1.1%, 2.0%, 0.6%, and 0.9%, respectively. Compared to (b) and (e), the performance on the COD10K dataset was significantly improved. The performance was improved by 2.4%, 1.0%, 2.2%, and 1.1%. This clearly demonstrates that the group attention module helps refine multi-scale features and improves the performance of camouflaged target detection.

[0108] Effectiveness of CFRM. We further investigated the contribution of CFRM. From Table 1, we observed that model (d) outperforms (a), (f) outperforms (b), and (g) outperforms (c). These enhancements clearly demonstrate that incorporating contextual features with CFRM does indeed improve COD performance. In particular, compared to model (d), (a) significantly improves performance on the CHAMELEON and COD10K datasets. These figures increased by 4.1% and 2.9%. The significant improvement in the evaluation metrics suggests that CFRM can restore clear boundaries by incorporating contextual information.

[0109] Effectiveness of RCAB. To verify the effectiveness of RCAB, we removed RCAB from model (h) and compared the results of model (j) and model (h). When RCAB was removed, the performance of our model decreased, indicating that utilizing the RCAB module can obtain global features of the image to guide low-level features, which is beneficial for improving the performance of COD.

[0110] Effectiveness of edge supervision. To further validate the effectiveness of our PENet edge supervision, we compared the performance with and without edge supervision. Specifically, compared to (j) and (i), we can find that with edge supervision, On the CAMO, CHAMELEON, COD10K, and NC4K datasets, improvements were achieved by 1.3%, 2.8%, 4.0%, and 0.9%, respectively. This is because edge supervision can guide the prediction results to recover clear boundaries. Therefore, edge supervision also contributes to improving detection performance.

[0111] In addition, such as Figure 7As shown, we visualize the feature maps of the localization, refinement, and restoration stages. From top to bottom, it represents the input image, ground-truth (GT), localization map, refinement map, and restoration map, respectively. The confidence level of red is higher than that of yellow. Apparently, in the localization stage, our network can accurately localize the position of the camouflaged object. In the refinement stage, the texture features of the camouflaged object are enhanced, and some boundary details are improved. In the restoration stage, the boundary of the camouflaged object is clear, and the object is complete. This clearly shows that our PENet can restore the clear boundary in an incremental enhancement manner in order to accurately detect the camouflaged object.

[0112] CONCLUSION

[0113] In the technical scheme of the embodiment, the "locate-refine-restore" strategy is formulated, and the clear and complete camouflaged object is gradually restored, which is helpful to improve the understanding and judgment of the camouflaged object. Specifically, we first propose an object localization module (OLM) to preliminarily locate the camouflage area. Then, we design a group attention module (GAM) to enhance the texture feature representation. Finally, we introduce a context feature restoration module (CFRM) to restore the clear boundary by fusing the context features. We conduct extensive experiments on four benchmark datasets using four widely used evaluation indicators, which shows that the proposed progressive enhancement network can achieve the state-of-the-art performance.

[0114] Although the preferred embodiments of the application have been described, those skilled in the art, once they know the basic inventive concept, can make additional changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the application.

[0115] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A progressive enhancement network for camouflaged object detection, characterized by: The system includes an object localization module, a group attention module, and a context feature recovery module. These modules are used to accurately locate the camouflaged object, refine the texture, and restore the boundary in a progressively enhanced manner, respectively. For an input RGB image I of size H×W, Res2Net-50 is used as the backbone. Features from layers f2, f3, f4, and f5 are extracted from the input image. Then, several object localization modules are used to locate potential camouflaged objects. The image at the initial position is then input to the group attention module to refine the details. Finally, several context feature recovery modules are used to restore the complete object to obtain a clear boundary. The object localization module includes a global module and a local module. The global module is implemented in a non-local manner to capture remote dependencies, thereby enhancing the contextual semantic representation from a global perspective. The local module extracts local information by using several convolutional layers. The global module and the local module explore potential object regions in a complementary manner. First, the receptive domain is expanded using a receptive domain module structure. Then, within the global module, three 1×1 convolutional layers are used to obtain three feature maps B, C, and D, where {B, C, D} ∈ R. C×H×W Then reshape it into R C×N Then, the transpose of B is multiplied by matrix C, and a softmax layer is executed to compute the global spatial attention map; matrix multiplication is used to obtain the semantically enhanced global feature f. g ∈R C×H×W Among them, R C×H×W Let R be the set of real numbers with channel C, height H, and width W. C×N Let N be the set of real numbers in channel C; the process is described as follows: Among them, gsa ij This represents the influence of position j on position i; η is initialized to 0 and gradually learns more weights, f t It is a feature of the t-th layer; In the local module, a 1×1 convolutional layer is used to obtain the feature map E, {E}∈R. C×H×W The BatchNorm2d and ReLU functions are used to obtain local features, and then element-wise addition is performed on the global and local features to obtain aggregated features; the process is described as follows: f l =ReLU(1×1Conv(ReLU(BN(E)))) (3) f a =ReLU(3×3Conv(f g +f l )) (4) In the formula, 1×1 represents a 1×1 convolutional layer, 3×3 represents a 3×3 convolutional layer, and f a For attention features, f l For local features, f g For global features; The group attention module comprises three convolutional blocks and three group attention modules. First, a global attention module is used to merge coarse predictions and feature maps to highlight the entire object region. Then, three convolutional blocks are used to obtain bottom-up feature f3. ′ f4 ′ and F5 ′ Then, a pyramid network is constructed by combining the features of f2. In each layer of the pyramid network from top to bottom, multi-scale features are mined through group attention modules. The regrouped features f∈R are obtained in the following way. C×H×W : Where M is the number of groups. and Let these represent the global feature and attention feature of the m-th group, respectively; The regrouped features are then fed into spatial attention and channel attention respectively to enhance texture features. In spatial attention, a 3×3 deconvolutional layer with one output channel is used to compress spatial information, and then the sigmoid function is used to normalize the interval (0,1), expressed as: SA=σ(Deconv(f)) (6) Where σ represents the sigmoid function, Deconv refers to a 3×3 deconvolution layer, and SA represents spatial attention features; In channel attention, global average pooling and two 1×1 convolutional layers are used to reduce dimensionality, and then ReLU and sigmoid functions are applied to obtain channel attention: CA=σ(1×1Conv(ReLU(1×1Conv(GAP(f))))) (7) Where 1×1Conv represents a 1×1 convolutional layer, and GAP represents global average pooling; Finally, the spatial attention features and channel attention features are added together to obtain the weighted feature f. ′ This serves as the input attention graph for the next group attention module: f′=UP(f·(SA+CA)) (8) Here, UP represents upsampling operation, and CA represents channel attention.

2. The progressive enhancement network for detecting camouflaged objects as described in claim 1, characterized in that, The context feature recovery module includes three branches: low-level features, global features, and high-level features. Add a 3×3 convolutional layer to each of the three branches, then concatenate them to use global features to guide the high-level and low-level features respectively. Next, use convolutional layers and join operations to obtain the next global feature. Finally, apply global average pooling to the low-level and high-level features to obtain the pooled features. and Finally, concatenation operations and convolutional layers are used to obtain the next global feature f′. global The process is represented as follows:

3. The progressive enhancement network for detecting camouflaged objects as described in claim 2, characterized in that, Simultaneously, element-wise addition is used to augment missing contextual features in both low-level and high-level features. Then, another set of 3×3 convolutional layers is added to each of the two branches to obtain the enhanced feature f′ respectively. low and f′ high Finally, connection operations and 3×3 convolutional layers are applied to fuse these contextual features. This process can be described as follows: f′=3×3Conv(Concat(f′ low ,f′ high ,f′ global )) (14)。 4. The progressive enhancement network for camouflaged object detection as described in claim 3, characterized in that, High-level features are fused with low-level features and supervised by an edge map; This involves performing residual channel attention blocks in high-level features as guiding information, and then combining them with low-level features, followed by two 3×3 convolutional layers, a normalization layer, and a ReLU layer to obtain the fused edge map f. edge This process can be described as follows: f edge =CBR(CBR(RCBR(f′5)+f′2)) (15) Here, RCBR is the attention block for the remaining channels, and CBR represents a combination of a 3×3 convolutional layer, BatchNorm2d, and ReLU.

5. The progressive enhancement network for camouflaged object detection as described in claim 1, characterized in that, The loss function of the progressive enhancement network combines weighted binary cross-entropy loss, cross-joint loss, and consistency enhancement loss. It calculates the difference between the center pixel and its surrounding environment, pays more attention to hard pixels to enhance the model's generalization ability, and considers the relationship between pixels and highlights the entire camouflage region. The loss function is defined as follows: Where λ is a hyperparameter, set to 1. For weighted binary cross-entropy loss, For IoU loss, L CEL To enhance consistency, losses are increased.

6. The progressive enhancement network for camouflaged object detection as described in claim 5, characterized in that, The Dice loss function is used to handle the strong imbalance between positive and negative samples; and the total loss function is expressed as: L total =L(P location ,GT)+L(P restoration ,GT)+L dice (P edge ,Edge) (17) Among them, P location It is a predicted location map, P restoration It is a predictive recovery plot, GT is the labeled Ground Truth plot, and L... dice It is the dice loss function, P edge It is the predicted edge map, and Edge is the labeled edge map.

Images