A camouflage object detection method based on a two-stage optimization network
By optimizing the network in two stages and utilizing the ResNet50 backbone network and object edge information, the problem of insufficient detection accuracy of disguised objects is solved, and more efficient recognition and segmentation of disguised objects is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNIV OF CHINESE ACAD OF SCI
- Filing Date
- 2021-10-25
- Publication Date
- 2026-07-03
AI Technical Summary
Existing camouflage detection technologies lack sufficient accuracy, making it difficult to accurately identify the boundary differences between camouflage objects and the background.
A two-stage optimization network approach is adopted. In the first stage, ResNet50 is used as the backbone network to locate and roughly map the camouflaged objects. In the second stage, the edge information of the objects is used for optimization. Effective information is extracted through the channel attention module and the global feature-local feature fusion module, and the edge ground truth map is introduced as supervision information for further differentiation.
It improves the accuracy and efficiency of camouflage object detection, and can better identify the boundary between camouflage objects and the background, thus achieving more accurate camouflage object segmentation.
Smart Images
Figure CN114187230B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of camouflaged object detection, and in particular to a camouflaged object detection method based on a two-level optimization network. Background Technology
[0002] As people's demands for smart living become more diverse, the application scope of object detection is also becoming increasingly widespread, with camouflage object detection being one of its important branches. It focuses on the relationship between an object and its surroundings, aiming to detect and segment camouflaged objects that "blend" into their environment. Camouflage is ubiquitous in human life and nature, especially common among animals. In the process of hunting or escaping predators, many animals change their body color, shape, and movements to reduce the difference and contrast between themselves and their surroundings, thereby enhancing their survival ability. These camouflage strategies are usually based on the judgment ability of a blurred observer.
[0003] Biological research shows that the human visual system (HVS) is most sensitive to large areas and color features, and it primarily perceives targets by observing the contrast between an object and its background. Therefore, the HVS may have difficulty recognizing camouflaged objects due to the low contrast between them and their environment.
[0004] However, in certain situations, the identification of camouflaged objects is essential. Besides the fact that this task itself can provide technical support for animal protection through the detection of animal camouflage, many passive camouflage phenomena still exist in life where objects and backgrounds are highly similar: in the medical field, subtle changes in highly similar background tissues may represent a certain lesion; and in the military field, detecting camouflage on the battlefield can turn the tide of battle. Therefore, the development of this task is of great significance.
[0005] In recent years, deep convolutional networks have gradually emerged in various computer vision tasks due to their powerful feature representation capabilities, and some existing methods for detecting camouflaged objects are based on them. Fan et al. proposed SINet, which hierarchically extracts features. These features from different layers are then fused and enhanced to help obtain localization and edge information, thereby achieving accurate detection of camouflaged targets. Yan et al. divided MirrorNet into an original image segmentation stream and a mirror segmentation stream to find visual differences between the original and flipped images, thus better locating camouflaged objects.
[0006] Although these methods are based on the properties of the camouflaged object, there is still room for improvement in edge processing. Therefore, in this invention, we further consider the boundary information of the camouflaged object, allowing the model to better learn the differences between the camouflaged object and the environment at the boundary, thereby more accurately locating and segmenting the camouflaged object. Summary of the Invention
[0007] The purpose of this invention is to address the problem of insufficient detection accuracy in existing camouflage object detection technologies. It proposes a detection method based on multi-task learning, which uses object boundary information as an aid to guide the network to better learn the differences between the texture of the camouflage object and the background texture at the boundary, thereby enabling the network to better locate and segment the camouflage object.
[0008] This invention discloses a method for detecting camouflaged objects based on a two-stage optimization network. The two-stage optimization network consists of two stages. The first stage follows an encoder-decoder structure, using ResNet50 as the backbone for feature extraction, to locate and identify camouflaged objects, forming a coarse mapping. The second stage uses a parallel decoder structure, using object edges as boundary information to encourage the network to focus on object edges and optimize the mapping generated in the first stage.
[0009] Furthermore, the first stage is the pre-feature fusion stage, in which ResNet50 is selected as the backbone network to ensure that deep features can be effectively extracted.
[0010] The goal of this stage is to obtain a rough mapping of the camouflaged object. Based on considerations of computational efficiency and detection accuracy, the following two modules are proposed:
[0011] (1) Channel attention module:
[0012] A channel attention mechanism is applied to the output of each layer of the encoder to retain useful information in shallow features and reduce redundant information.
[0013] Its purpose is to extract useful information, which can be represented as:
[0014]
[0015] Where Attention represents the attention module for this channel. This is the output of the attention module for the i-th channel from bottom to top. This refers to the i-th coded block in the encoding phase.
[0016] The channel attention module has four layers: the first convolutional layer is 1×1 in size, which reduces the number of channels to 32; then there are two 3×3 convolutional layers, each of which is normalized. After these two layers, the image channels remain at 32 and the size remains unchanged; finally, a ReLU function is applied to obtain the final features.
[0017] (2) Global and local feature fusion module:
[0018] This module is implemented in the decoder stage, and its structure is almost symmetrical to the encoder. Each layer of the decoder includes two 3×3 convolutional layers followed by normalization and ReLU functions. This module also introduces cSE and sSE modules to obtain more accurate detection results. These modules can better establish the dependencies between different channels and guide the network to focus on features related to camouflaged objects. In addition, a pyramid pooling module is used on the output of the last layer of the encoder to obtain global features. The input of each layer of the decoder is a combination of the output of the corresponding channel attention module and the upsampled output of the previous layer.
[0019]
[0020]
[0021] Where GLFA represents the decoder module in the global and local feature fusion module, PPM represents the introduced pyramid pooling model, Cat represents the feature map connection, and Upsample represents the upsampling process. This is the output of the bottom-up attention module for the i-th channel. This is the output of the i-th layer in the global and local feature fusion module.
[0022] Thus, the decoder can learn more comprehensive semantic information and construct a prediction module to obtain the final result, which contains a 3×3 convolutional layer, an ELU activation function, and a 1×1 convolutional layer, which can be represented as:
[0023]
[0024] Where ELU represents the ELU activation function, Conv represents the two convolutional layers applied here, and Upsample represents upsampling. This indicates the output of the 4th layer from the bottom up of this module, so that the predicted graph and the final ground truth graph have the same size.
[0025] Furthermore, the second stage is the optimization stage, which aims to further distinguish the camouflaged object from the background using object edge information. In this stage, an edge ground truth map is introduced as supervision information to make the model focus more on the differences in objects at their edges. Specifically:
[0026] The optimization module uses the same decoder structure as the global and local feature fusion modules and forms a parallel correspondence with them. The input of each layer in this module is also a combination of the output of the corresponding channel attention module and the output of the previous layer after upsampling. Therefore, the optimization module can further utilize the features in the previous feature fusion stage to constrain its extraction process and make the feature reconstruction process more comprehensive, thereby refining the final prediction map.
[0027] The final prediction result at this stage can be expressed as:
[0028]
[0029] Where ELU represents the ELU activation function, Conv represents the two convolutional layers applied here, and Upsample represents upsampling. This represents the output of the encoder at the bottom-up 4th layer in this stage, ensuring that the predicted map and the final edge ground truth map have the same size.
[0030] The loss of the two-stage optimization network is obtained by summing the prediction losses of the two decoders. Binary cross-entropy loss is chosen as the loss function, and the overall loss function is:
[0031]
[0032] Where L total Indicates the overall loss. This represents the loss in the pre-fusion stage, where pred_c is the prediction result of the pre-feature fusion module and GT is the ground truth map. This represents the loss of the optimization module, pred_e is the prediction result of the edge optimization module, and GT_edge is the edge ground truth map calculated from the ground truth map.
[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0034] (1) Good performance. The results on the publicly available camouflage object detection dataset show that the present invention achieves the best results in four different evaluation metrics.
[0035] (2) High efficiency: In the architecture adopted by this method, only the extracted useful features are input into the decoding process, which greatly reduces the number of convolution operations, making this method more practical. Attached Figure Description
[0036] Figure 1 This is a schematic diagram of the model framework;
[0037] Figure 2 This is a schematic diagram of the channel attention module. Detailed Implementation
[0038] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.
[0039] The two-stage optimization network consists of two phases. The first phase follows an encoder-decoder structure, using ResNet50 as the backbone for feature extraction to locate and identify camouflaged objects, forming a coarse mapping. The second phase uses a parallel decoder structure, employing object edges as boundary information to encourage the network to focus on these edges and optimize the mapping generated in the first phase.
[0040] As a preferred embodiment of the above, the first stage is the pre-feature fusion stage, in which ResNet50 is selected as the backbone network to ensure that deep features can be effectively extracted.
[0041] The goal of this stage is to obtain a rough mapping of the camouflaged object. Based on considerations of computational efficiency and detection accuracy, the following two modules are proposed:
[0042] (1) Channel attention module:
[0043] In CNNs, different channels will respond to different semantics, and features at different levels contain different levels of detailed information and full-text information. In the process of feature extraction by ResNet50-based encoders, although the output of deep convolutions can see a wider range of the original image, it loses a lot of detailed information. Although shallow outputs retain some detailed information, it is not all useful information. Therefore, a channel attention mechanism is applied to the output of each layer of the encoder to retain useful information in shallow features and reduce redundant information.
[0044] Its purpose is to extract useful information, which can be represented as:
[0045]
[0046] Where Attention represents the attention module for this channel. This is the output of the attention module for the i-th channel from bottom to top. This refers to the i-th coded block in the encoding phase.
[0047] In addition, because the number of channels input to each layer of the decoder becomes 32 after passing through this channel attention module, the number of parameters in the model is greatly reduced, the model size is reduced, and the training and inference speeds are accelerated. This channel attention module has four layers: the first convolutional layer is 1×1 in size, used to reduce the number of channels to 32; followed by two 3×3 convolutional layers, each after which normalization is applied. After these two layers, the image channels remain at 32, and the size remains unchanged; finally, a ReLU function layer is applied to obtain the final features.
[0048] (2) Global and local feature fusion module:
[0049] This module is implemented in the decoder stage, and its structure is almost symmetrical to the encoder. Each layer of the decoder includes two 3×3 convolutional layers followed by normalization and ReLU functions. This module also introduces cSE and sSE modules to obtain more accurate detection results. These modules can better establish the dependencies between different channels and guide the network to focus on features related to camouflaged objects. In addition, a pyramid pooling module is used on the output of the last layer of the encoder to obtain global features. The input of each layer of the decoder is a combination of the output of the corresponding channel attention module and the upsampled output of the previous layer.
[0050]
[0051]
[0052] Where GLFA represents the decoder module in the global and local feature fusion module, PPM represents the introduced pyramid pooling model, Cat represents the feature map connection, and Upsample represents the upsampling process. This is the output of the bottom-up attention module for the i-th channel. This is the output of the i-th layer in the global and local feature fusion module.
[0053] Thus, the decoder can learn more comprehensive semantic information and construct a prediction module to obtain the final result, which contains a 3×3 convolutional layer, an ELU activation function, and a 1×1 convolutional layer, which can be represented as:
[0054]
[0055] Where ELU represents the ELU activation function, Conv represents the two convolutional layers applied here, and Upsample represents upsampling. This represents the output of the 4th layer from the bottom up of this module. This ensures that the predicted graph and the final ground truth graph have the same size.
[0056] As a preferred embodiment of the above, the second stage is the optimization stage. The camouflage object detection task is challenging precisely because the object is highly similar to the environment. Therefore, the optimization stage aims to further distinguish the camouflage object from the background using object edge information. In this stage, an edge ground truth map is introduced as supervision information to make the model focus more on the differences in objects at their edges. Specifically:
[0057] The optimization module uses the same decoder structure as the global and local feature fusion modules and forms a parallel correspondence with them. The input of each layer in this module is also a combination of the output of the corresponding channel attention module and the output of the previous layer after upsampling. Therefore, the optimization module can further utilize the features in the previous feature fusion stage to constrain its extraction process and make the feature reconstruction process more comprehensive, thereby refining the final prediction map.
[0058] The final prediction result at this stage can be expressed as:
[0059]
[0060] Where ELU represents the ELU activation function, Conv represents the two convolutional layers applied here, and Upsample represents upsampling. This represents the output of the encoder at the 4th layer from the bottom up in this stage. This ensures that the predicted map and the final ground truth map have the same size.
[0061] The loss of the two-stage optimization network is obtained by summing the prediction losses of the two decoders. Binary cross-entropy loss is chosen as the loss function, and the overall loss function is:
[0062]
[0063] Where L tot8l Indicates the overall loss. This represents the loss in the pre-fusion stage, where pred_c is the prediction result of the pre-feature fusion module and GT is the ground truth map. This represents the loss of the optimization module, pred_e is the prediction result of the edge optimization module, and GT_edge is the edge ground truth map calculated from the ground truth map.
[0064] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for detecting camouflaged objects based on a two-level optimization network, characterized in that, The two-stage optimization network consists of two stages. The first stage follows an encoder-decoder structure, using ResNet50 as the backbone for feature extraction, which is used to locate and identify camouflaged objects and form a coarse mapping. The second stage uses a parallel decoder structure, with object edges as boundary information, to encourage the network to focus on object edges and optimize the mapping generated in the first stage. The second-stage decoder has the same structure as the first-stage decoder and forms a parallel correspondence with the first-stage decoder. It is used to perform edge optimization on the coarse mapping generated in the first stage. The first stage is the pre-feature fusion stage, in which ResNet50 is selected as the backbone network to ensure that deep features can be effectively extracted. The goal of this stage is to obtain a rough mapping of the camouflaged object. Based on considerations of computational efficiency and detection accuracy, the following two modules are proposed: (1) Channel attention module: A channel attention mechanism is applied to the output of each layer of the encoder to retain useful information in shallow features and reduce redundant information. Its purpose is to extract useful information, specifically as follows: ; in, For the first The output of the attention module for each channel, Indicates the encoder's first... The output features of the layer The channel attention module has four layers: the first convolutional layer is 1×1 in size, which reduces the number of channels to 32; then there are two 3×3 convolutional layers, each of which is normalized. After these two layers, the image channels remain at 32 and the size remains unchanged; finally, a ReLU function is applied to obtain the final features. (2) Global and local feature fusion module: This module is implemented in the decoder stage, and its structure is almost symmetrical to the encoder. Each layer of the decoder includes two 3×3 convolutional layers followed by normalization and ReLU functions. This module also introduces cSE and sSE modules to obtain more accurate detection results. These modules can better establish the dependencies between different channels and guide the network to focus on features related to camouflaged objects. In addition, we also use a pyramid pooling module on the output of the last layer of the encoder to obtain global features. The input of each layer of the decoder is a combination of the output of the corresponding channel attention module and the upsampled output of the previous layer. ; ; in This represents the decoder module within the global and local feature fusion module. Indicates the connection of feature maps, Indicates the upsampling process. For the first The output of the attention module for each channel, The first in the global and local feature fusion module Layer output, PPM represents pyramid pooling module; Thus, the decoder can learn more comprehensive semantic information and build a prediction module to obtain the final result, which contains a 3×3 convolutional layer, an ELU activation function, and a 1×1 convolutional layer.
2. The method for detecting camouflaged objects based on a two-level optimization network as described in claim 1, characterized in that, The second stage is the optimization stage, which aims to further distinguish camouflaged objects from the background using object edge information. In this stage, an edge ground truth map is introduced as supervision information to make the model focus more on the differences in objects at their edges. Specifically: The optimization module uses the same decoder structure as the global and local feature fusion modules and forms a parallel correspondence with them. The input of each layer in this module is also a combination of the output of the corresponding channel attention module and the output of the previous layer after upsampling. Therefore, the optimization module can further utilize the features in the previous feature fusion stage to constrain its extraction process and make the feature reconstruction process more comprehensive, thereby refining the final prediction map. The loss of the two-stage optimization network is obtained by summing the prediction losses of the two decoders. Binary cross-entropy loss is chosen as the loss function, and the overall loss function is: ； in Indicates the overall loss. This indicates the losses during the pre-fusion phase. This represents the loss of the optimization module, i.e., the prediction result of that module. and edge truth map The loss between them, pred_c represents the prediction result of the pre-fusion stage, and GT represents the ground truth map.