Camouflage object segmentation method based on high and low resolution interaction and multi-scale edge perception

Abstract

The embodiment of the invention provides a camouflage object segmentation system and generation method based on high and low resolution interaction and multi-scale edge perception. The segmentation system is composed of an encoder, a channel attention module, a high-low resolution interaction module, a multi-scale edge sensing module and a context feature fusion module. According to the system, important channel information is obtained by performing channel attention on an input image, additional branch detection edge information is fused with channel attention information, so that the model pays more attention to the edge information, and finally, a final segmentation image is obtained through an effective context feature fusion module. The system is of an end-to-end structure, and no extra operation is needed. The method mainly comprises the following steps: (1) building a system development platform; (2) dividing and reading the segmentation data of the camouflage object; (3) constructing a disguise object segmentation system; (4) training and testing a segmentation system; and (5) evaluating the camouflage object segmentation system. According to the method, the segmented image is comprehensively evaluated through the four evaluation indexes, and the result shows that the image segmentation result is more accurate. Through the construction of the segmentation system and the proposal of the generation method, the purpose of accurately positioning and segmenting the camouflage object can be achieved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, camouflaged object segmentation, and deep learning. Specifically, it involves a camouflaged object segmentation method based on the interaction of high and low resolution and multi-scale edge perception. This method aims to achieve comprehensive detection of camouflaged object areas and ensure the accuracy of segmentation results. This method enables more efficient identification and segmentation of camouflaged objects in images, providing strong technical support for applications in the field of computer vision. Background Art

[0002] The relentless pursuit of predators is the driving force behind prey evolution, prompting them to develop a variety of escape strategies. Camouflage is undoubtedly the most sophisticated and effective of these methods. To evade predators, prey have gradually evolved the ability to seamlessly blend into their surroundings. Conversely, predators, driven by survival, have also developed keen observational skills to decipher camouflage strategies. This is a challenging task, as prey camouflage strategies aim to deceive the observer's visual perception system by mimicking the surrounding environment, creating a visual appearance that is highly similar to the background. To decipher this camouflage, predators need to possess extensive visual perception knowledge to distinguish subtle differences between foreground objects and the background.

[0003] Camouflaged object segmentation is a challenging task. Its core goal is to simulate the predator's visual system and accurately locate foreground targets that exhibit a high degree of complex similarity to their surroundings. Traditional camouflaged object segmentation methods rely primarily on hand-crafted low-level features, such as texture, 3D convexity, and motion, to attempt to decipher the prey's camouflage strategies. However, these methods are often limited by their limited discriminability, making it difficult to accurately identify camouflaged objects in complex natural environments. With the continuous development of deep learning, convolutional neural networks, with their powerful feature extraction capabilities, have brought new solutions to the camouflaged object segmentation task. Existing deep learning methods strive to achieve efficient and accurate camouflaged object segmentation. They borrow from the strategy used by predators to capture their prey: first, they use global localization to locate the prey's approximate position, then focus on details for precise identification, ultimately successfully capturing the target. Additionally, some methods utilize frequency learning to suppress high-frequency texture information, further highlighting the difference between the camouflaged object and the background.

[0004] However, although the camouflaged object segmentation task has achieved remarkable success, it still faces huge challenges. The main problems are: (1) The camouflaged objects and their backgrounds have a high degree of complex similarity, making it difficult to roughly locate those camouflaged targets; (2) Due to the extremely fuzzy object boundaries, even if rough positioning is achieved, it is difficult to obtain accurate segmentation; (3) High-resolution features will lose detailed information when downsampling, and low-resolution features will introduce noise when upsampling, resulting in inaccurate segmentation; (4) When faced with complex scenes such as multiple objects and occlusion, the performance is significantly reduced. Summary of the Invention

[0005] In order to solve the problems existing in the camouflaged object segmentation task and further improve the accuracy of segmentation, the present invention proposes a camouflaged object segmentation method based on the interaction of high and low resolution and multi-scale edge perception.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] An embodiment of the present invention provides, on one hand, a camouflaged object segmentation system with high-low resolution interaction and multi-scale edge perception, including:

[0008] It consists of an encoder, a channel attention module, a high-low resolution interaction module, a multi-scale edge perception module and a context feature fusion module.

[0009] The encoder, composed of a PyramidVisionTransformer, aims to accurately extract the initial feature set of the camouflaged object image. It progressively refines the input image to capture and integrate multi-level features. It also leverages a self-attention mechanism to capture both local and global information, ensuring comprehensive and in-depth extraction of camouflaged object features.

[0010] The channel attention module can fuse features of different scales and significantly improve the performance of camouflaged object segmentation by emphasizing key channels in feature maps. This module consists of two parts: a convolutional layer and a channel attention layer. The convolutional layer extracts local features of the image through two 3×3 convolution operations with the same number of channels. The channel attention layer consists of a pooling module and a channel reduction and expansion module. It performs weighted processing based on the channel attention weights of the input feature map, thereby highlighting important channel information while suppressing those relatively minor channels. The difference is that the activation function used in the convolutional layer is the PReLU activation function. Compared with the ReLU activation function, it is no longer completely unresponsive to negative inputs, thereby better fitting the data.

[0011] The high- and low-resolution interaction module aims to improve the accuracy of segmentation targets through a complementary approach. Specifically, it refines low-resolution representations using high-resolution features, while using low-resolution representations to provide feedback to high-resolution features, thereby ensuring the accuracy of segmentation results. Specifically, the four different levels of features output by the channel attention module are x1, x2, x3, and x4. The interaction between x1 and x3, and the interaction between x2 and x4, ensure the effective fusion of features at different scales, thereby obtaining accurate segmentation targets.

[0012] The multi-scale edge perception module, consisting of a Canny edge detection module and a multi-scale boundary attention module, performs multi-scale edge perception on the edges of camouflaged objects to obtain detailed boundary information and improve segmentation accuracy. First, the Canny edge detection module accurately locates edges by smoothing the image using a Gaussian filter to reduce noise interference on edge detection. Subsequently, the image's gradient magnitude and direction are calculated to capture edge strength and direction information. Finally, non-maximum suppression techniques are used to further refine edge locations. The multi-scale boundary attention module captures and fuses edge information at different scales, effectively highlighting and enhancing key edge regions while suppressing non-edge regions.

[0013] The role of the context feature fusion module is to enhance the representation ability of the backbone features by integrating the context information of four different levels, and to obtain the final prediction output map by aggregating all the hierarchical information into the x2 feature level.

[0014] Another embodiment of the present invention provides a method for generating a camouflaged object segmentation system based on high- and low-resolution interaction and multi-scale edge perception, comprising the following steps:

[0015] (1) Build a development platform for implementing a camouflaged object segmentation system based on high- and low-resolution interaction and multi-scale edge perception. The hardware platform of the present invention is a server based on an AMD EPYC 7542 CPU and an NVIDIA GeForce RTX 4090. The server has 24GB of video memory and 50GB of internal memory. The software platform is an Ubuntu 18.04 operating system with an operating environment of CUDA 11.3.0, Pytorch 1.10.2, and Python 3.8.

[0016] (2) Data division and reading of camouflaged object segmentation images. COD10K (3040 images for training, 2026 images for testing), CAMO (1000 images for training, 250 images for testing), and CHAMELEON (76 images for testing) were selected. The input image size was 704×704, and the batch size was set to 8.

[0017] (3) Construct a camouflaged object segmentation system based on the interaction of high and low resolution and multi-scale edge perception. The constructed segmentation system is as described above.

[0018] (4) Training and testing of the segmentation system. The present invention uses a batch processing-based method for training, selecting 4040 images for training and the remaining images for testing. During the training process, in order to improve the convergence speed and convergence ability of the model, the present invention uses an exponentially decaying learning rate to further optimize the system. The AdamW optimization algorithm is used to perform gradient descent operations based on the changes in the loss function to update the network model parameters.

[0019] (5) Evaluation of the camouflaged object segmentation system. In order to verify the performance of the segmentation system, the present invention uses the mean absolute error (MAE), weighted F-measure, enhanced alignment metric E-measure and structural metric Sα for evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] Figure 1 It is the overall framework diagram of the system of the present invention;

[0022] Figure 2 is a schematic diagram of the channel attention module of the system of the present invention;

[0023] Figure 3 is a schematic diagram of the high- and low-resolution interaction modules in the system of the present invention;

[0024] Figure 4 is a schematic diagram of a multi-scale edge perception module in the system of the present invention;

[0025] Figure 5 Schematic diagram of the context feature fusion module in the system of the present invention;

[0026] Figure 6 is a flow chart of the system generation method of the present invention;

[0027] Figure 7 Schematic diagram comparing the actual segmentation map and the predicted segmentation map obtained by the system of the present invention on the COD10K dataset;

[0028] Figure 8 Schematic diagram comparing the actual segmentation map and the predicted segmentation map obtained by the system of the present invention on the CAMO dataset;

[0029] Figure 9 It is a schematic diagram comparing the real segmentation map and the predicted segmentation map obtained by the system of the present invention on the CHAMELEON dataset. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0031] Figure 1 The overall framework diagram of the system of the present invention is shown in FIG. The system consists of an encoder, a channel attention module, a multi-scale edge perception module, a high-low resolution interaction module, and a context feature fusion module.

[0032] Figure 2 This is a schematic diagram of the channel attention module in the system of the present invention. Specifically, it consists of a convolutional layer and a channel attention layer, which emphasizes important channels in the features to obtain important feature information of the camouflaged object. The convolutional layer is composed of two 3×3 convolution blocks with a constant number of channels, a stride of 1, and a padding of 1. An activation function PReLU is included between the two convolution blocks. The channel attention layer includes an adaptive average pooling, which performs global average pooling on each channel of the input feature map to obtain globally important information. The other is a channel attention layer, which includes a 1×1 channel reduction convolution block, a ReLU activation function, a 1×1 channel expansion convolution block, and a Sigmoid block. Its purpose is to learn the importance weight of each channel. The feature map that has passed the convolution layer is then multiplied by the learned channel weights to obtain a weighted feature map. Finally, the original feature map and the weighted feature map are added to obtain an output feature map that incorporates the channel attention information, which can better capture and utilize useful information in the image.

[0033] Figure 3 This is a schematic diagram of the high-low resolution interaction module in the system of the present invention. The high-low resolution interaction module receives the outputs x1, x2, x3, and x4 from the channel attention modules at different levels. Convolve x1 to the size of x3 to obtain the feature map x1down, interpolate x3 to the size of x1 to obtain the feature map x3up, and then perform x1×x3up and x3×x1down. Similarly, convolve x2 to the size of x4 to obtain the feature map x2down, interpolate x4 to the size of x2 to obtain the feature map x4up, and then perform x2×x4up and x4×x2down. Finally, feature maps x1H, x2H, x3L, and x4L at different levels that compensate for the feature information are obtained.

[0034] Figure 4This is a schematic diagram of the multi-scale edge perception module in the system of the present invention. It consists of two parts: a Canny edge detection module that detects edges using the Canny edge detection algorithm (with a low threshold set to 50 and a high threshold set to 150); and a multi-scale boundary attention module that has three branches: a 3×3 convolution block with a stride of 1 and a padding of 1; a 5×5 convolution block with a stride of 1 and a padding of 2; and a 7×7 convolution block with a stride of 1 and a padding of 3. The goal is to extract edge information at different scales from edge features, then combine the information from the three branches to fuse features at different scales. A 1×1 convolution block is used to return the number of channels to their original number, and a sigmoid block is used to generate an attention map. Finally, the attention map is multiplied with the original input feature map to obtain an edge-enhanced weighted feature map, which allows the feature map to retain more boundary information.

[0035] Figure 5 This is a schematic diagram of the contextual feature fusion module in the system of the present invention. It effectively fuses feature maps from the high- and low-resolution interaction module and the multi-scale edge-aware module. Specifically, after obtaining x1H, x1msba, x2H, x2msba, x3L, x3msba, x4L, and x4msba, the feature map of x1H is multiplied by x1msba and then added. This allows x1msba, which contains edge information, to emphasize the boundary regions of x1H, thereby making x1H focus more on the boundary. x1H is then added back to the multiplied result, preserving the original feature information while better integrating low-level and high-level features and boundary information. The principle of x2H and x2msba, x3L and x3msba, and x4L and x4msba is the same. Finally, the feature maps at the four different levels are reduced to 1 through a 1×1 convolution, resulting in a binary segmentation map. These four binary segmentation maps are then resized to the same size as x2 and added together to obtain the final binary segmentation map. This is then upsampled to the input image size and output.

[0036] Figure 6 The flowchart of the system generation method of the present invention is divided into five steps: (1) building a system development platform; (2) dividing and reading camouflaged object segmentation data; (3) building a camouflaged object segmentation system; (4) training and testing the segmentation system; and (5) evaluating the camouflaged object segmentation system.

[0037] Mean absolute error (MAE), weighted F-measure, enhanced alignment metric E-measure and structural metric Sα

[0038] Figure 7The figure below compares the true and predicted segmentation maps obtained by the proposed system on the COD10K dataset. To validate the performance of the proposed system, this example was evaluated on COD10K, the largest dataset of camouflaged objects currently available. The mean absolute error, weighted F-measure, enhanced alignment metric E-measure, and structural metric Sα on the COD10K dataset reached 0.025, 0.791, 0.868, and 0.929, respectively. The proposed system's segmentation results on the COD10K dataset are more accurate and better capture camouflaged objects, further demonstrating the performance of the proposed system.

[0039] Figure 8 The figure below shows a comparison between the true and predicted segmentation maps obtained by the system of the present invention on the CAMO dataset. To verify the performance of the system of the present invention, this example was evaluated on the CAMO dataset. The mean absolute error, weighted F-measure, enhanced alignment metric E-measure, and structural metric Sα on the CAMO dataset reached 0.060, 0.792, 0.842, and 0.899, respectively. The segmentation results of the system of the present invention on the CAMO dataset are closer to the true segmentation maps and have less noise, further demonstrating the performance of the system of the present invention.

[0040] Figure 9 The figure below compares the true and predicted segmentation maps obtained by the system of the present invention on the CHAMELEON dataset. To verify the performance of the system of the present invention, this example was evaluated on the CHAMELEON dataset. The mean absolute error, weighted F-measure, enhanced alignment metric E-measure, and structural metric Sα on the CHAMELEON dataset reached 0.020, 0.890, 0.965, and 0.925, respectively. The segmentation results of the system of the present invention on the CHAMELEON dataset are low in noise, with significant detail, and closer to the true segmentation map, further demonstrating the advantages of the system of the present invention.

Claims

1. A camouflaged object segmentation system based on high-low resolution interaction and multi-scale edge perception, characterized by include: An encoder, used to accurately extract the initial feature set of the camouflaged object image; The channel attention module is used to fuse features of different scales and improve the performance of camouflaged object segmentation by emphasizing key channels in the feature map. The high-low resolution interaction module is used to refine the low-resolution representation through high-resolution features and use the low-resolution representation to provide feedback to the high-resolution features to improve the accuracy of segmentation targets. Multi-scale edge perception module, used for multi-scale perception detection of the edges of camouflaged objects; The context feature fusion module is used to integrate context information at four different levels to enhance the representation ability of backbone features.

2. The system according to claim 1, wherein: The high-low resolution interaction module realizes the effective fusion of features of different scales through the interaction between x1 and x3, and x2 and x4, thereby improving the accuracy of the segmentation results.

3. The system according to claim 1, wherein: The Canny edge detection module of the multi-scale edge perception module can accurately locate the edges of camouflaged objects, and further capture and fuse edge information of different scales through the multi-scale boundary attention module, thereby improving the accuracy of segmentation.

4. The system according to claim 1, wherein: The context feature fusion module integrates four levels of context information and aggregates this information into the x2 feature level, thereby enhancing the representation ability of the backbone features and outputting the final prediction result graph.

Images