Dual-domain fusion iterative refinement camouflage object detection method

Abstract

The invention discloses a camouflage object detection method based on double-domain fusion iteration refinement, and the method comprises the steps: extracting the features of a space domain and a frequency domain: extracting the multi-scale features of an image through a MambaVision backbone network, and obtaining the enhanced multi-scale information through feature fusion; the two-stage segmentation model comprises a coarse-grained positioning stage and a fine-grained optimization stage, in the coarse-grained positioning stage, a frequency-space fusion module and a double-domain difference convolution module are adopted to enhance initial features, and a coarse camouflage object positioning map is obtained by using a multi-scale fusion module; in the fine granularity optimization stage, detail information of a camouflage object is recovered through an iterative optimization process by using an iterative mask strategy, and finally a fine segmented image is generated. The FSF module enhances the fusion of low-frequency and high-frequency information and spatial features, so that the structural information of the camouflage object is displayed; the DDC supplements information lost in the spatial domain by calculating the difference between the frequency domain and spatial domain features; the IMS gradually supplements and optimizes high resolution information to improve the fine-grained segmentation performance of the camouflage object.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a dual-domain fusion iterative refinement camouflaged object detection method. Background Art

[0002] Camouflaged object detection (COD) has become a research hotspot in computer vision in recent years. Its goal is to detect and segment objects that are almost perfectly embedded in their environment. Unlike traditional object detection tasks, COD presents a greater challenge because camouflaged objects are often highly similar to the background, with blurred boundaries and low contrast, making them difficult to detect using standard detection methods. This concealed nature of camouflaged objects is reflected in various applications, such as polyp segmentation and lung infection segmentation in medical imaging, as well as in certain artistic creations.

[0003] Previous research has mostly focused on analyzing target and background features in the RGB domain. However, biological and psychological research indicates that many animals use frequency-dependent visual filtering mechanisms to better perceive camouflaged objects when hunting. In contrast, the human visual system (HVS) often struggles to discern these subtle camouflaged features. This suggests that the task of camouflaged object detection (COD) requires not only emulating the capabilities of the HVS in the RGB domain but also going beyond human vision to exploit potential cues in the frequency domain to better identify these hidden targets. Furthermore, frequency-domain learning better preserves image information during preprocessing compared to traditional spatial downsampling methods, thereby improving accuracy. After an image is transformed into the frequency domain through a transformation (such as the discrete cosine transform (DCT) or Fourier transform) (FT), low-frequency values are typically concentrated in the upper left corner of the transformation matrix. These values correspond to global, stable features in the image, such as large background blocks or the main outlines of objects. These low-frequency components are not affected by image noise or small pixel variations. Furthermore, research has demonstrated that neural networks tend to preferentially learn low-frequency information, demonstrating its discriminative and robust nature.

[0004] Common difficulties in camouflaged object detection tasks include: 1) In complex backgrounds, the structural information of foreground objects is often masked by high-frequency noise in the background, making camouflaged objects difficult to distinguish; 2) The high similarity between the foreground and background in terms of texture and color makes it impossible for spatial domain-based detection methods to effectively distinguish camouflaged objects; 3) When dealing with targets with fine edges and complex structures, a single spatial domain feature is not sufficient to accurately characterize the fine boundaries of the target. Summary of the Invention

[0005] In order to overcome the above problems, the present invention provides a camouflaged object detection method with dual-domain fusion and iterative refinement.

[0006] The technical solution adopted in the present invention is:

[0007] A dual-domain fusion iterative refinement method for camouflaged object detection, which includes spatial and frequency domain feature extraction and a two-stage segmentation model;

[0008] Spatial and frequency domain feature extraction uses the MambaVision backbone network to extract multi-scale features of the image, and obtains enhanced multi-scale information through feature fusion;

[0009] The two-stage segmentation model includes a coarse-grained localization stage and a fine-grained optimization stage. The coarse-grained localization stage uses the frequency-space fusion (FSF) module and the dual-domain difference convolution (DDC) module to enhance the initial features, and uses the multi-scale fusion (MSF) module to obtain a rough camouflaged object localization map; the fine-grained optimization stage uses the iterative mask strategy (IMS) through an iterative optimization process to restore the detailed information of the camouflaged object, and finally generates a fine segmentation image.

[0010] Among them, MambaVision extracts four scale features from the input image, represented as The input image I is reduced to half its original size by downsampling, that is, the size becomes H / 2 and W / 2, the downsampled image is passed to MambaVision, and the downsampled feature map is extracted; the downsampled feature map is restored to the resolution of the original feature map using bilinear interpolation; each downsampled feature map is element-wise added to the original feature map as a new spatial domain feature This operation can retain the detailed information of low-level features and improve feature expression capabilities.

[0011] The FSF module captures the correlation between the frequency and spatial domains by establishing an attention mechanism for high and low frequencies, achieving feature fusion and improving the model's detection performance for camouflaged objects. Specifically, the FSF module captures the correlation within and between frequency domains through two main mechanisms:

[0012] Intra-frequency interaction: There is a significant difference between high-frequency information and low-frequency information, representing edge and texture details in the image (high frequency) and global information in the image (low frequency), respectively. In intra-frequency interaction, high-frequency and low-frequency features are first processed differently, using a smaller convolution kernel for high-frequency features and a larger convolution kernel for low-frequency features. Then, spatial attention is applied to the high-frequency features. Finally, multi-head self-attention is used to find self-correlations between high-frequency and low-frequency features, establishing intra-frequency interaction.

[0013] Out-of-frequency interaction: In addition to intra-frequency interaction, the FSF module also establishes out-of-frequency interaction that fuses the spatial and frequency domains. By combining the frequency and spatial domains, the model can fully utilize information from different domains. The frequency domain provides a rich representation of details and global information, while the spatial domain focuses more on the shape and position of objects. The FSF module adopts an attention fusion strategy, with frequency domain features used as queries and spatial domain features as keys and values to combine information from the two feature domains, thereby enhancing the model's ability to recognize camouflaged objects.

[0014] Among them, DDC is used to calculate the difference between the frequency domain and the spatial domain, and further improves the detection effect of camouflaged objects by supplementing spatial information. It aims to enhance the spatial domain features by utilizing the learnable convolution kernel constructed by dual-domain differences. Specifically, the output of the FSF module is used as the frequency domain feature, and the original spatial domain feature is passed through two convolution layers as the spatial domain feature. First, the difference feature map between the spatial domain feature representation and the frequency domain feature representation is calculated; then, the difference feature map is linearly projected to generate a learnable convolution kernel, which is used to adjust the input spatial domain features to generate a reconstructed feature representation; finally, the reconstructed spatial domain features are connected through residual connections to generate the final feature representation, which is expressed as follows:

[0015]

[0016] This fusion feature combines information from the spatial and frequency domains to more accurately capture the shape, boundary, and texture features of camouflaged objects, thereby enhancing the detection capability of the model.

[0017] Among them, the multi-scale feature fusion (MSF) module progressively fuses features of different scales, adopts step-by-step upsampling and convolution operations, combines feature maps of multiple scales to extract richer contextual information, and outputs a coarse positioning rough map; it can enhance the ability to understand complex scenes and avoid the information loss caused by simple splicing features.

[0018] Among them, the Iterative Mask Strategy (IMS) optimizes the coarse localization map and gradually supplements the multi-scale features with the patch information of the high-resolution image to improve the detail resolution of the segmentation map. The core idea of IMS is to use the coarse prediction map output by the first stage to construct a mask through diffusion, supplement the high resolution to guide the model to more accurately capture the detailed features of the camouflaged object. Specifically, for the nth iteration, the coarse prediction map is first subjected to a Softmax normalization operation. For all pixels with a value greater than 0.5, a pooling operation is applied to diffuse to its neighboring areas to better cover and capture the potential area of the camouflaged object. After the diffusion is completed, the resulting map is used as the mask for the current iteration. Finally, the generated mask is fused with the original image to supplement the high-resolution information of the mask area; the fused image is re-encoded through the decoder to generate a finer segmentation map; this process not only enriches the feature expression, but also enables the model to focus on more detailed and local camouflaged object features; the feature enhancement and re-segmentation operations guided by this mask enable the model to gradually converge to a more accurate camouflaged object prediction through an iterative manner; through the gradual optimization of IMS, the model can effectively correct the erroneous areas of the coarse map in the first stage and better restore the detailed structure of the camouflaged object; this strategy has significant advantages especially in complex backgrounds and can achieve a good balance between detail capture and global consistency.

[0019] The advantages of the present invention are as follows:

[0020] Coarse-grained camouflaged target positioning is achieved by introducing the FSF module for cross-domain feature interaction. The FSF module enhances the fusion of low-frequency and high-frequency information with spatial features by establishing an intra-frequency and extra-frequency interaction mechanism, allowing the structural information of the camouflaged object to be revealed. In this stage, the correlation between the frequency domain and the spatial domain is established. The present invention also introduces dual-domain difference convolution (DDC) to supplement the lost information in the spatial domain by calculating the difference between the frequency domain and spatial domain features. The DDC module focuses on amplifying features that are strong in the frequency domain but relatively weak in the spatial domain, thereby enhancing the fine boundary segmentation capability of the camouflaged object.

[0021] In the iterative refinement stage, the present invention introduces an iterative masking strategy (IMS) to gradually supplement and optimize high-resolution information to improve the fine-grained segmentation performance of camouflaged objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 is a baseline schematic diagram of the present invention;

[0024] Figure 2 This is a first visualization effect display diagram of the present invention;

[0025] Figure 3 This is a second visualization effect display diagram of the present invention;

[0026] Figure 4 This is a third visualization effect display diagram of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] The baseline of the present invention is Figure 1 As shown, the input image is represented as I∈R C×H×W , where H and W represent the height and width of the image respectively, and the RGB color space of the image (expressed as x rgb ∈R 3×H×W ) is processed by the encoder to extract four levels of features, which are combined through initial feature fusion to form a spatial information stream. The input image I is converted into the YCbCr color space, and then the discrete cosine transform (DCT) is applied to convert it into the frequency domain. The first stage is used for coarse positioning. In this stage, the frequency domain stream and the spatial stream are input to the frequency-spatial fusion (FSF) module to establish the association between the frequency domain and the spatial domain. Dual-domain difference convolution (DDC) is used to supplement the spatial information. The multi-scale fusion (MSF) module then performs multi-scale feature integration to generate a coarse segmentation map as the output of the decoder. The focus of the second stage is iterative optimization. The coarse segmentation map is diffused to generate a mask, and the iterative mask strategy (IMS) is used to gradually enhance the high-resolution details of the mask, and finally an accurate segmentation map is obtained. The various modules provided in the present invention will be introduced in detail below.

[0029] Frequency-space fusion module:

[0030] x ycbcr It is divided into a series of 8×8 patches, each of which is processed by DCT into a spectrum, and the final frequency domain stream is denoted as x ycbcr ∈R 192×H×W. These DCT features are downsampled and divided into high-frequency and low-frequency components. For high-frequency features, a smaller receptive field is used to more accurately capture local information and avoid interference from global semantics during local feature extraction. In contrast, for low-frequency features, a larger receptive field is used to ensure that the model can effectively capture the overall characteristics of the image. In addition, considering that high-frequency features contain rich details, a spatial attention mechanism is used to further process high-frequency features. In order to enhance the representation ability of high-frequency and low-frequency information, intra-frequency interaction is designed. Multi-head self-attention (MHSA) is applied to the processed high- and low-frequency information to model the internal information of each frequency band, and obtain X H and X L This approach allows each to capture features and dependencies in its own space without losing important details due to mixing between bands.

[0031] In addition to the intra-frequency interaction, the inter-frequency interaction including spatial information fusion also establishes the correlation between the frequency domain and the spatial domain, thereby improving the robustness and generalization ability of the model. A fusion attention block (FAB) for adaptive feature fusion is designed to achieve multi-level information interaction between the frequency domain and the spatial domain. The fusion operation F(A, B) is defined as:

[0032]

[0033] Here, W Q , W K , W V is the weight matrix, is the scaling factor.

[0034] The high-frequency and low-frequency features are concatenated to reconstruct the complete Y, Cb, and Cr features. These concatenated feature maps are then merged into an integrated feature map, and FAB is used to fuse the complete frequency domain and spatial information. The processed features are residually connected with the initial DCT features to produce an enhanced feature representation:

[0035]

[0036] Through this operation, detail capture and background noise immunity are balanced, and the output is used as enhanced frequency domain information to be input into the next module.

[0037] Dual-domain difference convolution:

[0038] By adopting the FSF module, the correlation information between the frequency domain and the spatial domain is used to enhance the robustness of spatial information. Subsequently, the focus is on using the difference between frequency and spatial information to enhance spatial information. A learnable kernel is used to represent the information difference between the frequency domain and the spatial domain.

[0039] Specifically, the features enhanced by FSF As frequency domain information, and the features with rich original features after passing through two convolutional layers As spatial input. After element-wise subtraction between the two and flattening along the spatial dimension, a linear projection is applied to obtain the differential representation τ i ∈R C×(H×W) .

[0040] Next, the shape before flattening is restored and downsampled to reduce the resolution to HW / 4. Then, the Softmax function is applied for channel-level normalization to balance the information across channels and prevent certain channels from having an excessive impact on the result, thus forming the difference convolution kernel.

[0041] Then, group convolution is used to transform τ′ i Applicable to Grouped convolution not only reduces computational cost, but also improves the extraction of different feature information by refining operations on specific channels. This process is called dual domain differential convolution (DDC). Finally, a residual connection is used to enhance the original features. Indicated as X i .

[0042] The FSF module and DDC operation explore the correlation and difference between the frequency domain and the spatial domain, which can effectively integrate information from both, enhance the robustness to noise, and more accurately capture the shape, boundary and texture characteristics of the camouflaged object.

[0043] Multi-scale fusion module:

[0044] To fuse the multi-scale features processed by DDC, a pyramidal fusion module is designed. This module integrates multi-scale feature maps through progressive upsampling and convolution operations, obtaining richer contextual information. It then generates a coarse localization map, which is fed into the refinement module in the next stage. Specifically, feature maps X2 and X3 are first upsampled and convolved, allowing the model to gradually restore the spatial resolution of the feature maps. This is followed by a fusion operation with X1 via multiplication.

[0045]

[0046] The upsampled feature map is concatenated with the original high-resolution feature map X1 and convolved to generate a new feature map:

[0047]

[0048] Finally, the fused feature map Perform convolution to generate the rough prediction map P of the first stage (1) .

[0049] Iterative masking strategy:

[0050] The segmentation map P output from the coarse localization stage (1) There is a lack of fine boundaries, and previous operations may lead to the loss of image details. To solve this problem, an iterative masking strategy (IMS) is proposed, which emphasizes the iterative utilization of high-resolution information in the refinement stage and alleviates the interference of background noise through a dynamic masking method. In the coarse segmentation map, the area where the sigmoid output exceeds a certain threshold is diffused to the high-resolution target area. This method is based on the assumption that these locations may potentially contain foreground objects. The high-resolution information is then supplemented in the masked area to better guide the model to capture the detailed features of the camouflaged object. Specifically, for the nth iteration, IMS involves the following steps. First, the coarse prediction map P is (n) Apply the sigmoid operation to generate the probability map S (n) Next, by taking S (n) A preliminary mask is constructed by applying a global average pooling operation to expand the mask to the adjacent area, helping to better cover and capture the potential area of the camouflaged object. (n) It is then defined as:

[0051]

[0052] Then the generated mask M (n) Applied to the RGB image xrgb, this method retrieves high-resolution information from the masked region. Considering that RGB images have only three channels, a repetition factor k is set to supplement this information, replicating the high-resolution information from three channels to 3×k. Through feature enhancement and resegmentation guided by this mask, the algorithm effectively corrects erroneous regions in the coarse map from the first stage, gradually converging to a more accurate prediction of the disguised object and better recovering its detailed structure.

[0053] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A dual-domain fusion and iterative refinement method for camouflaged object detection, characterized by: The method includes spatial and frequency domain feature extraction and a two-stage segmentation model; Spatial and frequency domain feature extraction uses the MambaVision backbone network to extract multi-scale features of the image, and obtains enhanced multi-scale information through feature fusion; The two-stage segmentation model consists of a coarse-grained localization stage and a fine-grained optimization stage. The coarse-grained localization stage uses a frequency-space fusion module and a dual-domain difference convolution module to enhance the initial features, and a multi-scale fusion module to obtain a rough camouflaged object localization map. The fine-grained optimization stage uses an iterative masking strategy to restore the detail information of the camouflaged object through an iterative optimization process, and finally generates a fine segmentation image.

2. The dual-domain fusion iterative refinement camouflaged object detection method according to claim 1, characterized in that: The MambaVision extracts four scale features from the input image, represented as The input image I is reduced to half its original size by downsampling, that is, the size becomes H / 2 and W / 2, the downsampled image is passed to MambaVision, and the downsampled feature map is extracted; the downsampled feature map is restored to the resolution of the original feature map using bilinear interpolation; each downsampled feature map is element-wise added to the original feature map as a new spatial domain feature 3. The dual-domain fusion iterative refinement camouflaged object detection method according to claim 1, characterized in that: The frequency-space fusion module captures the correlation between the frequency and spatial domains by establishing an attention mechanism for high and low frequencies, achieving feature fusion and improving the model's detection performance for camouflaged objects. Specifically, the frequency-space fusion module captures the correlation within and between frequency domains through two main mechanisms: Intra-frequency interaction: There are significant differences between high-frequency information and low-frequency information, representing edge and texture details in the image, and global information in the image, respectively. In intra-frequency interaction, high-frequency and low-frequency features are first processed differently, using smaller convolution kernels for high-frequency features and larger convolution kernels for low-frequency features. High-frequency features are then subjected to spatial attention. Finally, multi-head self-attention is used to find self-correlations between high-frequency and low-frequency features, establishing intra-frequency interaction. Out-of-band interaction: In addition to intra-band interaction, the frequency-space fusion module also establishes out-of-band interaction that integrates the spatial and frequency domains. The frequency domain provides rich representation of details and global information, while the spatial domain focuses more on the shape and position of objects; the frequency-space fusion module adopts an attention fusion strategy, with frequency domain features used as queries and spatial domain features as keys and values to combine the information in the two feature domains.

4. The dual-domain fusion iterative refinement camouflaged object detection method according to claim 1, characterized in that: The dual-domain difference convolution is used to calculate the difference between the frequency domain and the spatial domain, further improving the detection of camouflaged objects by supplementing spatial information. It aims to enhance spatial domain features by using a learnable convolution kernel constructed using the dual-domain difference. Specifically, the output of the frequency-space fusion module is used as the frequency domain feature, and the original spatial domain feature is passed through two convolution layers as the spatial domain feature. First, a difference feature map between the spatial domain feature representation and the frequency domain feature representation is calculated. Then, the difference feature map is linearly projected to generate a learnable convolution kernel, which is used to adjust the input spatial domain features to generate a reconstructed feature representation. Finally, the reconstructed spatial features are connected through residual connections to generate the final feature representation, which is expressed as follows:

5. The dual-domain fusion iterative refinement camouflaged object detection method according to claim 1, characterized in that: The multi-scale feature fusion module progressively fuses features of different scales, uses step-by-step upsampling and convolution operations, combines feature maps of multiple scales to extract richer contextual information, and outputs a coarse positioning rough map.

6. The dual-domain fusion iterative refinement camouflaged object detection method according to claim 1, characterized in that: The iterative masking strategy optimizes the coarse localization map and gradually supplements the multi-scale features with patch information from the high-resolution image, thereby improving the detail resolution of the segmentation map. The core idea of the iterative masking strategy is to use the coarse prediction map output by the first stage to construct a mask through diffusion, supplementing the high-resolution image to guide the model to more accurately capture the detailed features of the camouflaged object. Specifically, for the nth iteration, the rough prediction map is first subjected to a Softmax normalization operation. For all pixels with values greater than 0.5, a pooling operation is applied to diffuse them to their neighboring areas. After the diffusion is completed, the resulting map is used as the mask for the current iteration. Then, the generated mask is fused with the original image to supplement the high-resolution information of the masked area. The fused image is re-encoded through the decoder to generate a more refined segmentation map; the feature enhancement and re-segmentation operations guided by this mask iteratively enable the model to gradually converge to a more accurate prediction of the camouflaged object.

Images