A weakly supervised camouflage target detection method and system based on a frequency domain method, a medium and a program product

Abstract

This invention relates to a weakly supervised camouflage target detection method, system, medium, and program product based on a frequency domain approach. It first decomposes the features extracted by the backbone network into high-frequency and low-frequency components. Then, it enhances each set of features separately and gradually optimizes them through a bottom-up coarse-to-fine strategy, effectively highlighting the discriminative features of the camouflage target. Compared with existing RGB methods for weakly supervised camouflage target detection, this invention more significantly highlights camouflage cues in the frequency domain, making the edges of camouflaged objects easier to detect and achieving better segmentation results. Furthermore, the information captured from the frequency domain can compensate for the limitations of sparse annotations in weakly supervised training.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a weakly supervised camouflage target detection method, system, medium, and program product based on frequency domain methods. Background Technology

[0002] The goal of camouflage target detection (COD) is to accurately segment targets that "seamlessly blend" into their environment. These targets often mimic their surroundings or other objects, and their boundaries with the background are blurred, making them difficult for traditional target detection algorithms to identify accurately. This makes COD a challenging task. Despite the challenges, research on camouflage targets still has wide-ranging applications, including species conservation, medical image segmentation, industrial defect detection, artistic creation, and agricultural pest monitoring.

[0003] With the rapid development of deep learning, methods specifically designed for COD tasks have gradually emerged, driving research progress in this field. However, traditional methods rely on low-level features such as color, optical flow, and texture, often performing poorly in scenes with complex backgrounds and diverse camouflage strategies. Recent research has advanced the COD field by introducing innovative model structures and learning strategies. Specifically, Fan et al. proposed SINet, which first coarsely searches for camouflaged targets and then progressively performs fine segmentation, creating the most extensive COD10K dataset containing 10,000 images. Mei et al. proposed using PFNet to detect and eliminate false positive and false negative predictions. Zhai et al. revealed an interactive graph learning framework that draws inspiration from the intricate interactions of multi-level relationships, achieving the dual objectives of detecting camouflaged targets and extracting perceptual edges. Yan et al. developed MirrorNet to advance the COD field. Lv et al. introduced the state-of-the-art Rank-Net for the simultaneous localization, segmentation, and ranking of camouflaged targets.

[0004] However, COD heavily relies on pixel-level annotations during training, which increases annotation costs and complexity due to the task's specific nature. It is understood that annotating a single image takes approximately 60 minutes. Against this backdrop, Weakly Supervised Camouflage Object Detection (WSCOD) emerged, using a small number of annotation points or graffiti in the foreground and background to detect camouflage objects, thus significantly reducing annotation costs and difficulty. He et al. proposed using SAM to generate dense segmentation masks from sparse annotations, introducing the first SAM-based weakly supervised framework, WS-SAM. He et al. constructed a COD dataset using graffiti annotations, marking the birth of the first dataset for the weakly supervised COD task.

[0005] Despite the progress made by WSCOD, many challenges remain. For example, most existing methods for camouflage object detection process image features in the RGB domain. However, due to the inherent similarity between camouflaged objects and the background, as well as the diversity of camouflage types, many discriminative camouflage cues are easily overlooked in the RGB domain, leading to unsatisfactory edge segmentation results for existing detectors. Furthermore, current weakly supervised camouflage object detection methods struggle to fully utilize graffiti annotation information during training, resulting in incomplete predictions. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention aims to provide a weakly supervised camouflage target detection method based on the frequency domain method, which highlights camouflage cues more significantly in the frequency domain, making the edges of camouflaged objects easier to detect and achieving better segmentation results.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A weakly supervised camouflage target detection method based on frequency domain methods, the method comprising the following steps:

[0009] Step 1: Process the given image I and its augmented form I using the ResNet-50 backbone network. aug Processing was performed to extract four sets of image features at different scales. i (i = 1, 2, 3, 4);

[0010] Step 2, Image Features f i It is fed into the frequency decomposition module for processing to obtain high-frequency features f iH and low-frequency characteristics f iL ;

[0011] Step 3: Enhance high-frequency features f using the Feature Enhancement Module (FEM). iH and low-frequency characteristics f iL Enhancement is performed to obtain the enhanced high-frequency feature F. iH and enhanced low-frequency features F iL ;

[0012] Step 4: Use a plug-and-play octave convolution method to learn the enhanced high-frequency features F online. iH and enhanced low-frequency features F iL The target detection results are obtained.

[0013] Step 2 is described in detail below:

[0014] First, for image features f i Perform a two-dimensional discrete Fourier transform to convert it into frequency domain coordinates F. i(u,v);

[0015] Then, for F i The spectrum corresponding to (u,v) is shifted, the low frequency is moved to the center position, and the distance from each frequency domain coordinate point to the center is calculated.

[0016] Finally, a cutoff frequency D0 is set, and the feature F is processed according to the Gaussian filter kernel. i Perform a low-pass filter to reduce the high-frequency characteristics f iH and low-frequency characteristics f iL Separate them.

[0017] Step 3 is as follows:

[0018] For low-frequency feature f iL Its dimensions are h×w×c, and it is processed through a low-frequency enhancement module: First, the low-frequency feature f iL Divide the image into four equal parts according to the channel, so that the feature size after division becomes h×w×c / 4; then, for each divided image feature region f ic (c=1,2,3,4), perform pooling operation; then for each image feature region f ic Perform convolution and upsampling operations to convert each image feature region f ic By restoring the original dimensions and aggregating them together, the enhanced low-frequency feature F in the frequency domain is obtained. iL ;

[0019] For high-frequency feature f iH The high-frequency enhancement module processes the data: First, a context capturer is used to enhance the high-frequency features f. iH The image is processed to obtain contextual information; then convolutional operations are performed to capture detailed local features; and a weighted layer is used to implement residual connections to further capture global information, thus obtaining the enhanced high-frequency features F in the frequency domain. iH .

[0020] In step 3, the low-frequency feature f is enhanced by the low-frequency enhancement module. iL During processing, adaptive pooling operations with different pooling kernel sizes were used for image feature regions of different channels.

[0021] Step 4 is as follows:

[0022] A 4×4 octave convolution kernel is divided into four distinct parts for interactions at different frequencies, ultimately generating high- and low-frequency features after inter-frequency communication to help obtain rich contextual information. Subsequently, an inverse Fourier transform is used to map the features back to the RGB domain, and spatial attention and channel attention are calculated in the RGB domain to enhance the camouflage discriminative features. For the output feature map of each layer, a bottom-up weighted fusion mechanism is adopted to effectively fuse the high-frequency and low-frequency features of the current layer with the coarse prediction map of the previous layer. Finally, after multiple fusions, the final optimized prediction result of the model, i.e., the object detection result, is obtained at the highest layer.

[0023] Step 4 further includes: calculating the loss for each layer of the generated coarse prediction map in order to continuously optimize the prediction results.

[0024] A weakly supervised camouflage target detection system based on a frequency domain method includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the weakly supervised camouflage target detection method based on a frequency domain method as described above.

[0025] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of a weakly supervised camouflage target detection method based on a frequency domain method as described above.

[0026] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of a weakly supervised camouflage target detection method based on a frequency domain method as described above.

[0027] By adopting the above scheme, this invention first decomposes the features extracted by the backbone network into high-frequency and low-frequency components. Then, it enhances each set of features and gradually optimizes them through a bottom-up coarse-to-fine strategy, effectively highlighting the discriminative features of camouflaged targets. Compared with existing RGB methods for weakly supervised camouflaged target detection, this invention more significantly highlights camouflage cues in the frequency domain, making the edges of camouflaged objects easier to detect and achieving better segmentation results. Furthermore, the information captured from the frequency domain can compensate for the limitations of sparse annotations in weakly supervised training. Attached Figure Description

[0028] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0029] like Figure 1 As shown, this invention discloses a weakly supervised camouflage target detection method based on the frequency domain method, which includes the following steps:

[0030] Step 1: Process the given image I and its augmented form I using the ResNet-50 backbone network. aug Processing was performed to extract four sets of image features at different scales. i (i = 1, 2, 3, 4).

[0031] Step 2, Image Features f i It is fed into the frequency decomposition module for processing to obtain high-frequency features f iH and low-frequency characteristics f iL .

[0032] For image feature f i Each pixel in the image has spatial coordinates (x, y), f i (x,y) are the image signal values ​​in the two-dimensional spatial domain. First, for the image features f i Perform a two-dimensional discrete Fourier transform to convert it into frequency domain coordinates F. i (u,v). Considering the feature F obtained by Fourier transform at this time. i The image contains information of different frequencies, requiring the separation of high-frequency and low-frequency information to enhance the image's contours and details separately. A Gaussian filter is a commonly used two-dimensional linear low-pass filter that effectively removes high-frequency noise. Furthermore, due to its smoothing effect, it avoids the ringing effect observed in the inverse Fourier transform. A smoothing effect is achieved by weighting the neighboring pixels of each pixel in the image using a Gaussian filter kernel. Specifically, for F... i The spectrum corresponding to (u,v) is shifted to move the low frequencies to the center position. Then, the distance from each frequency domain coordinate point to the center is calculated. A cutoff frequency D0 is then set, and the feature F is filtered using a Gaussian filter kernel. i Perform a low-pass filter to reduce the high-frequency characteristics f iH and low-frequency characteristics f iL Separate them.

[0033] Step 3: Enhance high-frequency features f using the Feature Enhancement Module (FEM). iH and low-frequency characteristics f iL Enhancement is performed to obtain the enhanced high-frequency feature F. iH and enhanced low-frequency features F iL This is to highlight camouflage clues in the frequency domain.

[0034] For low-frequency feature f iL Its dimensions are h×w×c, and it is processed by a low-frequency enhancement module (LFEM). Specifically, firstly, the low-frequency feature f is... iL Divide the image into four equal parts according to the channel, resulting in a feature size of h×w×c / 4. Then, for each divided image feature region f...ic (c=1,2,3,4) Pooling operations are performed to capture and enhance low-frequency features in the image. Pooling operations better preserve low-frequency information in the feature map and remove some noise by summarizing or aggregating each image feature region. Furthermore, considering that different feature channels contain different information, their frequency separation thresholds naturally differ. For image feature regions f of different channels... ic Adaptive pooling with different sized pooling kernels was employed. This design, compared to fixed-scale pooling, better reduces the loss of spatial information in the image and preserves the target's contour. This method enhances low-frequency features along the channel dimension. Subsequently, for each image feature region f... ic Perform convolution and upsampling operations to convert each image feature region f ic By restoring the original dimensions and aggregating them together, the enhanced low-frequency feature F in the frequency domain is obtained. iL .

[0035] The high-frequency components of an image encompass more complex details, especially in regions of rapid change or frequent fluctuation, such as object edges. In the design of the High-Frequency Enhancement Module (HFEM), residual connections are applied to the acquired high-frequency features. This effectively captures the local details and long-range dependencies of the target, helping to emphasize the high-frequency details of the image. Specifically, a context capturer is first used to process the high-frequency features f... iH The image is processed to obtain contextual information, followed by convolutional operations to capture detailed local features such as edges and textures. Furthermore, residual connections are implemented through a weighted layer to further capture global information. Based on these operations, the enhanced high-frequency features F in the frequency domain are obtained. iH .

[0036] Step 4: Use a plug-and-play octave convolution method to learn the enhanced high-frequency features F online. iH and enhanced low-frequency features F iL The target detection results are obtained.

[0037] A 4×4 octave convolution kernel is divided into four distinct parts for interactions at different frequencies, ultimately generating high- and low-frequency features after inter-frequency communication to help obtain rich contextual information. Subsequently, an inverse Fourier transform is used to map the features back to the RGB domain, and spatial and channel attention are calculated in the RGB domain to enhance the camouflaged discriminative features. For the output feature map of each layer, a bottom-up weighted fusion mechanism is employed to effectively fuse the high-frequency and low-frequency features of the current layer with the coarse prediction map of the previous layer. Finally, after multiple fusions, the optimized prediction result, i.e., the object detection result, is obtained at the highest layer.

[0038] In addition, a loss is calculated for each layer of the generated coarse prediction map to continuously optimize the prediction results.

[0039] This invention can make full use of sparse graffiti annotation information, and compared with the current weakly supervised camouflage target detection methods that process images in the RGB domain, it is more sensitive to the discrimination clues of camouflage objects, especially the edges, and has better edge segmentation effect.

[0040] To illustrate the effects achieved by the present invention, the present invention compares with some currently performing methods, as shown in Table 1. The method of the present invention is compared with currently performing methods (including salient target detection, camouflaged target detection, weakly supervised camouflaged target detection and weakly supervised salient target detection).

[0041] In Table 1, CAMO, COD10K, and NC4K are commonly used datasets for camouflage target detection, containing images of camouflaged targets.

[0042] MAE stands for Mean Absolute Error, which is the average of the absolute values ​​of the differences between the predicted and actual values; S m It is a structure metric used to evaluate the structural similarity between predicted segmentation results and actual segmentation; E m It is an enhanced-alignment measure, which is also used to evaluate the structural similarity between the segmentation result and the true segmentation, but it also considers the global average of image and local pixel matching. It is a weighted F-measure, which is a weighted average of precision and recall. Precision measures the number of correctly classified pixels in the segmentation result, while recall measures the number of correctly classified pixels out of all true positives. It evaluates the overall performance of the segmentation result by balancing precision and recall.

[0043] CVPR is an abbreviation for IEEE Conference on Computer Vision and Pattern Recognition; ICCV is the IEEE International Conference on Computer Vision; AAAI is the Association for the Advance of Artificial Intelligence; ECCV is the European Conference on Computer Vision; LJCAI is the International Joint Conference on Artificial Intelligence; and TPAMI is IEEE Transactions on Pattern Analysis and Machine Intelligence.

[0044] Table 1: Comparison of the present invention with existing methods

[0045]

[0046] As can be seen from Table 1, the method of the present invention achieves competitive performance in the task of weakly supervised camouflaged target detection.

[0047] The present invention also provides a weakly supervised camouflage target detection system based on a frequency domain method, which includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a weakly supervised camouflage target detection method based on a frequency domain method as described above.

[0048] The present invention also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist alone and not assembled into the electronic device.

[0049] The aforementioned computer-readable medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0050] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0051] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0052] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0053] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A weakly supervised camouflage target detection method based on frequency domain methods, characterized in that, The method includes the following steps: Step 1, processing a given image I and its augmented forms I aug through a backbone network to extract four groups of image features f i at different scales, where i = 1, 2, 3, 4; Step 2, Image Features f i It is fed into the frequency decomposition module for processing to obtain high-frequency features f iH and low-frequency characteristics f iL ; Step 3: Enhance the high-frequency feature f using the feature enhancement module. iH and low-frequency characteristics f iL Enhancement is performed to obtain the enhanced high-frequency feature F. iH and enhanced low-frequency features F iL ; Step 3 is described in detail below: For low-frequency feature f iL Its dimensions are h×w×c, and it is processed through a low-frequency enhancement module: First, the low-frequency feature f iL Divide the image into four equal parts according to the channel, so that the feature size after division becomes h×w×c / 4; then, for each divided image feature region f ic Where c=1,2,3,4, pooling operation is performed; then for each image feature region f ic Perform convolution and upsampling operations to convert each image feature region f ic By restoring the original dimensions and aggregating them together, the enhanced low-frequency feature F in the frequency domain is obtained. iL ; For high-frequency feature f iH The high-frequency enhancement module processes the data: First, a context capturer is used to enhance the high-frequency features f. iH The image is processed to obtain contextual information; then convolutional operations are performed to capture detailed local features; and a weighted layer is used to implement residual connections to further capture global information, thus obtaining the enhanced high-frequency features F in the frequency domain. iH ; Step 4: Use a plug-and-play octave convolution method to learn the enhanced high-frequency features F online. iH and enhanced low-frequency features F iL The target detection results are obtained; Step 4 is described in detail below: A 4×4 octave convolution kernel is divided into four distinct parts for interactions at different frequencies, ultimately generating high- and low-frequency features after inter-frequency communication to help obtain rich contextual information. Subsequently, an inverse Fourier transform is used to map the features back to the RGB domain, and spatial attention and channel attention are calculated in the RGB domain to enhance the camouflage discriminative features. For the output feature map of each layer, a bottom-up weighted fusion mechanism is adopted to effectively fuse the high-frequency and low-frequency features of the current layer with the coarse prediction map of the previous layer. Finally, after multiple fusions, the final optimized prediction result of the model, i.e., the object detection result, is obtained at the highest layer.

2. The weakly supervised camouflage target detection method based on frequency domain method according to claim 1, characterized in that, Step 2 is described in detail below: First, for image features f i Perform a two-dimensional discrete Fourier transform to convert it into frequency domain coordinates F. i (u,v); Then, for F i The spectrum corresponding to (u,v) is shifted, the low frequency is moved to the center position, and the distance from each frequency domain coordinate point to the center is calculated. Finally, a cutoff frequency D0 is set, and the feature F is processed according to the Gaussian filter kernel. i Perform a low-pass filter to reduce the high-frequency characteristics f iH and low-frequency characteristics f iL Separate them.

3. The weakly supervised camouflage target detection method based on frequency domain method according to claim 1, characterized in that, In step 3, the low-frequency feature f is enhanced by the low-frequency enhancement module. iL During processing, adaptive pooling operations with different pooling kernel sizes were used for image feature regions of different channels.

4. The weakly supervised camouflage target detection method based on the frequency domain method according to claim 1, characterized in that, Step 4 further includes: calculating the loss for each layer of the generated coarse prediction map in order to continuously optimize the prediction results.

5. A weakly supervised camouflage target detection system based on frequency domain methods, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a weakly supervised camouflage target detection method based on a frequency domain method as described in any one of claims 1-4.

6. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of a weakly supervised camouflage target detection method based on a frequency domain method as described in any one of claims 1-4.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of a weakly supervised camouflage target detection method based on a frequency domain method as described in any one of claims 1-4.

Images