Weak supervision camouflage target detection method and system based on frequency domain method, medium and program product

Abstract

The invention relates to a frequency domain method-based weak supervision camouflage target detection method and system, a medium and a program product, and the method comprises the steps: firstly decomposing features extracted by a backbone network into high frequency and low frequency, then respectively enhancing each group of features, and carrying out the step-by-step optimization through a coarse-to-fine strategy from bottom to top, thereby effectively highlighting the discrimination features of a camouflage target. Compared with an existing RGB method for weak supervision camouflage target detection, the method has the advantages that camouflage clues are more obviously highlighted in a frequency domain, so that the edge of a camouflage object is more easily perceived, and a better segmentation effect is achieved. In addition, the information captured from the frequency domain can compensate for the limitation of sparse annotation in weak supervised training.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a weakly supervised camouflage target detection method, system, medium and program product based on a frequency domain method. Background Art

[0002] The goal of camouflaged object detection (COD) is to accurately segment objects in a scene that "blend seamlessly" with their environment. These objects often mimic their surroundings or other objects, and the boundary between them and the background is blurred, making it difficult for traditional object detection algorithms to accurately identify them, making COD a challenging task. Despite the challenges, research on camouflaged objects still has a wide range of applications, including species protection, medical image segmentation, industrial defect detection, artistic creation, and agricultural pest monitoring.

[0003] With the rapid development of deep learning, methods dedicated to COD tasks have gradually emerged, driving the research development in this field. However, traditional methods rely on low-level features such as color, optical flow, and texture, and often perform poorly in scenes with complex backgrounds and diverse camouflage strategies. Recent studies have promoted the development of the COD field by introducing innovative model structures and learning strategies. Specifically, Fan et al. proposed SINet, which first searches for camouflaged targets coarsely and then gradually refines the segmentation and creates the most extensive COD10K dataset containing 10,000 images. Mei et al. proposed PFNet to detect and eliminate false positive and false negative predictions. Zhai et al. revealed an interactive graph learning framework that draws inspiration from the intricate interactions of multi-level relationships to achieve the dual goals of detecting camouflaged targets and extracting perceptual edges. Yan et al. developed MirrorNet to promote the development of the COD field. Lv et al. introduced an advanced Rank-Net for simultaneous localization, segmentation, and ranking of camouflaged targets.

[0004] However, COD relies heavily on pixel-level annotations during training, which increases the annotation cost and complexity due to the particularity of its task. It is understood that it takes about 60 minutes to annotate a single image. In this context, weakly supervised camouflaged object detection (WSCOD) came into being, using a small number of annotation points or graffiti on the foreground and background for camouflaged object detection, which significantly reduced the annotation cost and difficulty. He et al. proposed using SAM to generate dense segmentation masks from sparse annotations, and introduced the first SAM-based weakly supervised framework WS-SAM. He et al. constructed a COD dataset using graffiti annotations, marking the birth of the first dataset for the weakly supervised COD task.

[0005] Although WSCOD has made some progress, it still faces many challenges. For example, most of the existing methods for camouflaged object detection process image features in the RGB domain. However, due to the inherent similarity between the camouflaged object and the background and the diversity of camouflage types, many discriminative camouflage clues are easily ignored in the RGB domain, resulting in unsatisfactory edge segmentation of camouflaged objects by existing detectors. In addition, the current weakly supervised camouflaged object detection methods are difficult to fully utilize graffiti annotation information during the training process, resulting in incomplete predictions in the prediction results. Summary of the invention

[0006] In view of the problems existing in the prior art, the purpose of the present invention is to provide a weakly supervised camouflaged target detection method based on a frequency domain method, which more significantly highlights the camouflage clues in the frequency domain, making the edges of camouflaged objects easier to detect and having a better segmentation effect.

[0007] To achieve the above object, the technical solution adopted by the present invention is: A weakly supervised camouflaged target detection method based on a frequency domain method, the method comprising the following steps: Step 1: Use the backbone network ResNet-50 to train the given image I and its enhanced form I aug Processing is performed to extract four groups of image features f at different scales i (i = 1, 2, 3, 4); Step 2: Image feature f i is fed into the frequency decomposition module for processing to obtain the high-frequency features f iH and low frequency features f iL ; Step 3: Use the feature enhancement module (FEM) to respectively iH and low frequency features f iL Enhance to obtain enhanced high-frequency features F iH and the enhanced low-frequency feature F iL ; Step 4: Use the plug-and-play octave convolution method to learn strong high-frequency features F online iH and the enhanced low-frequency feature F iL , and get the target detection result.

[0008] The step 2 is specifically as follows: First, the image feature f i Perform a two-dimensional discrete Fourier transform and convert it into frequency domain coordinates F i (u,v); Then, for F iThe spectrum corresponding to (u,v) is translated to move the low frequency to the center, and the distance from each frequency domain coordinate point to the center is calculated; Finally, a cutoff frequency D0 is set and the feature F is filtered according to the Gaussian filter kernel. i Perform a low-pass filter to remove the high-frequency features f iH and low frequency features f iL To separate.

[0009] The step 3 is as follows: For the low frequency feature f iL , whose size is h×w×c, is processed by the low-frequency enhancement module: First, the low-frequency feature f iL Divide the image into four equal parts according to the channel, that is, the feature size after division becomes h×w×c / 4; then, for each divided image feature region f ic (c=1,2,3,4), perform pooling operation; then for each image feature area f ic Perform convolution and upsampling operations to convert each image feature area f ic Restore to the original size and aggregate together to obtain the enhanced low-frequency features F in the frequency domain iL ; For the low frequency feature f iH , processed by the high-frequency enhancement module: First, the high-frequency feature f is iH Processing is performed to obtain contextual information; then a convolution operation is performed to capture local detailed features in the image; a residual connection is implemented through a weight layer to further capture global information, thereby obtaining the enhanced high-frequency features F in the frequency domain iH .

[0010] In step 3, the low-frequency feature f is enhanced by the low-frequency enhancement module. iL During processing, adaptive pooling operations with pooling kernels of different sizes are used for image feature areas of different channels.

[0011] The step 4 is specifically as follows: A 4×4 octave convolution kernel is divided into four different parts for interactions at different frequencies, ultimately generating high- and low-frequency features after inter-frequency communication to help obtain rich contextual information; the inverse Fourier transform is then used to map the features back to the RGB domain, and in the RGB domain, spatial attention and channel attention are calculated to enhance the discriminative features of camouflage; for the output feature map of each layer, a bottom-up weighted fusion mechanism is used to effectively fuse the high-frequency and low-frequency features of the current layer with the rough prediction map of the previous layer; finally, after multiple fusions, the final optimized prediction result of the model, i.e., the target detection result, is obtained at the highest layer.

[0012] The step 4 also includes: calculating the loss for the rough prediction graph generated for each layer to continuously optimize the prediction result.

[0013] A weakly supervised camouflaged target detection system based on a frequency domain method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the weakly supervised camouflaged target detection method based on a frequency domain method as described above. A computer-readable storage medium stores a computer program / instruction thereon, and when the computer program / instruction is executed by a processor, the steps of a weakly supervised camouflaged target detection method based on a frequency domain method as described above are implemented. A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of a weakly supervised camouflaged target detection method based on a frequency domain method as described above. After adopting the above scheme, the present invention first decomposes the features extracted by the backbone network into high frequency and low frequency, and then enhances each group of features respectively, and gradually optimizes through a bottom-up coarse-to-fine strategy to effectively highlight the discriminative features of the camouflaged target. Compared with the existing RGB method for weakly supervised camouflaged target detection, the present invention more significantly highlights the camouflage clues in the frequency domain, making the edges of the camouflaged objects easier to detect and having a better segmentation effect. In addition, the information captured from the frequency domain can make up for the limitations of sparse annotations in weakly supervised training. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION

[0015] like Figure 1 As shown, the present invention discloses a weakly supervised camouflaged target detection method based on a frequency domain method, which comprises the following steps: Step 1: Use the backbone network ResNet-50 to train the given image I and its enhanced form I aug Processing is performed to extract four groups of image features f at different scales i (i = 1, 2, 3, 4).

[0016] Step 2: Image feature f i is fed into the frequency decomposition module for processing to obtain the high-frequency features f iH and low frequency features f iL .

[0017] For the image feature f i Each pixel in the space domain has coordinates (x, y), f i(x, y) is the image signal value in the two-dimensional space domain. First, the image feature f i Perform a two-dimensional discrete Fourier transform and convert it into frequency domain coordinates F i (u,v). Considering the feature F obtained by Fourier transform at this time i The image contains information of different frequencies, and the high-frequency and low-frequency information needs to be separated to enhance the contour and details of the image respectively. Gaussian filter is a commonly used two-dimensional linear low-pass filter that can be used to effectively remove high-frequency noise. At the same time, due to the smoothing effect of the Gaussian filter, the ringing effect of the image after inverse Fourier transform can be avoided. The Gaussian filter kernel is used to perform weighted averaging of the neighboring pixels of each pixel in the image to achieve a smoothing effect. Specifically, F i The spectrum corresponding to (u, v) is translated to move the low frequency to the center, and then the distance from each frequency domain coordinate point to the center is calculated. Then a cutoff frequency D0 is set, and the feature F is filtered according to the Gaussian filter kernel. i Perform a low-pass filter to remove the high-frequency features f iH and low frequency features f iL To separate.

[0018] Step 3: Use the feature enhancement module (FEM) to respectively iH and low frequency features f iL Enhance to obtain enhanced high-frequency features F iH and the enhanced low-frequency feature F iL , to highlight camouflage cues in the frequency domain.

[0019] For the low frequency feature f iL , whose size is h×w×c, is processed by the low-frequency enhancement module (LFEM). Specifically, first, the low-frequency feature f iL Divide the image into 4 equal parts according to the channels, that is, the feature size after division becomes h×w×c / 4. Then, for each divided image feature region f ic (c=1,2,3,4), perform pooling operation to capture and enhance low-frequency features in the image. Pooling operation summarizes or aggregates each image feature area to better preserve the low-frequency information in the feature map and remove some noise. In addition, considering that different feature channels contain different information, their frequency separation thresholds are naturally different. For image feature areas f of different channels ic Adaptive pooling with different pooling kernel sizes is used. Compared with fixed-scale pooling, this design can better reduce the loss of spatial information in the image and preserve the outline of the target. In this way, low-frequency features are enhanced in the channel dimension. After that, for each image feature region f icPerform convolution and upsampling operations to convert each image feature area f ic Restore to the original size and aggregate together to obtain the enhanced low-frequency features F in the frequency domain iL .

[0020] The high-frequency components of an image cover more complex details, especially in areas that change rapidly or fluctuate frequently, such as the edges of objects. In the design of the High-Frequency Enhancement Module (HFEM), residual connections are applied to the acquired high-frequency features. This effectively captures the local details and long-range dependencies of the target, helping to emphasize the high-frequency details of the image. Specifically, the high-frequency feature f is firstly iH The image is processed to obtain contextual information, and then a convolution operation is performed to capture local detailed features in the image, such as edges and textures. In addition, a residual connection is implemented through a weight layer to further capture global information. Based on these operations, the enhanced high-frequency features F in the frequency domain are obtained. iH .

[0021] Step 4: Use the plug-and-play octave convolution method to learn strong high-frequency features F online iH and the enhanced low-frequency feature F iL , and get the target detection result.

[0022] A 4×4 octave convolution kernel is divided into four different parts for interactions at different frequencies, ultimately generating high- and low-frequency features after inter-frequency communication to help obtain rich contextual information. The inverse Fourier transform is then used to map the features back to the RGB domain, and in the RGB domain, spatial attention and channel attention are calculated to strengthen the camouflage discriminative features. For the output feature map of each layer, a bottom-up weighted fusion mechanism is used to effectively fuse the high-frequency and low-frequency features of the current layer with the rough prediction map of the previous layer. Finally, after multiple fusions, the final optimized prediction result of the model, that is, the target detection result, will be obtained at the highest layer.

[0023] In addition, the loss is calculated for the coarse prediction map generated by each layer to continuously optimize the prediction results.

[0024] The present invention can make full use of sparse graffiti annotation information, and compared with the current weakly supervised camouflage target detection method that processes images in the RGB domain, it is more sensitive to the discriminative clues of camouflaged objects, especially the edges, and has a better edge segmentation effect.

[0025] To illustrate the effect achieved by the present invention, the present invention is compared with some current methods with better performance. As shown in Table 1, the method of the present invention is compared with the current methods with better performance (including salient target detection, camouflaged target detection, weakly supervised camouflaged target detection and weakly supervised salient target detection).

[0026] In Table 1, CAMO, COD10K, and NC4K are all commonly used datasets for camouflaged target detection, containing pictures of camouflaged targets.

[0027] MAE is Mean Absolute Error, which is the average of the absolute values ​​of the difference between the predicted value and the true value; m It is the Structure Measure structural metric, which is used to evaluate the structural similarity between the predicted segmentation result and the true segmentation; E m It is the enhanced-alignment measure, which is also used to evaluate the structural similarity between the segmentation result and the true segmentation, but it also considers the global average of the image and local pixel matching; It is the weighted F-measure, which is the weighted average of precision and recall, where precision measures the number of correctly classified pixels in the segmentation result, while recall measures the number of correctly classified pixels in all true positive examples. It evaluates the overall performance of the segmentation result by balancing precision and recall.

[0028] CVPR is the abbreviation of IEEE Conference on Computer Vision and Pattern Recognition; ICCV is the IEEE International Conference on Computer Vision; AAAI is the Association for the Advance of Artificial Intelligence; ECCV is the European Conference on Computer Vision; LJCAI is the International Joint Conferences on Artificial Intelligence; TPAMI is the IEEE Transactions on Pattern Analysis and Machine Intelligence.

[0029] Table 1: Comparison between the present invention and the existing method

[0030] It can be seen from Table 1 that the method of the present invention achieves competitive performance in the weakly supervised camouflaged target detection task.

[0031] The present invention also provides a weakly supervised camouflaged target detection system based on a frequency domain method, which includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of a weakly supervised camouflaged target detection method based on a frequency domain method as described above. The present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments. It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied. Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure. Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A weakly supervised camouflaged target detection method based on frequency domain method, characterized in that: The method comprises the following steps: Step 1: Use the backbone network to analyze the given image I and its enhanced form I aug Processing is performed to extract four groups of image features f at different scales i , where i = 1, 2, 3, 4; Step 2: Image feature f i is fed into the frequency decomposition module for processing to obtain the high-frequency features f iH and low frequency features f iL ; Step 3: Use the feature enhancement module to enhance the high-frequency features f iH and low frequency features f iL Enhance to obtain enhanced high-frequency features F iH and the enhanced low-frequency feature F iL ; Step 4: Use the plug-and-play octave convolution method to learn strong high-frequency features F online iH and the enhanced low-frequency feature F iL , and get the target detection result.

2. According to the weakly supervised camouflaged target detection method based on the frequency domain method in claim 1, it is characterized in that: The step 2 is specifically as follows: First, the image feature f i Perform a two-dimensional discrete Fourier transform and convert it into frequency domain coordinates F i (u,v); Then, for F i The spectrum corresponding to (u,v) is translated to move the low frequency to the center, and the distance from each frequency domain coordinate point to the center is calculated; Finally, a cutoff frequency D0 is set and the feature F is filtered according to the Gaussian filter kernel. i Perform a low-pass filter to remove the high-frequency features f iH and low frequency features f iL To separate.

3. According to the weakly supervised camouflaged target detection method based on the frequency domain method of claim 1, it is characterized in that: The step 3 is as follows: For the low frequency feature f iL , whose size is h×w×c, is processed by the low-frequency enhancement module: First, the low-frequency feature f iL Divide the image into four equal parts according to the channel, that is, the feature size after division becomes h×w×c / 4; then, for each divided image feature region f ic , where c=1,2,3,4, and perform pooling operation; then for each image feature area f ic Perform convolution and upsampling operations to convert each image feature area f ic Restore to the original size and aggregate together to obtain the enhanced low-frequency features F in the frequency domain iL ; For the low frequency feature f iH , processed by the high-frequency enhancement module: First, the high-frequency feature f is iH Processing is performed to obtain contextual information; then a convolution operation is performed to capture local detailed features in the image; a residual connection is implemented through a weight layer to further capture global information, thereby obtaining the enhanced high-frequency features F in the frequency domain iH .

4. According to the weakly supervised camouflaged target detection method based on the frequency domain method of claim 3, it is characterized in that: In step 3, the low-frequency feature f is enhanced by the low-frequency enhancement module. iL During processing, adaptive pooling operations with pooling kernels of different sizes are used for image feature areas of different channels.

5. The weakly supervised camouflaged target detection method based on the frequency domain method according to claim 1 is characterized in that: The step 4 is specifically as follows: A 4×4 octave convolution kernel is divided into four different parts for interactions at different frequencies, ultimately generating high and low frequency features after inter-frequency communication to help obtain rich contextual information; The inverse Fourier transform is then used to map the features back to the RGB domain, and in the RGB domain, spatial attention and channel attention are calculated to enhance the camouflage discriminative features; For the output feature map of each layer, a bottom-up weighted fusion mechanism is used to effectively fuse the high-frequency and low-frequency features of the current layer with the rough prediction map of the previous layer; finally, after multiple fusions, the final optimized prediction result of the model, i.e., the target detection result, will be obtained at the highest layer.

6. The method for detecting a weakly supervised camouflaged target based on a frequency domain method according to claim 5, characterized in that: The step 4 also includes: calculating the loss for the rough prediction graph generated for each layer to continuously optimize the prediction result.

7. A weakly supervised camouflaged target detection system based on frequency domain method, characterized in that: It includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of a weakly supervised camouflaged target detection method based on a frequency domain method as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by the processor, the steps of a weakly supervised camouflaged target detection method based on a frequency domain method as described in any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by the processor, the steps of a weakly supervised camouflaged target detection method based on a frequency domain method as described in any one of claims 1 to 6 are implemented.

Images