Camouflage target detection system and method based on multi-scale and context feature learning

Abstract

The invention discloses a camouflage target detection system and method based on multi-scale and context feature learning. The system comprises a feature extraction trunk, a frequency enhancement dense interaction decoder, a spatial frequency attention module and a frequency sensing multi-scale fusion module. The method comprises the following steps: firstly, extracting bottom layer features of a camouflage target image to obtain a feature map; then high-frequency and low-frequency features are separated, a new feature map based on a frequency domain is obtained, spatial features and channel features of the new features are extracted, and a coarse positioning map is obtained; secondly, entanglement learning of spatial features and frequency domain features is carried out on the obtained feature map to obtain a feature map with richer feature information; and finally, fusing the coarse positioning map with the features of different scales to generate a camouflage target detection result. According to the method, the edge information of the camouflage target can be effectively extracted, misjudgment of an algorithm on a non-target area is inhibited, and the effect and precision of camouflage target detection are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a camouflage target detection system and method based on multi-scale and context feature learning. BACKGROUND

[0002] The camouflage target is usually extremely similar to the surrounding environment in color, texture and shape, etc., and it is extremely difficult to separate the camouflage target from the background, especially in the case of low illumination, occlusion, small object size and high background similarity. Therefore, when the target in the camouflage scene is detected, the edge positioning is inaccurate, the prediction result is fuzzy, and the target is easily disturbed by the non-target area, which leads to problems such as overlarge prediction area, limited robustness, poor prediction result and the like, thereby affecting the accuracy and detection precision of target detection.

[0003] Camouflaged object detection is of great significance for medical image judgment and information acquisition. Traditional camouflaged object detection algorithms are mostly based on 3D convexity, color, edge and texture of camouflaged object images, but due to the high similarity between camouflaged objects and backgrounds, the effect is usually not good. In recent years, with the rapid development of deep learning technology in the field of image, the research on using deep learning technology to detect camouflaged objects in images has also increased. For example, by using the powerful feature representation ability of deep convolutional neural network, the features of camouflaged objects in different scenes are learned, and they are separated from the background to realize object detection. However, the existing camouflaged object detection methods are mostly based on spatial domain to extract features of camouflaged object images, which cannot effectively utilize the channel feature information and local context information, lacks information interaction between channels, and ignores the effectiveness of frequency domain feature extraction for camouflaged objects, resulting in unclear edges of detected objects and easy misjudgment of non-background areas as background areas. For example, document 1 (MSCAF-Net: A General Framework for Camouflaged Object Detection via Learning Multi-Scale Context-Aware Features) proposes a camouflaged object detection algorithm using context information, but it ignores the frequency domain information of camouflaged objects, resulting in inaccurate detection image edges and easy misjudgment of background areas. Document 2 (Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection) proposes a zooming strategy to obtain features at different levels, but it fails to combine multi-scale semantic information and fully utilize context information, resulting in inaccurate feature extraction and unclear edges in the detection results. Chinese patent CN118298282A discloses a camouflaged object detection algorithm based on edge refinement and enhancement network, which enhances the extraction ability of edge features, combines context information fusion features, and improves the accuracy of camouflaged object detection, but this method still does not consider the role of frequency domain features in camouflaged object detection. Chinese patent CN117475134A discloses a camouflaged object detection algorithm based on multi-scale cross-layer feature fusion network, which locates the edge features of camouflaged objects through an edge guiding module, cross-layer fuses feature information through a multi-scale feature aggregation module, further enhances the structural information in high-resolution feature maps through a multi-element information fusion module, and distinguishes the edges of camouflaged objects from the background while using element subtraction to invert the background features and using element addition to enhance the foreground features, but this method also does not consider the feature information embodied by frequency domain features. SUMMARY

[0004] The present application aims to provide a camouflage target detection system and method based on multi-scale and context feature learning, which has strong edge detection capability and low misjudgment rate, so as to improve the target detection accuracy and efficiency.

[0005] The technical solution for achieving the object of the present application is:

[0006] Compared with the prior art, the present application has the following advantages: (1) effectively improves the edge detection accuracy of the camouflage target, and improves the detection accuracy in low light environment, occlusion, small size target and high background similarity; (2) effectively utilizes the frequency domain feature information and context information of the camouflage target in the image, and solves the problem of misjudgment of the background area as the target area after detection; (3) by integrating spatial attention and frequency attention mechanism in the attention component part, the spatial features and frequency features of the camouflage target are fully combined, the fusion of spatial features and frequency domain features is realized, and the detection effect of the edge of the camouflage target is refined; (4) combined with frequency domain enhancement and multi-scale feature fusion, the utilization of context information is deepened, the extraction of camouflage target features is strengthened, and the accuracy of target detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is a flowchart of the camouflage target detection method based on multi-scale and context feature learning of the present application.

[0008] Figure 2 is a hardware structure block diagram of the camouflage target detection system in the embodiment of the present application.

[0009] Figure 3 is a result comparison chart before and after using the method of the present application for camouflage target detection in the embodiment of the present application. DETAILED DESCRIPTION

[0010] As shown in Figure 1 The present application provides a camouflage target detection system based on multi-scale and context feature learning, which comprises a feature extraction backbone, a frequency enhancement dense interactive decoder, a spatial frequency attention module and a frequency perception multi-scale fusion module, wherein:

[0011] The feature extraction backbone uses a pyramid structure feature extraction backbone to extract the bottom layer features of the camouflage target image, and obtains a feature map;

[0012] The frequency enhancement dense interactive decoder comprises four frequency perception modules, one dense interactive decoder and one spatial and channel collaborative attention module, generates frequency domain features through the frequency perception module, transmits the frequency domain features fused in different stages into the dense interactive decoder, and forms a coarse positioning map of the camouflage target after the spatial and channel collaborative attention module.

[0013] The spatial frequency attention module includes a frequency attention module, a spatial attention module and a feedforward network, which captures long-range information, enhances feature extraction in the spatial and frequency domains, enhances the perception of the network to the camouflage target area, and strengthens the extraction of edge features.

[0014] The frequency perception multi-scale fusion module includes a multi-scale feature fusion branch and a frequency domain enhancement branch, which are used to fuse spatial and frequency domain features obtained at different scales, refine the obtained feature map through the frequency domain enhancement branch, and use 3x3 convolution to generate a final prediction image.

[0015] As a specific example, the frequency perception module in the frequency enhancement dense interaction decoder, the dense interaction decoder and the spatial and channel collaborative attention module are as follows:

[0016] The frequency perception module is an octave convolution including an average pooling operation, which is used to divide features into high-frequency feature maps and low-frequency feature maps, and obtain global information.

[0017] The dense interaction decoder includes a 1x1 convolution layer, a 3x3 convolution layer and an up-sampling layer, which combines frequency features at different scales to form a coarse positioning map.

[0018] The spatial and channel collaborative attention module includes a spatial attention branch and a channel attention branch. The spatial attention branch extracts spatial information at different semantic levels through a multi-scale deep convolution layer. The channel attention branch uses a self-attention mechanism to refine the semantic understanding of local sub-features using the overall feature map, and reduces the semantic difference caused by multi-scale convolution in the spatial attention branch.

[0019] As a specific example, the calculation process of the octave convolution is as follows:

[0020] ;

[0021] ;

[0022] wherein, represents the i-th feature map, represents the high-frequency component of the output feature map, represents the low-frequency component of the output feature map, is the high-frequency component of the input image, is the low-frequency component of the input image;

[0023] is the input, and the learnable parameter is convolution operation, is the weight of the convolution kernel, ​denote the convolution weight from the high frequency component to the high frequency component, denote the convolution weight from the low frequency component to the low frequency component, denote the convolution weight from the low frequency component to the high frequency component, denote the convolution weight from the high frequency component to the low frequency component; denote the convolution operation with the input and the learnable parameter ; denote the convolution operation with the input and the learnable parameter ; denote the convolution operation with the input and the learnable parameter ;

[0024] denote the average pooling operation with the input and the convolution kernel , denote the average pooling operation with the input and the convolution kernel ; denote the convolution operation with the input and the learnable parameter ;

[0025] denote the up-sampling operation with the input and the nearest neighbor interpolation coefficient , denote the up-sampling operation with the input and the nearest neighbor interpolation coefficient .

[0026] As a specific example, the spatial frequency attention module includes a frequency attention module, a spatial attention module and a feedforward network, and specifically as follows:

[0027] The frequency attention module is used for a fast Fourier transform operation, an inverse fast Fourier transform operation and a feature fusion operation, the fast Fourier transform operation is used for extracting frequency features of the features, and the inverse fast Fourier transform operation is used for supplementing low frequency information;

[0028] The spatial attention module includes a 3x3 deep convolution layer, a 5x5 deep convolution layer and a feature fusion module;

[0029] The feedforward network includes a 3×3 deep convolution branch and a fast Fourier transform branch to achieve entanglement learning of spatial and frequency domain features. The 3×3 deep convolution branch is used to extract new features after the fusion of spatial and frequency features, and the fast Fourier transform branch is used to enhance the frequency domain part of the new features.

[0030] As a specific example, in the frequency-aware multi-scale fusion module, the calculation process of the multi-scale feature fusion branch is as follows:

[0031] ;

[0032] in, This is a characteristic of the current stage. This is a characteristic of the next stage. For normalized convolution kernels with a scale of Convolution operation, and From a rough location map The generated scale and offset parameters, , These represent element addition and multiplication, respectively.

[0033] As a specific example, in the frequency-aware multi-scale fusion module, the calculation process of the frequency enhancement branch is as follows:

[0034]

[0035]

[0036] in, Indicates to , Perform frequency convolution operation. For inverse fast Fourier transform operation, This is a normalization operation.

[0037] This invention also provides a camouflage target detection method based on multi-scale and contextual feature learning. This method is based on the aforementioned camouflage target detection system based on multi-scale and contextual feature learning, and includes the following steps:

[0038] Step 1: Use the Feature Pyramid Transformer to extract the low-level features of the camouflaged target image in the feature extraction backbone to obtain the feature map;

[0039] Step 2: The extracted feature map is passed through four frequency sensing modules in the frequency enhancement dense interactive decoder to separate high-frequency and low-frequency features and obtain new frequency domain-based feature maps. The new feature maps are then passed into the dense interactive decoder to achieve feature information fusion at different scales. The fused features are then passed into the spatial and channel collaborative attention module to fully extract the spatial and channel features of the new features, resulting in a coarse localization map, which is used to modulate the frequency sensing multi-scale fusion module.

[0040] Step 3: Input the feature map obtained in Step 1 into 4 spatial frequency attention modules. By performing entanglement learning on features at different scales in the spatial and frequency domains, feature maps with richer feature information can be obtained.

[0041] Step 4: Input the feature map obtained in Step 3 into the frequency-aware multi-scale fusion module. By fusing features at different levels, the extraction of frequency domain features is enhanced, and the final predicted image is generated.

[0042] The present invention also provides a mobile terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the camouflage target detection method based on multi-scale and contextual feature learning.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the camouflage target detection method based on multi-scale and contextual feature learning.

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0045] Example

[0046] This embodiment uses a computer equipped with an NVIDIA RTX 3090 GPU as the image processor, and collects corresponding camouflage target data through cameras and drones. The system hardware structure is as follows: Figure 2 As shown.

[0047] The method of this invention is used to detect camouflaged target images captured by cameras, etc., and to save the processed image visualization results. Figure 3 The results show the comparison of the detection of camouflaged target images collected under different camouflage scenarios before and after detection. The first column is the original image, the second column is the true value, the third column is the method proposed in this invention, and the fourth to seventh columns are the detection results of four other algorithms. This is a structural metric in the field of camouflaged target detection, used to evaluate the structural similarity between predicted segmentation results and ground truth segmentation. It considers factors such as edge connectivity, region integrity, and boundary offset of the segmentation results; the closer the value is to 1, the more similar the segmentation result is to the ground truth segmentation structure. To enhance similarity in the field of camouflaged target detection, this measure the consistency between the predicted image and the real image at the pixel and image levels. The closer the value is to 1, the more similar the segmentation result is to the real image. It is calculated by increasing the weight of different regions and comprehensively considering the harmonic mean of precision and recall; the closer the value is to 1, the better the detection effect. MAE is the mean absolute error, which represents the difference between the predicted result and the true label at the pixel level; the closer the value is to 0, the better the effect. Table 1 shows the comparison results of using the method of this invention for camouflaged target detection with other algorithms before and after.

[0048] Table 1. Comparison of camouflaged target detection results using the method of this invention with other algorithms.

[0049]

[0050] As can be seen from the data listed in Table 1, when using the method of the present invention to detect camouflaged targets, compared with other methods that have better or more advanced effects, the method of the present invention requires fewer parameters and the detection results are better than other algorithms. It is evident that the method of the present invention can effectively improve the edge detection accuracy of camouflaged targets, improve the detection accuracy in low light environment, occlusion, small-sized targets and high background similarity, and improve the accuracy of camouflaged target detection.

[0051] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A camouflaged target detection system based on multi-scale and contextual feature learning, characterized in that, It includes a feature extraction backbone, a frequency-enhanced dense interactive decoder, a spatial frequency attention module, and a frequency-aware multi-scale fusion module, among which: The feature extraction backbone uses a pyramid structure to extract low-level features from the camouflaged target image to obtain a feature map. The frequency-enhanced dense interactive decoder includes four frequency sensing modules, one dense interactive decoder, and one spatial and channel collaborative attention module. The frequency sensing modules generate frequency domain features, and the frequency domain features fused at different stages are fed into the dense interactive decoder. After passing through the spatial and channel collaborative attention module, a coarse localization map of the camouflaged target is formed. The spatial frequency attention module includes a frequency attention module, a spatial attention module, and a feedforward network. By capturing long-range information, it enhances feature extraction in the spatial and frequency domains, improves the network's perception of camouflaged target areas, and strengthens the extraction of edge features. The frequency-aware multi-scale fusion module includes a multi-scale feature fusion branch and a frequency domain enhancement branch, which are used to fuse spatial frequency domain features obtained at different scales, refine the obtained feature map through the frequency domain enhancement branch, and use 3×3 convolution to generate the final predicted image.

2. The camouflaged target detection system based on multi-scale and contextual feature learning according to claim 1, characterized in that, The frequency-aware module, dense interaction decoder, and spatial and channel collaborative attention module in the frequency-enhanced dense interaction decoder are as follows: The frequency sensing module is an octave convolution including average pooling, used to divide features into high-frequency feature maps and low-frequency feature maps, and obtain global information; The dense interactive decoder includes a 1×1 convolutional layer, a 3×3 convolutional layer, and an upsampling layer, which combine frequency features at different scales to form a coarse localization map. The spatial and channel collaborative attention module includes a spatial attention branch and a channel attention branch. The spatial attention branch extracts spatial information at different semantic levels through multi-scale deep convolutional layers. The channel attention branch uses a self-attention mechanism to refine the semantic understanding of local sub-features using the overall feature map, thereby mitigating the semantic differences caused by multi-scale convolution in the spatial attention branch.

3. The camouflaged target detection system based on multi-scale and contextual feature learning according to claim 2, characterized in that, The calculation process for the octave band convolution is as follows: ； ； in, Indicates the first Each feature map This represents the high-frequency components of the output feature map. This represents the low-frequency components of the output feature map. For the high-frequency components of the input image, The low-frequency components of the input image; For input Learnable parameters are Convolution operation, The weights of the convolution kernel, This represents the convolution weights from high-frequency components to high-frequency components. This represents the convolution weights from low-frequency components to low-frequency components. This represents the convolution weights from low-frequency components to high-frequency components. This represents the convolution weights from high-frequency components to low-frequency components; Indicates that the input is Learnable parameters are Convolution operations; Indicates that the input is Learnable parameters are Convolution operations; Indicates that the input is Learnable parameters are Convolution operations; Indicates that the input is Convolution kernel is Average pooling operation, Indicates that the input is Convolution kernel is Average pooling operation; Indicates that the input is Learnable parameters are Convolution operations; Indicates that the input is The nearest neighbor interpolation coefficients are upsampling operation, Indicates that the input is Nearest neighbor interpolation coefficients Upsampling operation.

4. The camouflaged target detection system based on multi-scale and contextual feature learning according to claim 3, characterized in that, The spatial frequency attention module includes a frequency attention module, a spatial attention module, and a feedforward network, as detailed below: The frequency attention module is used for fast Fourier transform (FST) operations, inverse fast Fourier transform (IFT) operations, and feature fusion operations. The FFT operation is used to extract the frequency features of the features, and the IFT operation is used to supplement low-frequency information. The spatial attention module includes a 3×3 deep convolutional layer, a 5×5 deep convolutional layer, and a feature fusion module. The feedforward network includes a 3×3 deep convolution branch and a fast Fourier transform branch to achieve entanglement learning of spatial and frequency domain features. The 3×3 deep convolution branch is used to extract new features after the fusion of spatial and frequency features, and the fast Fourier transform branch is used to enhance the frequency domain part of the new features.

5. The camouflaged target detection system based on multi-scale and contextual feature learning according to claim 4, characterized in that, In the frequency-aware multi-scale fusion module, the calculation process of the multi-scale feature fusion branch is as follows: ； in, This is a characteristic of the current stage. This is a characteristic of the next stage. For normalized convolution kernels with a scale of Convolution operation, and From a rough location map The generated scale and offset parameters, , These represent element addition and multiplication, respectively.

6. The camouflaged target detection system based on multi-scale and contextual feature learning according to claim 5, characterized in that, In the frequency-aware multi-scale fusion module, the calculation process of the frequency enhancement branch is as follows: ； ； in, Indicates to , Perform frequency convolution operation. For inverse fast Fourier transform operation, This is a normalization operation.

7. A method for detecting camouflaged targets based on multi-scale and contextual feature learning, characterized in that, This method is based on the camouflaged target detection system based on multi-scale and contextual feature learning as described in any one of claims 1 to 6, and the method includes the following steps: Step 1: Use the Feature Pyramid Transformer to extract the low-level features of the camouflaged target image in the feature extraction backbone to obtain the feature map; Step 2: The extracted feature map is passed through four frequency sensing modules in the frequency enhancement dense interactive decoder to separate high-frequency and low-frequency features and obtain new frequency domain-based feature maps. The new feature maps are then passed into the dense interactive decoder to achieve feature information fusion at different scales. The fused features are then passed into the spatial and channel collaborative attention module to fully extract the spatial and channel features of the new features, resulting in a coarse localization map, which is used to modulate the frequency sensing multi-scale fusion module. Step 3: Input the feature map obtained in Step 1 into 4 spatial frequency attention modules. By performing entanglement learning on features at different scales in the spatial and frequency domains, feature maps with richer feature information can be obtained. Step 4: Input the feature map obtained in Step 3 into the frequency-aware multi-scale fusion module. By fusing features at different levels, the extraction of frequency domain features is enhanced, and the final predicted image is generated.

8. A mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the camouflage target detection method based on multi-scale and contextual feature learning as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the camouflage target detection method based on multi-scale and contextual feature learning as described in claim 7.

Images