Camouflage object detection method with structural adaptability

Abstract

The invention discloses a camouflage object detection method with structural adaptability. The method comprises the following steps: supplementing structural information from point to surface; selectively supplementing high-resolution information in a later stage of the model so as to enhance positioning and fine edge segmentation of small objects in a complex environment; specifically, the method realizes a camouflage object detection task in three stages: a feature extraction stage, a coarse-grained camouflage target positioning stage and a final fine-grained optimization stage. In the first stage, features are extracted and fused to ensure that basic features are sufficient to competent for tasks; in the second stage, a key point enhancement module is adopted, and key positions are concerned in the early stage of model processing; in order to supplement high-resolution information, the invention provides mixed resolution attention, and a low-level high-resolution feature is utilized to enhance a high-level feature which is full of semantic information but insufficient in resolution information.

Description

Technical Field

[0001] The invention belongs to the technical field of target detection, and in particular relates to a camouflaged object detection method with structural adaptability. Background Art

[0002] Camouflaged object detection has long been a challenging task, which aims to find hidden, camouflaged, or difficult-to-detect objects in images. Camouflaged objects are highly integrated with the background environment. Common camouflaged objects include mimicry, military camouflage uniforms, and camouflaged tanks.

[0003] Early camouflaged object detection methods mainly relied on manually annotated features. With the development of deep learning and the introduction of large-scale data sets, the implementation methods of this task have made rapid progress. At present, camouflaged object detection methods based on deep learning can be roughly divided into two categories: bionic design and auxiliary clues. For the former, it is often to imitate the working mechanism of the human eye to scale the image at multiple scales or focus on certain specific locations. For the latter, it is to find one or more auxiliary clues to assist the model and focus its attention on the foreground. Common auxiliary clues include boundaries, frequency, depth, and supplementary high-resolution information.

[0004] Most of the existing auxiliary clue methods rely on the structural features of the foreground camouflage object. However, due to the complex environment of the camouflage object, the structural information of the background object in the picture may be much stronger than the foreground. Therefore, the auxiliary clue methods that rely on structural information will be disturbed and misled, especially the methods that supplement high-resolution information may supplement a large amount of high-resolution noise.

[0005] There are several common difficulties in camouflaged object detection: 1) In complex environments, background objects usually have stronger structural cues than foregrounds, resulting in significant noise in structural information. 2) For multiple targets and detailed camouflaged objects, it is very difficult to extract large, complex and scattered structural information. 3) When the target is rich in detail information and the foreground is similar to the background, fine segmentation becomes difficult. 4) For extremely difficult to distinguish camouflaged objects, the foreground and background show highly similar structures, which may lead to the model's misjudgment and mistake the background for the foreground. 5) In an environment with a complex background environment and small camouflaged targets, the structural information of the foreground is significantly weaker than that of the background, which makes positioning and boundary recognition difficult. Human visual discrimination is also challenged, and methods that simulate human vision may not be competent when dealing with small targets. Improper handling of structural information may have two common effects on the model: misleading and interference. Using inaccurate information as a priori or auxiliary cues can be very dangerous because it may mislead the model's decision and cause the model to mistake structurally similar or prominent background targets for the foreground. In addition, high-resolution information is essential for the segmentation and detail optimization of small targets. However, over-supplementing high-resolution information may introduce high-resolution noise, weaken the confidence of the model, and lead to unstable prediction results. Summary of the invention

[0006] In order to overcome the above problems, the present invention provides a camouflaged object detection method with structural adaptability.

[0007] The technical solution adopted by the present invention is:

[0008] A structurally adaptive camouflaged object detection method adaptively adjusts the learning strategy according to the shape of the camouflaged object, avoids background interference and pays more attention to foreground key information. The method includes the following stages:

[0009] The first stage: feature extraction stage:

[0010] SwinTransformer is used to extract multi-scale features from the input image, and then the feature division module is used to divide and fuse the features into low-level features X 1 and X 2 , intermediate feature X 3 and high-level features X 4 ;

[0011] The second stage: coarse-grained camouflage object positioning stage:

[0012] Different enhancement strategies are applied to the three levels and four features obtained; 1 and X 2Use the key point enhancement module and mixed resolution attention mechanism to enhance features; for high-level features X 4 Use spatial attention for enhancement; the processed X 1 , X 2 , X 4 With the unprocessed intermediate feature X 3 Decode using the decoder, and use the spatial channel feature fusion module to perform feature fusion to obtain a rough positioning map M;

[0013] The third stage: fine-grained optimization stage:

[0014] Using the obtained rough positioning map M, the patch containing the foreground target is selected from the high-resolution image using the structure adaptation patch method, and is spliced ​​along the channel dimension to the output X of the spatial channel feature fusion module. out Then, a new round of feature fusion is performed. After two repeated operations, the final fine segmentation map M1 is obtained.

[0015] Among them, the feature division method in the feature extraction stage is: in order to retain the high-resolution characteristics of low-level features and the rich semantic information of high-level features, the low-level features X extracted by the backbone network are 11 and X 21 , intermediate feature X 31 and high-level features X 41 , respectively X 11 and X 21 , X 21 and X 31 , X 31 and X 41 Put together and you get X 1 , X 2 and X 3 ; X 11 , X 21 , X 31 and X 41 Put together and you get X 4 ; Generate new low-level features X 1 and X 2 , New intermediate feature X 3 and new advanced features X 4 , X 1 , X 2 , X 3 and X 4 The resolutions are 1 / 8, 1 / 16, 1 / 32, and 1 / 32 of the original input image respectively.

[0016] Among them, the key point enhancement module in the coarse-grained camouflaged object positioning stage finds key points with strong structural information in the early stage of the model baseline as suspicious points where camouflaged objects may exist, and enhances their features. The specific implementation method is as follows: for feature X 1 and X 2 First, the Gaussian difference is used to calculate the key point information, and then the 1x1 convolution operation is used to adjust the number of channels of the key point information matrix; the Sigmoid operation is used to convert it into a weight matrix, and the weights are thresholded to enhance only some very significant feature points; finally, the weight matrix after thresholding is used to convolve the original feature X 1 , X 2 Perform element-wise multiplication to achieve weighted enhancement.

[0017] Among them, the decoding operation in the coarse-grained camouflage object positioning stage is to use four simple decoders to decode the enhanced features, and the implementation method is:

[0018]

[0019] Where Bn(·) denotes batch normalization, Conv(·) denotes a 3×3 convolution operation; the activation function ReLU(·) is applied after each batch normalization and convolution operation; these operations are applied to the four branches in the baseline respectively; the input of the simple decoder is denoted as X i , the output is recorded as

[0020] Among them, the spatial channel feature fusion in the coarse-grained camouflage object positioning stage adopts a pyramid structure to fully integrate the features of four scales, which is a progressive fusion method.

[0021] Among them, the structure adaptation patch method in the fine-grained optimization stage is to select valid patches containing foreground information according to the coarse map positioning map M when splicing high-resolution image patches, so as to avoid the interference of background patches; this part adopts a two-step amplification strategy to gradually supplement the coarse feature map with multi-scale high-resolution information to improve the fine-grained boundary segmentation;

[0022] Specific implementation method: In the structure adaptation patch, in order to effectively utilize the high-resolution information, a two-step enlargement strategy is adopted, which is a surface structure information enhancement method used to gradually enlarge the predicted image to the original input size;

[0023] First, high-resolution splicing is performed on the input feature map to enhance its high-resolution information;

[0024] Subsequently, it is fully decoded using a simple decoder and concatenated with the mixed feature f output from the spatial channel feature fusion module;

[0025] Finally, bilinear interpolation is used to double the size of the feature map, and the previous concatenation and decoding operations are repeated at a higher resolution;

[0026] In the second step, it is directly scaled up by a factor of four to generate the final prediction map.

[0027] Among them, the splicing in the fine-grained optimization stage is to divide the coarse positioning map M and the original high-resolution image I into 64 patches in equal proportion, and find the patch with foreground target in the coarse positioning map; because it is an equal-proportional segmentation, the corresponding patch carrying foreground information in the high-resolution image I is found based on this, and it is spliced ​​with the feature map to be enhanced; this implementation method avoids a large amount of high-resolution background noise, so that the high-resolution information supplemented by the model comes as much as possible from the foreground target; the specific method is:

[0028] The coarse localization map M and the high-resolution original image I are divided into eight equal rows and eight columns; find the blocks containing foreground information in the coarse prediction map, that is, the blocks whose maximum value exceeds 0.5 after applying the Sigmoid function; since the segmentation is proportional, the corresponding blocks in the high-resolution input image I can be located and spliced ​​together along the channel dimension; given that the RGB image has three channels, the eight-row and eight-column segmentation may have a maximum of 64 foreground blocks, that is, a maximum of 192 channels can be generated; in order to ensure that all 192 channels contain high-resolution information of suspicious foregrounds, in a batch, check the number of high-resolution slices in the corresponding coarse localization map; if there are less than 192, copy the channels to 192 through division and modulo operations.

[0029] The advantages of the present invention are as follows:

[0030] The present invention proposes a structural adaptive method to supplement structural information from point to surface. High-resolution information is selectively supplemented in the later stage of the model to enhance the positioning and fine edge segmentation of small objects in complex environments. Specifically, the present method implements the camouflaged object detection task in three stages: feature extraction stage, coarse-grained camouflaged target positioning stage, and finally fine-grained optimization stage. In the first stage, features are extracted and fused to ensure that the basic features are sufficient for the task. The second stage adopts a key point enhancement module to focus on key positions in the early stage of model processing. To supplement high-resolution information, the present invention proposes mixed resolution attention, which uses low-level high-resolution features to enhance high-level features that are rich in semantic information but lack resolution information.

[0031] When obtaining the structural prior, the spatial channel feature fusion module adopts a reliable multi-scale feature hybrid method to carefully obtain an effective structural prior. In the fine-grained optimization stage, a two-step upscaling strategy is used to process multi-scale features. The structure-adaptive patch enhances the final fine segmentation by selectively integrating high-resolution foreground patches while minimizing background noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 is a baseline schematic diagram of the present invention;

[0034] Figure 2 This is a visualization effect display diagram of the present invention, wherein the portion framed in red indicates that the model is disturbed. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] like Figure 1-Figure 2 As shown, the baseline of the present invention is as follows:

[0037] The baseline of the structure-adaptive camouflaged object detection model includes three stages: feature extraction stage, coarse-grained camouflaged object localization stage, and fine-grained optimization stage. First, based on the input image, the basic feature extraction stage uses SwinTransformer to extract features at four scales, which are then fused to effectively integrate multi-scale information while achieving the effect of dividing feature levels. In the coarse-grained camouflaged object localization stage, the extracted features are used to locate the camouflaged target and establish a reliable structural prior. Since low-level features are crucial for small object detection, the present invention provides a key point enhancement module to enhance these features.

[0038] The hybrid resolution attention module provided by the present invention uses high-resolution features to enhance low-resolution features. The enhanced features are decoded by four simple decoders, and further integrated by the spatial channel feature fusion module to form a rough structural prior. In the fine-grained optimization stage, the present invention provides a structural adaptation patch guided by the shape of the structural prior and combined with high-resolution information to obtain the final refined prediction map.

[0039] The various modules provided in the present invention are introduced in detail.

[0040] Key point enhancement module:

[0041] The key point enhancement module is used to enhance the low-level features X 1 and X 2 . This module adopts a point enhancement strategy to carefully enhance selected key features. First, a Gaussian difference (DoG) operation is performed on the input low-level features. This involves convolving the features with Gaussian kernels of different standard deviations σ to extract high-frequency information. Multi-layer Gaussian convolutions are used to achieve stereo smoothing of the feature map and calculate the difference. In this module, the choice of σ value is crucial to effectively capture details of different scales. Specifically, for the jth Gaussian convolution (j=1,2), use:

[0042] σ i =3.0×(0.9 γ+j-1 )

[0043] Where γ represents the scale selection parameter, which is set to 3 in the experiment. By using these exponentially decaying σ values, it is ensured that the Gaussian kernel can capture a variety of spatial details from wide to fine. The Gaussian difference can be expressed as:

[0044] DoG(X i ,σ 1 ,σ 2 )=G(X i ,σ 1 )-G(X i ,σ 2 )

[0045] Where G(X i ,σ j ) represents the same as σ j The corresponding Gaussian convolution, i, j = 1, 2. G 1 (X i )-G 2 (X i ) Calculate X i The difference between the two Gaussian convolution results.

[0046] Use 1x1 convolution to deform the DoG result so that its shape is the same as X iThe deformed DoG differences are converted to weights using the Sigmoid function. The Sigmoid function scales the differences to between 0 and 1 and the weights to between 0 and 2 by element-wise multiplication. A threshold is applied to these weights: weights above the threshold remain unchanged, and weights below the threshold are set to 1. This thresholding process converts continuous edges in the weights into discrete salient points, which are considered suspicious points and weighted back to the original feature map accordingly to enhance the original features.

[0047] The process is formalized as follows:

[0048] W=T(2⊙S(C 1 (DoG)))

[0049] Among them, C 1 (·) represents 1x1 convolution, the symbol ⊙ represents element-by-element multiplication, and S(·) represents the Sigmoid function. T(·) represents the threshold operation. In the experiment, the threshold is set to 1. After the above process, W represents the key point weight. The weight is then used to weight the input feature X of the key point enhancement module. i This process is expressed as:

[0050]

[0051] Mixed-resolution attention:

[0052] Contains rich resolution information, which is crucial for detecting small objects. Based on the high-resolution features of , this paper designs a mixed-resolution attention mechanism.

[0053] Specifically, for the input high-resolution feature map and low-resolution feature maps Depth-wise convolution is applied to extract spatial features. Channel features are extracted using point-wise convolution. The information extracted from the high-resolution feature map is recorded as Q, and the information extracted from the low-resolution feature map is recorded as K and V. The similarity matrix between the high-resolution Q and the low-resolution K is calculated and then converted into weights through the Sigmoid operation. The enhanced low-resolution feature V′ is obtained by performing element-wise multiplication of V with the weights.

[0054] This process is expressed as:

[0055] V'=S(Q·K T )⊙V.

[0056] Spatial channel feature fusion module:

[0057] In order to fully fuse the four scale features after decoding, the present invention introduces a spatial channel feature fusion module, which is designed as a pyramidal feature fusion architecture. This method uses iterative element-by-element multiplication and feature concatenation to obtain mixed features with different complexities at each iterative step. In order to enhance the new features after each feature concatenation or feature upsampling, a Bc module containing 3x3 convolution and batch normalization is applied to further decode the mixed features obtained at each iterative step.

[0058] Specifically, and Perform element-wise multiplication to obtain This feature integrates and Similarly, we will and Multiply them together and you get After being processed by an independent Bc module, it is recorded as The subscript indicates that the feature retains its original feature and does not contain features of other scales. Through subsequent concatenation and element-by-element multiplication, the comprehensive fusion of all features is gradually achieved, and finally

[0059] Optimize spatial information through 3x3 convolution, and perform channel compression through 1x1 convolution to generate a coarse prediction map X with a resolution of 64x64 pixels. out .X out Through 1x1 convolution processing to obtain the coarse prediction map M 1 The intermediate features obtained during the fusion process and (denoted as f) will also be output for subsequent refinement steps.

[0060] Structural Adaptation Patches:

[0061] High-resolution information is crucial for the fine-grained optimization stage of the model. A common way to supplement high-resolution information is to shrink the original image and then concatenate it with the feature map. This method not only reduces the resolution details of the original image, but also introduces a large amount of high-resolution background information, which may interfere with the model's judgment.

[0062] The present invention adopts a strategy called high-resolution stitching. The coarse positioning map M and the high-resolution original image I are divided into eight equal rows and eight columns. Find the blocks containing foreground information in the coarse prediction map, that is, the blocks whose maximum value exceeds 0.5 after applying the Sigmoid function. Since the segmentation is proportional, the corresponding blocks in the high-resolution input image I can be located and stitched together along the channel dimension. Given that the RGB image has three channels, the eight-row and eight-column segmentation may have a maximum of 64 foreground blocks, that is, a maximum of 192 channels can be generated. In order to ensure that all 192 channels contain high-resolution information of suspicious foregrounds, in a batch, the number of high-resolution slices in the corresponding coarse positioning map is checked. If there are less than 192, the channels are copied to 192 by division and modulo operations.

[0063] In the structure adaptation patch, in order to effectively utilize the high-resolution information, a "two-step enlargement" strategy is adopted, which is a face structure information enhancement method for gradually enlarging the prediction map to the original input size. The input feature map is spliced ​​at high resolution to enhance its high-resolution information. It is fully decoded using a simple decoder and spliced ​​with the mixed feature f output from the spatial channel feature fusion module. The feature map size is doubled using bilinear interpolation, and the previous splicing and decoding operations are repeated at a higher resolution. In order to improve computational efficiency, it is directly enlarged four times in the second step to generate the final prediction map.

[0064] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting camouflaged objects with structural adaptability, characterized in that: According to the shape of the camouflaged object, the learning strategy is adaptively adjusted to avoid background interference while paying more attention to the key foreground information. The method includes the following stages: The first stage: feature extraction stage: SwinTransformer is used to extract multi-scale features from the input image, and then the feature division module is used to divide and fuse the features into low-level features X1 and X2, mid-level features X3, and high-level features X4; The second stage: coarse-grained camouflage object positioning stage: Different enhancement strategies are applied to the four features of the three levels obtained; the key point enhancement module and the mixed resolution attention mechanism are used to enhance the low-level features X1 and X2; the spatial attention is used to enhance the high-level feature X4; the processed X1, X2, X4 and the unprocessed intermediate feature X3 are decoded by the decoder, and the spatial channel feature fusion module is used to fuse the features to obtain a rough positioning map M; The third stage: fine-grained optimization stage: Using the obtained rough positioning map M, the patch containing the foreground target is selected from the high-resolution image using the structure adaptation patch method, and is spliced ​​along the channel dimension to the output X of the spatial channel feature fusion module. out Then, a new round of feature fusion is performed. After two repeated operations, the final fine segmentation map M1 is obtained.

2. The method for detecting a camouflaged object with structural adaptability according to claim 1, characterized in that: The feature division method in the feature extraction stage is as follows: for the low-level feature X extracted by the backbone network 11 and X 21 , intermediate feature X 31 and high-level features X 41 , respectively X 11 and X 21 , X 21 and X 31 , X 31 and X 41 Put together to get X1, X2 and X3; 11 , X 21 , X 31 and X 41 Spliced ​​together, we get X4; generate new low-level features X1 and X2, a new mid-level feature X3 and a new high-level feature X4. The resolutions of X1, X2, X3 and X4 are 1 / 8, 1 / 16, 1 / 32 and 1 / 32 of the original input image respectively.

3. The method for detecting a camouflaged object with structural adaptability according to claim 1, characterized in that: The key point enhancement module in the coarse-grained camouflaged object positioning stage is to find key points with strong structural information in the early stage of the model baseline as suspicious points where camouflaged objects may exist, and enhance their features. The specific implementation method is as follows: for features X1 and X2, first use Gaussian difference to calculate the key point information, and then use 1x1 convolution operation to adjust the number of channels of the key point information matrix; use Sigmoid operation to convert it into a weight matrix, and perform threshold processing on the weights; finally, use the weight matrix after threshold processing to perform element-by-element multiplication of the original features X1 and X2.

4. The method for detecting a camouflaged object with structural adaptability according to claim 1, characterized in that: The decoding operation in the coarse-grained camouflaged object positioning stage is to use four simple decoders to decode the enhanced features, and the implementation method is: Where Bn(·) denotes batch normalization, Conv(·) denotes a 3×3 convolution operation; the activation function ReLU(·) is applied after each batch normalization and convolution operation; these operations are applied to the four branches in the baseline respectively; the input of the simple decoder is denoted as X i , the output is recorded as 5. The method for detecting a camouflaged object with structural adaptability according to claim 1, characterized in that: The spatial channel feature fusion in the coarse-grained camouflage object positioning stage adopts a pyramid structure to fully fuse the features of four scales.

6. The method for detecting a camouflaged object with structural adaptability according to claim 1, characterized in that: The structure adaptation patch method in the fine-grained optimization stage is to select valid patches containing foreground information according to the coarse map positioning map M when splicing high-resolution map patches; this part adopts a two-step amplification strategy to gradually supplement the coarse feature map with multi-scale high-resolution information to improve the fine-grained boundary segmentation; Specific implementation method: In the structure adaptation patch, a two-step amplification strategy is adopted; First, high-resolution splicing of the input feature maps is performed; Subsequently, it is fully decoded using a simple decoder and concatenated with the mixed feature f output from the spatial channel feature fusion module; Finally, bilinear interpolation is used to double the size of the feature map, and the previous concatenation and decoding operations are repeated at a higher resolution; In the second step, it is directly enlarged by a factor of four.

7. The method for detecting a camouflaged object with structural adaptability according to claim 1, characterized in that: The stitching in the fine-grained optimization stage is to divide the coarse positioning map M and the original high-resolution image I into 64 patches in equal proportion, and find the patch with the foreground object in the coarse positioning map; Since it is an equal-proportional segmentation, the corresponding patch carrying foreground information in the high-resolution image I is found based on this, and it is spliced ​​with the feature map to be enhanced; the specific method is: The coarse localization map M and the high-resolution original image I are divided into eight equal rows and eight columns; find the blocks containing foreground information in the coarse prediction map, that is, the blocks whose maximum value exceeds 0.5 after applying the Sigmoid function; since the segmentation is proportional, the corresponding blocks in the high-resolution input image I can be located and spliced ​​together along the channel dimension; given that the RGB image has three channels, the eight-row and eight-column segmentation may have a maximum of 64 foreground blocks, that is, a maximum of 192 channels can be generated; in order to ensure that all 192 channels contain high-resolution information of the suspected foreground, in a batch, check the number of high-resolution slices in the corresponding coarse localization map; If there are fewer than 192 channels, the channels are copied to 192 channels through division and modulo operations.

Images