Explicit high-order attention network and detection method for precise camouflage target detection

Abstract

The application provides an explicit high-order attention network and a detection method for precisely detecting a camouflaged target, the network comprising a high-order semantic attention module, a high-order boundary attention module, a high-order reverse attention module, a cross-layer high-order fusion module and a high-order attention module as a basic module, the network first uses a visual attention network architecture to generate multiple levels of features, inputs the features into a receptive field block to generate corresponding context features, and generates multiple enhanced features through the cross-layer high-order fusion module; then, the high-order semantic attention module is used to roughly locate the camouflaged target for the enhanced features of the high layer, the high-order boundary attention module is used to predict the boundary of the camouflaged target for the enhanced features of the low layer, and finally, the high-order reverse attention module is used to mine interference items in the enhanced features of the high layer under the guidance of an initial positioning map generated by the high-order semantic attention module and a boundary prediction map predicted by the high-order boundary attention module. The network can accurately detect the camouflaged target.

Description

Technical Field

[0001] This invention mainly relates to the field of image processing technology, specifically an explicit high-order attention network and detection method for accurate camouflaged target detection. Background Technology

[0002] Camouflage is crucial for wildlife. It allows them to blend seamlessly into their environment, avoiding detection by predators and enemies. However, segmenting camouflaged targets from a scene—camouflage target detection—presents a significant challenge. Edge segmentation, in particular, is more challenging because the differences between camouflaged targets and the background are often very subtle.

[0003] Currently, many methods for detecting camouflaged targets segment them from a single image using attention mechanisms. While these methods have achieved improvements, they are still limited by the relatively low order of attention. Based on the order of attention, these methods are mostly divided into four categories, using 1st, 2nd, 3rd, and implicit higher-order attention, respectively. Methods based on 1st-order attention typically employ common techniques (such as global attention and the sigmoid function) to enhance the features of the camouflaged object. Networks based on 2nd-order attention mechanisms first compute the attention distribution map of the query and key, and then perform feature enhancement on its input graph inference module. The low order of attention in these methods makes them weaker in capturing subtle differences between the camouflaged target and the background. Summary of the Invention

[0004] To address the shortcomings of current technologies, this invention, combining existing techniques and focusing on practical applications, provides an explicit high-order attention network and detection method for accurate camouflaged target detection. This network effectively detects highly camouflaged edges and, using this as a basic unit, designs a deep network for accurate target detection. The technical solution of this invention is as follows:

[0005] According to one aspect of the present invention, an explicit high-order attention network for accurate camouflaged target detection is provided. The network includes a high-order semantic attention module HSA, a high-order boundary attention module HEA, a high-order reverse attention module HRA, a cross-layer high-order fusion module CHF, and a high-order attention module EHA as a base module.

[0006] The network first uses a visual attention network architecture to generate features at multiple levels. These features are then input into receptive field blocks to generate corresponding contextual features. The contextual features are gradually integrated through a cross-layer high-order fusion module (CHF) and a high-order attention module (EHA) to generate multiple enhanced features. Next, the high-level enhanced features are roughly located by a high-order semantic attention module (HSA), while the low-level enhanced features are predicted by a high-order boundary attention module (HEA) to predict the boundaries of the camouflaged targets. Finally, guided by the initial localization map generated by the high-order semantic attention module (HSA) and the boundary prediction map predicted by the high-order boundary attention module (HEA), the high-order reverse attention module (HRA) is used to mine interference terms in the high-level enhanced features, refine the camouflaged targets, and output a segmentation prediction map.

[0007] Furthermore, the network first uses a visual attention network architecture to generate four levels of features F1, F2, F3, and F4, and corresponding context features R1, R2, R3, and R4. The enhanced features generated by the cross-layer high-order fusion module CHF are E1, E2, E3, E4, and E5. The high-level enhanced features E3, E4, and E5 are roughly located by the high-order semantic attention module HSA. The low-level enhanced features E1 and E2 are predicted by the high-order boundary attention module HEA. The high-order reverse attention module HRA is used to mine the interference terms in the enhanced features E3, E4, and E5 to refine the camouflage target.

[0008] Furthermore, the cross-layer high-order fusion module CHF uses five fusion units CHFM. Each fusion unit CHFM integrates three feature inputs at different levels from top to bottom. Among them, context features R4, R4, and R4 are fused on the basis of the high-order attention module EHA to obtain enhanced feature E5. Features R2, R3, and E5 are fused to obtain feature E4. Features E4 and E5 are concatenated to obtain feature E. 4，5 Features R2, R3, E 4，5 The fusion yields feature E3, features R1, R3, and E. 4，5 The fusion yields feature E2, features E2, E3, and E 4，5 The fusion yields feature E1.

[0009] Furthermore, for each fusion unit CHFM, if the three feature input spaces have different sizes, upsampling or downsampling is performed to make the spaces the same.

[0010] Furthermore, in the high-order semantic attention module HSA, based on the high-order attention module EHA, feature E5 is fused with downsampled E3 and E4, and then high-order attention operations are performed to promote the representation of semantic features, and finally the initial localization map of the camouflaged target is output.

[0011] Furthermore, in the higher-order boundary attention module HEA, features E1 and E2 are first encoded through convolution, then merged and fed into the higher-order attention module EHA in the downsampling space. They are then regularized using channel adaptive ReLU to generate a weight map, which participates in the weighted summation of features E1 and E2, and finally outputs the camouflage target boundary prediction map.

[0012] Furthermore, in the higher-order inverse attention module HRA, features E3, E4, and E5 are first guided into the forward and inverse branches based on the segmentation prediction map from the previous stage. Then, they are enhanced by the higher-order attention module EHA to detect false positives and false negatives. Finally, the corrected features are fused with edge features and edge map features from the higher-order boundary attention module HEA to obtain the camouflaged target segmentation prediction map.

[0013] Furthermore, the loss function used by the network is as follows:

[0014]

[0015] Among them, L edge The loss function for the camouflaged target boundary prediction map is expressed as follows:

[0016] L edge =-∑[E g log(E p )+(1-E g log(1-E) P )]

[0017] L det The loss function for segmenting the predicted image of the camouflaged target is expressed as follows:

[0018]

[0019] In the above formula, E g and E p These are the predicted camouflage target boundary map and the labeled camouflage target boundary map, respectively. These are the predicted camouflaged target segmentation map and the labeled camouflaged target segmentation map, respectively. α = β = 1, i = 3, 4, and 5 are three segmentation prediction maps output by the high-order reverse attention module HRA, and i = 6 represents the initial localization map output by the high-order semantic attention module HSA.

[0020] According to another aspect of the present invention, a method for detecting camouflaged targets based on the above-described explicit high-order attention network for accurate camouflaged target detection is provided, comprising the following steps:

[0021] S1. Divide the dataset into a training set and a test set, and preprocess these two datasets respectively;

[0022] S2. Set the calculation parameters related to epoch and initial learning rate;

[0023] S3. Feed the processed training set into the backbone network of the pre-trained model to encode the data and generate four features F1, F2, F3, and F4.

[0024] S4. Input the four encoded features F1, F2, F3, and F4 into the receptive field block to generate context features R1, R2, R3, and R4.

[0025] S5. The output of the receptive field block is sent to the cross-layer high-order fusion module CHF to achieve the fusion of three adjacent features. The cross-layer high-order fusion module CHF outputs enhanced features E1, E2, E3, E4, and E5.

[0026] S6. Input the three high-level features E3, E4 and E5 output by the cross-layer high-order fusion module CHF into the high-order semantic attention module HSA to obtain the initial localization map.

[0027] S7. Input the two low-level features E1 and E2 output by the cross-layer high-order fusion module CHF into the high-order boundary attention module HEA to obtain the boundary prediction map.

[0028] S8. Guided by the initial localization map and the boundary prediction map, the three high-level features E3, E4 and E5 of the cross-layer high-order fusion module CHF are sent into the high-order reverse attention module HRA to obtain the segmentation prediction map.

[0029] S9. Feed the initial localization map, boundary prediction map, and segmentation prediction map into the loss function to perform gradient update;

[0030] S10. After each epoch, the test set is fed into the network and the above steps are followed to obtain the segmentation prediction map. The mean absolute error is used as the indicator. If the indicator is less than the indicator of the previous epoch, the model parameters are saved.

[0031] The beneficial effects of this invention are:

[0032] 1. Due to the high similarity between the background and foreground, low-order attention struggles to detect certain boundary regions. Because the high-order attention network of this invention utilizes only the maximum-order attention and ignores low-order attention, the high-order attention of this invention can detect these regions more accurately.

[0033] 2. The network of the present invention can capture subtle relationships between spatial elements and channels, thus enabling the network to better detect small camouflaged objects.

[0034] 3. The detection method of the present invention can capture subtle differences between the camouflaged target and the background, and can achieve accurate detection of the camouflaged target. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the network architecture of the present invention.

[0036] Figure 2 This is a schematic diagram of the CHF (Cross-Layer High-Order Fusion Module) of this invention.

[0037] Figure 3 This is a schematic diagram of the high-order semantic attention module HSA of this invention.

[0038] Figure 4 This is the schematic diagram of the High-Order Boundary Attention Module (HEA).

[0039] Figure 5 This is the schematic diagram of the High-Order Reverse Attention (HRA) module.

[0040] Figure 6 This is a schematic diagram of the input image.

[0041] Figure 7 yes Figure 6 The prediction chart. Detailed Implementation

[0042] The present invention will be further described in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.

[0043] Example 1

[0044] This embodiment provides an explicit high-order attention network for accurate camouflaged target detection. (Reference) Figure 1 The diagram shown is a diagram of the main network architecture.

[0045] Since all details of a camouflaged target are important, and some of these details can provide valuable clues for camouflaged target detection, this application proposes a high-order semantic attention module (HSA), a high-order boundary attention module (HEA), and a high-order reverse attention module (HRA) for coarse target localization, target boundary prediction, and refinement, respectively, and a cross-layer high-order fusion module (CHF) to fully fuse features from different levels.

[0046] Because Taylor expansion reveals that low-order attention modules have a weak ability to model highly camouflaged boundaries, this application uses the high-order attention module EHA as the foundation for these four modules. The high-order attention module EHA is as follows: Figure 1The lower half is shown.

[0047] exist Figure 1 In the first part, the network first uses the visual attention network VAN-B3 as the backbone to generate four levels of features F. t (t = 1, 2, 3, 4). Next, F... t Input is fed into the receptive field block RFB to generate contextual features R t (t = 1, 2, 3, 4). To better perform feature fusion, the cross-layer high-order fusion module CHF is combined with the high-order attention module EHA, and R is gradually integrated. t Generate enhanced features E t (t = 1, 2, 3, 4, 5). Then, based on the higher-order attention module EHA, a higher-order semantic attention module HSA is constructed, utilizing the higher-order attention at the highest 3 layers of features E. t (t=3, 4, 5) Roughly locate the camouflaged target; in the lowest 2 layers of feature E t (t=1,2) The high-order boundary attention module HEA is used to predict the camouflaged target boundary. Finally, guided by the initial localization map generated by the high-order semantic attention module HSA and the boundary map predicted by the high-order boundary attention module HEA, the high-order reverse attention module HRA is used to mine feature E. t The interference terms (false positives and false negatives) in (t=3,4,5) are used to further refine the boundaries of the camouflaged target.

[0048] refer to Figure 2 The diagram shown is a schematic of the cross-layer high-order fusion module CHF of this application. The cross-layer high-order fusion module CHF uses five fusion units CHFM, integrating three adjacent features at different levels from top to bottom. The fusion method is as follows: (R4, R4, R4) → E5, (R2, R3, E5) → E4. Connecting E4 and E5 yields feature E. 4，5 (R2, R3, E) 4，5 →E3, (R1, R3, E 4，5 →E2, (E2, E3, E) 4，5→ E1. For each fusion unit CHFM, the three input features are labeled ①, ②, and ③. For the three inputs ①, ②, and ③, if the three input spatial sizes are the same, no upsampling or downsampling operation is performed; if the spatial sizes are different, the features are downsampled or upsampled to make their spatial sizes consistent. Taking feature E5 fusion as an example, among the three input features, ① and ② are concatenated, and then the concatenated features are convolved with 3*3 and fed into the higher-order attention module EHA. The output of the higher-order attention module EHA is then concatenated with ③ and fed into the higher-order attention module EHA to obtain the output. The three input features are fused through two higher-order attention modules EHA and placed in the residual structure to effectively capture subtle edges.

[0049] refer to Figure 3 The diagram illustrates the underlying principles of the High-Order Semantic Attention Module (HSA) of this application. For some regions (e.g., the head), which may reveal the location of camouflaged objects, some methods first coarsely estimate the gaze point. However, these methods rely on low-order attention mechanisms and have limited ability to distinguish subtle differences. Considering that high-level features contain more semantic information than low-level features, this application proposes a High-Order Semantic Attention Module (HSA) to generate semantically enhanced features based on high-order attention. Figure 3 As shown, feature E5 is a fusion of downsampled E3 and E4, followed by higher-order attention operations to enhance semantic feature representation. Specifically: feature E4 is downsampled, element-wise multiplied with feature E5, then passed through a 3x3 convolution. The output of the convolution is element-wise added to E4, and then passed through the first higher-order attention module EHA. The output of the first higher-order attention module is element-wise multiplied with the downsampled feature E3, then element-wise added to the downsampled feature E3 through a convolution, and then passed through the second higher-order attention module EHA. Subsequently, the output of the first higher-order attention module EHA is element-wise added to feature E5, then convolved with the output of the second higher-order attention module EHA through a convolution, and finally passed through a convolution to obtain a coarse camouflage target segmentation map (also the initial camouflage target localization map).

[0050] refer to Figure 4 The diagram illustrates the principles of the High-Order Boundary Attention (HEA) module in this application. Accurate boundary detection is crucial for camouflaged target detection. Therefore, existing technologies have introduced various methods, including those utilizing graph inference techniques and simple attention mechanisms. However, these methods have limitations in modeling subtle differences between the camouflaged object and the background due to low-order attention. This application introduces a high-order boundary attention module based on high-order attention to fuse features from two lower-level layers of the backbone network. As shown... Figure 4As shown, E1 and E2 are first encoded by 3×3 convolution, then merged and fed into the high-order attention module EHA in the downsampling space. They are then regularized by channel adaptive ReLU to generate a weight map. This weight map is then used in the weighted summation of E1 and E2 to obtain the camouflage target boundary map.

[0051] refer to Figure 5 The diagram shown illustrates the principles of the high-order inverse attention module (HRA) of this application. A coarse segmentation map of the camouflaged object and accurate boundary prediction need to be combined to achieve accurate detection. Since interference is unavoidable, inspired by [unspecified factors], this application designs both forward and inverse HRA for its high-order attention module. Figure 5 In the middle, E i (i = 3, 4, 5) First, guided by the semantic mapping flow from the previous stage, the system enters the forward and reverse branches. Then, it is enhanced through higher-order attention to detect false positives and false negatives, reducing uncertainty in the semantic feature flow. Finally, the corrected features are fused with their edge features and edge encodings from HEA. Specifically, feature E i (i = 3, 4, 5) First, element-wise multiplication is performed with the camouflaged target segmentation map from the previous stage to enter the forward branch, and then element-wise multiplication is performed with the reverse camouflaged target segmentation map to enter the reverse branch. The forward branch passes through the higher-order attention module (EHA), and the reverse branch also passes through the EHA. The output of the reverse branch through the EHA is subtracted from the camouflaged target segmentation feature map from the previous stage, and then added to the output of the forward branch to obtain intermediate features. Subsequently, the intermediate features are multiplied with the boundary map and then added to obtain the output camouflaged target segmentation prediction map.

[0052] Regarding the network's loss function, since the network predicts a camouflage target boundary prediction map and four levels of camouflage target segmentation prediction maps from the input image, this application introduces two types of loss functions, namely L... edge and L det Based on weighted binary cross-entropy (BCE) and weighted intersection-union ratio (IOU), the loss function for the camouflaged target boundary prediction map is:

[0053] L edge =-∑[E g log(E p )+(1-E g log(1-E) P )]

[0054] Where, and , are the predicted camouflage target boundary prediction map and the labeled camouflage target boundary prediction map, respectively.

[0055] The loss function for the camouflaged target segmentation prediction map is:

[0056]

[0057] in These are the predicted segmentation map of the camouflaged target and the labeled segmentation map of the camouflaged target, respectively.

[0058] Therefore, the total loss function is:

[0059]

[0060] Where α=β=1, i=3, 4, 5 are three segmentation prediction maps output by the high-order reverse attention module HRA, and i=6 represents the initial localization map output by the high-order semantic attention module HSA.

[0061] Example 2

[0062] This embodiment provides a detection method based on the explicit high-order attention network of Embodiment 1, which can be used for accurate detection of disguised objects. The main steps of this detection method are as follows:

[0063] 1. First, divide the dataset into training and test sets, and then preprocess these two datasets separately, including normalization and flipping.

[0064] 2. Set the epoch to 150 and the initial learning rate to 1e. -4 The optimizer is Adam, and the learning rate decreases by a factor of 10 every 50 epochs.

[0065] 3. Feed the processed training set into the backbone network of the pre-trained model to encode the data.

[0066] 4. Encode the four features F t (t = 1, 2, 3, 4) are fed into the receptive field module to expand the receptive field and enhance the contextual representation capability of the features.

[0067] 5. Convert the output R of the receptive field module t (t = 1, 2, 3, 4), are fed into the cross-layer high-order fusion module CHF to achieve the fusion of three adjacent features. The cross-layer high-order fusion module CHF outputs the enhanced feature E. t (t = 1, 2, 3, 4, 5).

[0068] 6. Integrate the three high-level features E of the cross-layer high-order fusion module CHF t (t=3,4,5) are fed into the high-order semantic attention module HSA to obtain the initial localization map.

[0069] 7. Combine the two low-level features E of the cross-layer high-order fusion module CHF. t(t=1,2) are fed into the high-order boundary attention module HEA to obtain the boundary prediction map.

[0070] 8. Guided by the initial localization map and boundary prediction map, the three high-level features E of the cross-layer high-order fusion module are... t (t=3,4,5) are fed into the high-order reverse attention module to obtain the predicted camouflage target segmentation prediction map.

[0071] 9. Input the initial localization map, boundary prediction map, and segmentation prediction map into the loss function L. total Gradient updates are performed during this process.

[0072] 10. After each epoch, the test set is fed into the network and processed through steps 3, 4, 5, 6, 7, and 8 in sequence to obtain the segmentation prediction map. The mean absolute error is used as the metric (the smaller the better). If the metric is less than the metric of the previous epoch, the model parameters are saved.

[0073] Using the above methods, refer to Figure 6 as well as Figure 7 As shown, in Figure 6 In the image, the cat is hidden in the background, with only subtle differences between it and the background. However, the method described in this application allows for its precise detection.

Claims

1. An explicit high-order attention network for accurate camouflaged target detection, capable of segmenting camouflaged targets from images, characterized in that, The network includes a high-order semantic attention module (HSA), a high-order boundary attention module (HEA), a high-order reverse attention module (HRA), a cross-layer high-order fusion module (CHF), and a high-order attention module (EHA) as the foundation module. The network first uses a visual attention network architecture to generate features at multiple levels. These features are then input into receptive field blocks to generate corresponding contextual features. The contextual features are gradually integrated through a cross-layer high-order fusion module (CHF) and a high-order attention module (EHA) to generate multiple enhanced features. Then, the high-level enhanced features are roughly located by a high-order semantic attention module (HSA), and the low-level enhanced features are predicted by a high-order boundary attention module (HEA) to predict the boundaries of the camouflaged targets. Finally, guided by the initial localization map generated by the high-order semantic attention module (HSA) and the boundary prediction map predicted by the high-order boundary attention module (HEA), the high-order reverse attention module (HRA) is used to mine interference terms in the high-level enhanced features, refine the camouflaged targets, and output a segmentation prediction map. The network first uses a visual attention network architecture to generate four levels of features. , , , Generate corresponding context features , The enhanced features generated by the cross-layer high-order fusion module CHF are , , , , Enhanced features for high-level areas , , The high-order semantic attention module HSA is used to roughly locate the disguised target, while the low-level... , Enhanced features are obtained by predicting the camouflage target boundary using a high-order boundary attention module (HEA) and mining enhanced features using a high-order reverse attention module (HRA). , , The interference items in the data are refined to conceal the target.

2. The explicit high-order attention network for accurate camouflaged target detection according to claim 1, characterized in that, The cross-layer high-order fusion module CHF uses five fusion units CHFM. Each fusion unit CHFM integrates three feature inputs at different levels from top to bottom, including context features. Enhanced features are obtained by fusing with the high-order attention module EHA. ,feature The fusion yields feature E4, and feature E4 and Obtain features ,feature Obtain features ,feature Features obtained through fusion ,feature Features obtained through fusion .

3. The explicit high-order attention network for accurate camouflaged target detection according to claim 2, characterized in that, For each fusion unit CHFM, if the three feature input spaces have different sizes, upsampling or downsampling is performed to make the spaces the same.

4. The explicit high-order attention network for accurate camouflaged target detection according to claim 1, characterized in that, In the Higher-Order Semantic Attention Module (HSA), based on the Higher-Order Attention Module (EHA), features are made... After fusion and downsampling and Then, higher-order attention operations are performed to facilitate the representation of semantic features, and finally the initial localization map of the camouflaged target is output.

5. The explicit high-order attention network for accurate camouflaged target detection according to claim 1, characterized in that, In the High-Order Boundary Attention Module (HEA), features and First, encoding is performed through convolution, then the results are merged and fed into a high-order attention module (EHA) in the downsampling space. Channel-adaptive ReLU regularization is then applied to generate a weight map, which participates in… and The weighted summation is used to finally output the camouflaged target boundary prediction map.

6. The explicit high-order attention network for accurate camouflaged target detection according to claim 1, characterized in that, In the high-order reverse attention module HRA, features , , First, guided by the segmentation prediction map from the previous stage, the system enters the forward and reverse branches. Then, it is enhanced by the higher-order attention module EHA to detect false positives and false negatives. Finally, the corrected features are fused with edge features and edge map features from the higher-order boundary attention module HEA to obtain the camouflaged target segmentation prediction map.

7. The explicit high-order attention network for accurate camouflaged target detection according to claim 1, characterized in that, The loss function used by the network is as follows: ， in, The loss function for the camouflaged target boundary prediction map is expressed as follows: ， The loss function for segmenting the predicted image of the camouflaged target is expressed as follows: ， In the above formula, and These are the predicted camouflage target boundary map and the labeled camouflage target boundary map, respectively. These are the predicted camouflaged target segmentation map and the labeled camouflaged target segmentation map, respectively. , =3, 4, and 5 are three segmentation prediction maps output by the Higher-Order Reverse Attention (HRA) module. =6 represents the initial localization map output by the Higher-Order Semantic Attention (HSA) module.

8. A method for detecting camouflaged targets based on an explicit high-order attention network for accurate camouflaged target detection as described in any one of claims 1-7, characterized in that, Includes the following steps: S1. Divide the dataset into a training set and a test set, and preprocess these two datasets respectively; S2. Set the calculation parameters related to epoch and initial learning rate; S3. Feed the processed training set into the backbone network of the pre-trained model to encode the data and generate four features. , , , ; S4. Encode the four features , , , Feed into the receptive field block to generate contextual features , ; S5. The output of the receptive field block is fed into the cross-layer higher-order fusion module (CHF) to achieve the fusion of three adjacent features. The enhanced feature output by the cross-layer higher-order fusion module (CHF) is... , , , , ; S6. The three high-level features output by the cross-layer high-order fusion module CHF , , The initial localization map is obtained by feeding it into the high-order semantic attention module HSA; S7. The two low-level features output by the cross-layer high-order fusion module CHF are... , The data is fed into the High-Order Boundary Attention (HEA) module to obtain the boundary prediction map. S8. Guided by the initial localization map and boundary prediction map, the three high-level features of the cross-layer high-order fusion module CHF are... , , The data is fed into the Higher-Order Reverse Attention (HRA) module to obtain a segmentation prediction map. S9. Feed the initial localization map, boundary prediction map, and segmentation prediction map into the loss function to perform gradient update; S10. After each epoch, the test set is fed into the network and the above steps are followed to obtain the segmentation prediction map. The mean absolute error is used as the indicator. If the indicator is less than the indicator of the previous epoch, the model parameters are saved.

Images