An attention edge interaction optical remote sensing image saliency target detection method

Abstract

The application discloses an attention edge interaction optical remote sensing image saliency target detection method, aiming at improving the detection accuracy of saliency targets in optical remote sensing images. The current saliency target detection of optical remote sensing images has the following two problems: due to the insufficient utilization of edge information, the saliency target prediction map is prone to boundary blur in some complex scenes of optical remote sensing images; in the process of gradually transferring the high-level semantic information extracted by the model to the shallow layer, the position information captured by the deeper layer may be gradually diluted at the same time. In view of the first problem, a multi-scale attention interaction module is designed to effectively fuse fine edge features. In view of the second problem, a semantic guidance fusion module is designed to reduce the information loss of low-level features in the fusion process. In combination with the above two designs, the model designed by the application can robustly and accurately detect the saliency targets in the optical remote sensing images, and has the ability to process various complex scenes, and is worth popularizing.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and relates to the improvement of salient target detection models, specifically a salient target detection method for optical remote sensing images based on attention edge interaction. Background Technology

[0002] Salience object detection is widely used as a preprocessing step for other computer vision tasks, its main purpose being to detect and segment the most attractive targets from images. Examples of applications of salient object detection in computer vision include unsupervised video object segmentation, semantic segmentation, and person re-identification. In recent years, with the continuous development of deep learning, research on salient object detection in optical remote sensing images has received widespread attention. Since optical remote sensing images are collected from high altitudes by remote sensors deployed on satellites or aircraft, salient objects in optical remote sensing images are typically more diverse and complex in terms of quantity, shape, size, location, and orientation compared to natural images, and are also more difficult to identify from the background. Therefore, salient object detection in optical remote sensing images is generally more challenging than salient object detection in natural images. Currently, there is limited research specifically targeting salient object detection in optical remote sensing images, and methods directly applying those used for salient object detection in natural images are ineffective. Therefore, this invention designs two modules to improve the accuracy of salient object detection in optical remote sensing images. The specific methods are as follows: In order to solve the problem that the predicted boundary of salient targets in some complex scenes of optical remote sensing images is easily blurred, a multi-scale attention interaction module is designed to interactively refine the features of salient regions and salient edge features, so that the network can fully explore the complementary information of salient target detection and edge detection; In order to solve the problem of semantic dilution in the feature fusion process, a semantic-guided fusion module is designed to enhance the propagation of semantic information, strengthen the expression of salient target features and suppress redundant background features. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] To address the shortcomings of existing technologies, this invention provides a salient target detection method for optical remote sensing images based on attention edge interaction. This solves the problem of poor detection performance of current optical remote sensing image salient target detection models in some complex scenes.

[0005] (II) Technical Solution

[0006] To achieve the above objectives, this invention proposes a salient target detection method for optical remote sensing images based on attention and edge interaction. This network can fully explore the complementarity between salient regions and edges using attention and edge perception mechanisms, thereby generating salient region features with high-quality edges. First, to improve the fusion quality of salient region and edge features, this invention proposes a multi-scale attention interaction (MAI) module, which consists of a multi-scale aggregation attention (MAA) module, a gating mechanism, and a feature interaction structure. The multi-scale aggregation attention module aims to explore multi-scale information of salient region features and embeds sequential channel-spatial attention enhancement to focus on salient features. The gating mechanism aims to polish edge features to reduce the influence of non-salient cues, while the feature interaction structure aims to interactively refine the features of the two preprocessed features. Second, to reduce information loss during feature fusion, this invention proposes a semantic-guided fusion (SF) module, which introduces high-level semantic information to guide the fusion of low-level features. Finally, a salient region feature map and an edge feature map are obtained. These two feature maps are then fed into a convolutional layer to obtain the final detection results of salient targets and salient edges.

[0007] The present invention provides a method for detecting salient targets in optical remote sensing images based on attention edge interaction, comprising the following steps:

[0008] S1. The overall architecture of the model is an encoder-decoder network. The encoder network uses EfficientNet-B7 to extract salient region features and salient edge features. The decoder network is designed as a two-branch interactive structure. Multi-scale attention interaction modules are added at the interaction nodes of the two branches. The aim is to use attention and edge awareness mechanisms to preprocess the salient region features and salient edge features, and then refine the interaction of the two features to obtain edge-enhanced salient region features and attention-enhanced edge features.

[0009] S2. After being refined through the multi-scale attention interaction module, high-level salient region features and edge features are gradually fused with low-level salient region features and edge features, respectively. In the final two fusion stages of the salient region features, i.e., s... 2 and The fusion stage of upsampling features and s 1 and In the upsampling feature fusion stage, a semantically guided fusion module is introduced to enhance the propagation of semantic information.

[0010] S3. With the collaboration of the multi-scale attention interaction module and the semantic-guided fusion module, high-level salient region features and edge features are gradually refined through interaction and fused with low-level salient region features and edge features, ultimately yielding a refined salient region feature map. and edge feature map These two feature maps are then fed into a convolutional layer to predict salient targets and salient edges.

[0011] (III) Beneficial Effects

[0012] This invention provides a method for salient target detection in optical remote sensing images based on attention edge interaction. It has the following beneficial effects:

[0013] This invention addresses the problem of insufficient fusion between salient region features and edge features by using a multi-scale attention interaction module. It also solves the problem of high-level semantic information dilution through a semantically guided fusion module.

[0014] The model proposed in this invention has good performance. Experimental results on three optical remote sensing image datasets, EORSSD, ORSSD, and ORSI-4199, show that the algorithm of this invention is superior to existing advanced algorithms. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating the overall framework structure of the present invention;

[0016] Figure 2 This is a structural diagram of the multi-scale attention interaction module constructed in this invention;

[0017] Figure 3 The semantic guidance fusion module constructed for this invention and its feature visualization diagram;

[0018] Figure 4 This is the saliency target prediction map of the present invention; Detailed Implementation

[0019] The technical method of this invention will now be clearly and completely described with reference to the accompanying drawings. A method for detecting salient targets in optical remote sensing images based on attention edge interaction, the specific implementation steps of which are as follows:

[0020] (S1): Designing an encoder-decoder network

[0021] The encoder network used in this invention is the high-efficiency, high-performance EfficientNet-B7, and the decoder network is designed with a dual-branch interactive structure. One branch is used for salient object detection, and the other for salient edge detection. The interaction node between the two branches is the proposed multi-scale attention interaction module, which enables the progressive refinement of salient region features and edge features. The salient object detection branch ends with two semantically guided fusion modules, which introduce high-level semantic information into the fusion process of low-level features to reduce the loss of important information.

[0022] (S2): Design a multi-scale attention interaction module.

[0023] like Figure 2 As shown in the left box, the Multi-Scale Attention Interaction (MAI) module proposed in this invention performs interactive optimization on edge features and salient region features. Essentially, the MAI module performs three steps: two separate feature processing operations followed by feature interaction.

[0024] First, salient region features are captured using a multi-scale convergent attention (MAA) module, providing comprehensive information within a single feature level. This is advantageous for capturing salient targets of various sizes and shapes in optical remote sensing images. The multi-scale convergent attention module, such as... Figure 2 As shown in the right box, this module consists of six parallel convolutional branches. Except for the first branch, which uses only a 1×1 convolution to preserve the original information of the input features, the remaining j-th (j=2,3,4,5,6) branches sequentially employ a 1×1 convolution, a 1×(2j-1) convolution, a (2j-1)×1 convolution, and a 3×3 dilated convolution with a dilation rate of 2j-1. Furthermore, top-down information propagation between the parallel convolutional branches effectively preserves the local details of salient targets. Specifically, when j=1,2,3,4,5, the output features of the j-th branch are fed into the (j+1)-th branch. This structure also features cascaded residuals, which force each branch to learn unique features different from the others, thereby reducing feature redundancy and achieving efficient and useful feature extraction. The output features of these branches are then integrated using channel concatenation and a 3×3 convolution, as shown below:

[0025]

[0026] To further enhance the integrated features, this invention employs a lightweight convolutional block attention module (CBAM) to perform sequential channel-spatial attention, which is then refined in an adaptive manner. The expression is as follows:

[0027]

[0028] ⊙ represents channel multiplication. It is element-wise multiplication. CA(·) and SA(·) represent channel attention and spatial attention operations, respectively.

[0029] Next, the edge features are polished through a gating mechanism. The specific process is as follows:

[0030]

[0031] Finally, the salient region features from attention processing and the edge features from gate processing are interactively fused through channel concatenation and multiplication, respectively, to generate edge-enhanced salient region features and attention-enhanced edge features. Afterward, they are further optimized through two consecutive 3×3 convolutions. Furthermore, a short connection is used in each branch to preserve the original information, ultimately generating the output features of the MAI-i module. and ).

[0032]

[0033] in This represents a channel connection operation, while It sums the elements.

[0034] (S3): Design semantic guidance fusion module.

[0035] like Figure 3 As shown in the bottom box, the Semantic Guided Fusion (SF) module combines the highest-level attention-edge interaction features through upsampling and element-wise multiplication. Injected into low-level features s i and In (i = 1, 2), short connections are then used to preserve the details of low-level features, followed by modulation using a 3×3 convolution. Finally, the two fused features are channel-connected and fed into a 3×3 convolution and a 1×1 convolution respectively for further refinement and channel adjustment, resulting in the following image.

[0036]

[0037] Figure 3 The feature maps with and without the semantically guided fusion module were also visualized and compared. It can be seen that when i=1, The salient target in the circle is more than Much more complete. This is because the former is... Under the guidance of [the relevant authority], the entire salient objective can be highlighted more precisely.

[0038] The effects of the present invention will be described in detail below with reference to experimental data and prediction graphs.

[0039] Table 1 compares the computational efficiency and accuracy of the proposed method with other methods on the EORSSD, ORSSD, and ORSI-4199 datasets, with the best scores highlighted in bold. The experimental results in Table 1 show that the proposed method, AEINet, outperforms the other 15 models, ranking first in 8 metrics and third in 1 metric across all datasets. Compared to the three best-performing models in the three network groups (ACCoNet, EMFINet, and MCCNet), the proposed model has the fewest parameters and FLOPs. Specifically, the model has only 67.4M parameters, while ACCoNet, EMFINet, and MCCNet have 102.55M, 107.26M, and 67.65M parameters, respectively. The model's FLOPs are only 15.04G, while ACCoNet, EMFINet, and MCCNet have 179.96G, 480.9G, and 112.8G FLOPs, respectively. From the above quantitative comparison and computational complexity comparison, it can be concluded that the proposed method is effective and efficient.

[0040] Table 1 compares the present invention with state-of-the-art methods on the EORSSD, ORSSD, and ORSI-4199 datasets.

[0041]

[0042]

[0043] Table 2 shows the impact of the modules proposed in this invention on model performance.

[0044]

[0045] Table 2 illustrates the effectiveness of the modules proposed in this invention. From the quantitative comparison shown in Table 2, it can be seen from top to bottom that as the number of proposed modules increases, the effectiveness decreases. and S m The increasing scores and decreasing MAE scores demonstrate the positive impact of each proposed module on the overall model performance. In summary, the complete model of this invention improves the "Baseline" by 1.15% on the EORSSD dataset. 1.95% S m And 0.29% MAE. The performance improvement is even more significant on the ORSSD dataset, meaning the complete model of this invention achieves [performance improvement]. S m The baseline was improved by 2.26%, 2.48%, and 0.38% on MAE, respectively.

[0046] Figure 4 The effectiveness of the model proposed in this invention is illustrated. The model proposed in this invention ( Figure 4 The last column can generate the most accurate saliency maps in different scenes such as large targets, low-contrast targets, slender targets, small targets, and truncated targets with cluttered backgrounds, effectively improving the segmentation accuracy of salient targets in some complex scenes of optical remote sensing images.

[0047] This invention proposes a salient target detection method for optical remote sensing images based on attention-edge interaction. It achieves interaction between the salient target detection and salient edge detection branches through a multi-scale attention interaction module, and provides edge-enhanced attention and attention-enhanced edges for refining salient regions and edge features. Furthermore, the proposed semantic-guided fusion module introduces high-level semantic information into low-level features for fusion, effectively alleviating the semantic dilution problem and further improving detection accuracy. Extensive experimental results demonstrate that the proposed method, by fully combining the performance advantages of attention and edge awareness mechanisms, outperforms other advanced algorithms in salient target detection for optical remote sensing images.

[0048] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for salient object detection of optical remote sensing images with attention edge interaction, characterized in that: Comprise the following steps: S1. The overall architecture of the model is an encoder-decoder network, the encoder network adopts EfficientNet-B7 to extract saliency region features and saliency edge features, and the decoder network is designed as a double-branch interaction structure, a multi-scale attention interaction module is added at the interaction node of the two branches, the structure of the multi-scale attention interaction module is composed of a multi-scale aggregation attention module, a gate mechanism and feature interaction, which aims to preprocess the saliency region features and saliency edge features by using attention and edge perception mechanisms, and then interact the two features to obtain edge-enhanced saliency region features and attention-enhanced edge features; S2. After the interaction and refinement of high-level salient region features and edge features by the multi-scale attention interaction module, they will gradually be fused with low-level salient region features and edge features, respectively. In the last two fusion stages of salient region features, i.e., the fusion stage of salient region features s 2 processed by the third multi-scale attention interaction module and the fusion stage of salient region features s 1 processed by the second multi-scale attention interaction module , a semantic guidance fusion module is introduced to enhance the propagation of semantic information, wherein the main steps of the semantic guidance fusion module are to perform semantic enhancement on the lower-level salient region features by using the highest-layer edge-enhanced salient region features, and then fuse the two enhanced features through channel concatenation. S3. Under the cooperation of the multi-scale attention interaction module and the semantic guidance fusion module, the high-level saliency region features and edge features are gradually interacted and refined, and are fused with the low-level saliency region features and edge features, and finally the refined saliency region feature map and edge feature map are obtained and edge feature map Then, the two feature maps are respectively input into a convolution layer to predict the saliency target and the saliency edge.

2. The multi-scale attention interaction module of claim 1, wherein: The saliency region features and edge features extracted by the interactive refinement encoder network are first preprocessed by the multi-scale attention interaction module, which adopts a multi-scale aggregation attention module and a gate mechanism to preprocess the saliency region features and edge features respectively, wherein the multi-scale aggregation attention module adopts six parallel convolution branches and adds top-down information propagation between the branches to capture different context information of the saliency region features, and the gate mechanism uses a sigmoid function to refine the edge features, these two preprocessing methods improve the quality of the saliency region features and edge features; Then, the preprocessed saliency region features and edge features are interactively fused through channel connection and element multiplication, and finally edge-enhanced saliency region features and attention-enhanced edge features are obtained.

3. The semantic guidance fusion module of claim 1, wherein: The semantic guidance fusion module injects semantic information into the two shallow features through upsampling and element multiplication of the deepest layer of interaction semantic features, and then fuses the two shallow features injected with semantic information, so as to effectively alleviate the problem of semantic information dilution.

Images