Optical remote sensing image saliency target detection method

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an optical remote sensing image saliency target detection method, which comprises a feature extractor PVT, three core modules and a decoder, the feature extractor PVT uses the PVT as an encoder of ADNet, the PVT is composed of four encoder blocks of different levels, and the decoder is composed of three core modules. The three core modules comprise a random weighted space attention module, a dynamic semantic fusion module and an adaptive edge enhancement module, parameters k2 and k3 of the dynamic semantic fusion module are adaptive convolution kernels dynamically generated by SKC, and features faeem, fdsfm and fswsam generated by the above modules are input into a significance feature decoder; the random weighted space attention module improves local detail information of advanced features through a local enhancement and weighted fusion mode, effectively overcomes the defects of PVT in local information processing, and enables the model to accurately position a significant target in an optical remote sensing image.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular is a method for detecting significant targets in optical remote sensing images. Background Art

[0002] Salient object detection aims to detect the most visually conspicuous objects or regions in an image. It is a key step in image segmentation and classification, and its main goal is to model detection based on human perception. It has a wide range of computer vision applications, such as human-computer interaction, visual tracking, image retrieval, object segmentation, and image compression. SOD contains two important categories, namely, salient object detection in natural scene images and salient object detection in optical remote sensing images.

[0003] In the past decade, the emergence of large-scale annotated datasets and the development of convolutional neural networks have greatly promoted the development of the field of salient object detection. Generally speaking, the existing salient object methods can be divided into two branches: traditional methods and deep learning methods. Traditional methods usually rely on hand-designed features and heuristic algorithms. These methods are based on low-level visual clues such as color, texture, and contrast, and determine salient areas by analyzing local and global features of the image. Among them, Wu et al. proposed a salient object detection method based on multi-scale frequency tuning features. Although traditional methods have a solid theoretical foundation, they may produce inaccurate detection results in the face of complex scenes, especially when there is a lot of background noise or complex structures. In contrast, deep learning methods are trained end-to-end. The method can automatically learn the high-level semantic features of images, has strong adaptability, and performs well in complex scenes, which significantly improves the detection performance. Among them, the representative ones are the CNN-based models ResNet and U2-Net, which have played an important role in the field of salient object detection. ResNet solves the problems of gradient vanishing and network degradation in deep networks by introducing residual structures, and has become a classic backbone network in multiple visual tasks. Especially in SOD tasks, ResNet is widely used as a basic feature extractor, which can capture multi-level visual features. U2-Net is famous for its double U-shaped network architecture. By combining shallow and deep features, it effectively extracts global semantic information and spatial details, and significantly improves the segmentation accuracy of salient objects in images.

[0004] Although these methods have surpassed most traditional methods and made significant progress on natural scene images, the detection of salient targets in optical remote sensing images is difficult due to their high resolution, multi-scale, complex background and various weather conditions. Existing deep learning models often have difficulty handling the details and complexity under these specific conditions.

[0005] Therefore, a salient target detection method for optical remote sensing images is proposed to address the above problems. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the shortcomings of the prior art, the present invention provides a method for detecting salient targets in optical remote sensing images, which solves the problem that the detection methods in the prior art are difficult to detect salient targets due to their high resolution, multi-scale, complex background and various weather conditions, and the existing deep learning models often find it difficult to handle the details and complexity under these specific conditions.

[0008] (II) Technical solution

[0009] To achieve the above object, the present invention provides the following technical solutions: a method for detecting significant targets in optical remote sensing images, comprising a feature extractor PVT, three core modules and a decoder;

[0010] Feature extractor PVT: PVT is used as the encoder of ADNet, and its input image size is set to 3×352×352. PVT consists of four different levels of Transformer encoder blocks, denoted as F i (i∈{1,2,3,4});

[0011] The three core modules include random weighted spatial attention module, dynamic semantic fusion module and adaptive edge enhancement module, and the steps are as follows:

[0012] Step 1: Transform the low-level features f 1 The input is sent to the adaptive edge enhancement module, which uses edge information extraction and the Sobel algorithm of the 8-directional template to enhance the expression of details and edge information and generate feature f aeem ;

[0013] Step 2: The intermediate level features f 2 and f 3 The input is sent to the dynamic semantic fusion module, which improves cross-scale semantic consistency by dynamically matching semantic information of different scales and using multi-scale channel attention mechanism to generate feature f dsfm ;

[0014] Step 3: Transform the high-level features f 4 The input is sent to the random weighted spatial attention module, which uses the spatial attention mechanism and random weighting mechanism to improve the model's attention to the key spatial area and generate the feature f swsam ;

[0015] The parameter k of the dynamic semantic fusion module 2 and k 3It is the adaptive convolution kernel dynamically generated by SKC;

[0016] The features f generated by the above module aeem 、f dsfm and f swsam The saliency features are input into the decoder, which generates the final saliency map by upsampling and fusing these features.

[0017] Preferably, the steps of the feature extractor PVT are:

[0018] S1: PVT extracts multi-scale features from the input image and generates four-level basic global features;

[0019] S2: Channel normalization is performed on these features, and the result is expressed as f i (i∈{1,2,3,4});

[0020] S3: Design different modules to handle f i .

[0021] Preferably, the randomly weighted spatial attention module takes the input feature f 4 Randomly shuffle the cards according to the channel dimension, and the feature representation after the shuffle operation is f s , and divide it into 4 subsets, each subset corresponds to a part of local features, denoted as f i s (i∈{1,2,3,4});

[0022] Use the spatial attention mechanism to process these sub-features separately and generate a spatial attention map a i (i∈{1,2,3,4}), where a i ∈(0,1), then, for a i Perform weighted summation, the formula is:

[0023]

[0024] Among them, w i (i∈{1,2,3,4}) is a learnable weight parameter;

[0025] to a w Perform convolution and sigmoid operations to obtain a detail-enhanced spatial attention map a s , the formula is:

[0026] a s =sigmoid(conv(a w ))

[0027] Among them, conv(·) is a normal convolutional layer, sigmoid(·) is the sigmoid activation function;

[0028] Using attention map a s Enhance the local details of high-level features and combine them with f s Fusion is performed to generate the output feature f of the randomly weighted spatial attention module swsam , the formula is:

[0029]

[0030] in, is element-wise multiplication, is element-wise addition.

[0031] Preferably, dynamic semantic fusion module: f swsam After the semantic knowledge compression module, the convolution kernel k is dynamically generated 2 and k 3 , the formula is:

[0032] k i =AP(DSConv(f swsam ))

[0033] Where i∈{2,3}, DSConv(·) is a depth-wise separable convolution, and AP(·) is an average pooling layer;

[0034] f 2 and f 3 After three different expansion rates (r 1 =1,r 2 =2,r 3 =3) and fuse them, where the convolution kernel of DDConv is k generated by SKC. 2 and k 3 ;

[0035] The fused features are convolved point by point, and the formula is:

[0036]

[0037] Among them, i∈{2,3}, r t ∈{1,2,3} is the expansion rate of DDConv;

[0038] right Perform depth-separable convolution to obtain features right First upsample and then use depth-separable convolution to get the features

[0039] This paper designs a multi-scale channel attention fusion module to enhance the expressiveness of feature fusion and improve the model's attention to the target area. The module introduces multi-scale information and combines the channel attention mechanism to achieve fine fusion of features of different scales. The module cascades and splices the enhanced features in the channel dimension to generate richer and more discernible feature representations. The formula is:

[0040]

[0041] Among them, Concat(·) means concatenating multiple input features in the channel dimension, and MSCAFM(·) is a multi-scale channel attention fusion module.

[0042] Preferably, the adaptive edge enhancement module uses a depth-separable convolution operation to perform an edge enhancement on the feature f 1 Perform preliminary processing to obtain features

[0043] The local edge features are extracted by average pooling operation, and the difference between the input feature and its pooled value is calculated to obtain the feature

[0044] The channel attention mechanism is introduced into the adaptive edge enhancement module. By calculating the adaptive weight, the formula is:

[0045]

[0046] in, is element-wise multiplication, is element-by-element addition, CA(·) is the channel attention mechanism; in particular, the adaptive edge enhancement module uses the Sobel algorithm of the 8-directional template to further extract the edge information of the image. The calculation formula is shown as follows:

[0047]

[0048] Among them, s i (i∈{1,2,3,4,5,6,7,8}) is the convolution result corresponding to the i-th direction template, and its formula is shown as follows:

[0049] s i =Conv(f 1 ,x i )

[0050] Among them, Conv(·) is a 3×3 convolution operation, x i (i∈{1,2,3,4,5,6,7,8}) is the convolution weight parameter corresponding to the i-th direction template.

[0051] Preferably, the decoder and loss function are: features are obtained from the random weighted spatial attention module, the dynamic semantic fusion module, and the adaptive edge enhancement module, respectively. swsam 、f dsfm and f aeem The resolutions of these features are 64×11×11, 128×44×44, and 128×88×88, respectively. These features are processed by the decoder to obtain the predicted image

[0052] After 4 times upsampling, the resolution is restored to the same as the input image to obtain the saliency map

[0053] P o Perform sigmoid operation to keep its value between (0,1) and obtain the saliency map P s ∈(0,1) 1×352×352 In the training phase, we use the combination of intersection-over-union (IoU) loss and binary cross entropy (BCE) loss as the total loss function of ADNet. The total loss function L total The calculation method is as follows:

[0054] L total =L iou (P s ,G)+L bce (P o ,G)

[0055] Among them, G∈{0,1} 1×352×352 is the true label, L iou is the IoU loss, L bce is the BCE loss, L iou and L bce The calculation formula is as follows:

[0056]

[0057]

[0058] Among them, P is the predicted image, G is the true label, N is the total number of pixels, and P i is the predicted value of the i-th pixel, G i ∈{0,1} is the true label value corresponding to the i-th pixel.

[0059] (III) Beneficial effects

[0060] Compared with the prior art, the present invention provides a method for detecting salient targets in optical remote sensing images, which has the following beneficial effects:

[0061] 1. This is a method for detecting salient targets in optical remote sensing images. The random weighted spatial attention module improves the local detail information of high-level features by local enhancement and weighted fusion, effectively supplementing the shortcomings of PVT in local information processing, enabling the model to accurately locate salient targets in optical remote sensing images.

[0062] 2. This is a method for salient object detection in optical remote sensing images. MSCAFM combines multi-scale convolution operations with channel attention mechanism to enhance the expressiveness of input features and context modeling capabilities.

[0063] 3. This is a method for detecting salient targets in optical remote sensing images. The adaptive edge enhancement module effectively extracts and enhances edge features by combining deep separable convolution with the Sobel algorithm of 8-directional templates. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0065] Figure 1 It is the ADNet structure diagram of the present invention;

[0066] Figure 2 It is the structural diagram of MSCAFM of the present invention;

[0067] Figure 3 It is an 8-directional template diagram of the Sobel algorithm of the present invention;

[0068] Figure 4 It is the PR curve diagram of the present invention;

[0069] Figure 5 It is a qualitative comparison diagram of ADNet of the present invention and other methods on the EORSSD dataset;

[0070] Figure 6 This is a qualitative comparison chart of ADNet of the present invention and other methods on the FORSSD dataset. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Specific examples are given below.

[0073] See also Figure 1-Figure 3 ,The present invention provides a method for detecting significant targets in optical remote sensing images, including a feature extractor PVT, three core modules and a decoder;

[0074] Feature extractor PVT: PVT is used as the encoder of ADNet, and its input image size is set to 3×352×352. PVT consists of four different levels of Transformer encoder blocks, denoted as F i (i∈{1,2,3,4});

[0075] The three core modules include random weighted spatial attention module, dynamic semantic fusion module and adaptive edge enhancement module, and the steps are as follows:

[0076] Step 1: Transform the low-level features f 1 The input is sent to the adaptive edge enhancement module, which uses edge information extraction and the Sobel algorithm of the 8-directional template to enhance the expression of details and edge information and generate feature f aeem ;

[0077] Step 2: The intermediate level features f 2 and f 3 The input is sent to the dynamic semantic fusion module, which improves cross-scale semantic consistency by dynamically matching semantic information of different scales and using multi-scale channel attention mechanism to generate feature f dsfm ;

[0078] Step 3: Transform the high-level features f 4 The input is sent to the random weighted spatial attention module, which uses the spatial attention mechanism and random weighting mechanism to improve the model's attention to the key spatial area and generate the feature f swsam ;

[0079] The parameter k of the dynamic semantic fusion module 2 and k 3 It is the adaptive convolution kernel dynamically generated by SKC;

[0080] The features f generated by the above modules are aeem 、f dsfm and f swsam The saliency features are input into the decoder, which generates the final saliency map by upsampling and fusing these features.

[0081] The steps of feature extractor PVT are:

[0082] S1: PVT extracts multi-scale features from the input image and generates four-level basic global features;

[0083] S2: Channel normalization is performed on these features, and the result is expressed as f i (i∈{1,2,3,4});

[0084] S3: Design different modules to handle f i .

[0085] The randomly weighted spatial attention module takes the input feature f 4 Randomly shuffle the cards according to the channel dimension, and the feature representation after the shuffle operation is f s , and divide it into 4 subsets, each subset corresponds to a part of local features, expressed as

[0086] Use the spatial attention mechanism to process these sub-features separately and generate a spatial attention map a i (i∈{1,2,3,4}), where a i ∈(0,1), then, for a i Perform weighted summation, the formula is:

[0087]

[0088] Among them, w i (i∈{1,2,3,4}) is a learnable weight parameter;

[0089] to a w Perform convolution and sigmoid operations to obtain a detail-enhanced spatial attention map a s , the formula is:

[0090] a s =sigmoid(conv(a w ))

[0091] Among them, conv(·) is a normal convolutional layer, sigmoid(·) is the sigmoid activation function;

[0092] Using attention map a s Enhance the local details of high-level features and combine them with f s Fusion is performed to generate the output feature f of the randomly weighted spatial attention module swsam , the formula is:

[0093]

[0094] in, is element-wise multiplication, is element-wise addition;

[0095] The random weighted spatial attention module improves the local detail information of high-level features through local enhancement and weighted fusion, effectively supplementing the shortcomings of PVT in local information processing, enabling the model to accurately locate salient targets in optical remote sensing images.

[0096] Dynamic semantic fusion module: f swsam After the semantic knowledge compression module, the convolution kernel k is dynamically generated 2 and k 3 , the formula is:

[0097] k i =AP(DSConv(f swsam ))

[0098] Where i∈{2,3}, DSConv(·) is a depth-wise separable convolution, and AP(·) is an average pooling layer;

[0099] f 2 and f 3 After three different expansion rates (r 1 =1,r 2 =2,r 3 =3) and fuse them, where the convolution kernel of DDConv is k generated by SKC. 2 and k 3 ;

[0100] The fused features are convolved point by point, and the formula is:

[0101]

[0102] Among them, i∈{2,3}, r t ∈{1,2,3} is the expansion rate of DDConv;

[0103] right Perform depth-separable convolution to obtain features right First upsample and then use depth-separable convolution to get the features

[0104] This paper designs a multi-scale channel attention fusion module to enhance the expressiveness of feature fusion and improve the model's attention to the target area. The module introduces multi-scale information and combines the channel attention mechanism to achieve fine fusion of features of different scales. The module cascades and splices the enhanced features in the channel dimension to generate richer and more discernible feature representations. The formula is:

[0105]

[0106] Among them, Concat(·) represents the concatenation of multiple input features in the channel dimension, and MSCAFM(·) is a multi-scale channel attention fusion module;

[0107] The dynamic semantic fusion module extracts salient information of features through average pooling and maximum pooling operations, and uses convolution kernels of different scales to extract multi-scale features.

[0108] The adaptive edge enhancement module uses a depth-wise separable convolution operation to perform an edge enhancement on the feature f 1 Perform preliminary processing to obtain features

[0109] The local edge features are extracted by average pooling operation, and the difference between the input feature and its pooled value is calculated to obtain the feature

[0110] The channel attention mechanism is introduced in the adaptive edge enhancement module. By calculating the adaptive weight, the formula is:

[0111]

[0112] in, is element-wise multiplication, is element-by-element addition, CA(·) is the channel attention mechanism; in particular, the adaptive edge enhancement module uses the Sobel algorithm of the 8-directional template to further extract the edge information of the image. The calculation formula is shown as follows:

[0113]

[0114] Among them, s i (i∈{1,2,3,4,5,6,7,8}) is the convolution result corresponding to the i-th direction template, and its formula is shown as follows:

[0115] s i =Conv(f 1 ,x i )

[0116] Among them, Conv(·) is a 3×3 convolution operation, x i (i∈{1,2,3,4,5,6,7,8}) is the convolution weight parameter corresponding to the i-th direction template;

[0117] The adaptive edge enhancement module performs a weighted fusion strategy on edge information from different sources through trainable weight coefficients, and further enhances the edge feature representation capability by means of convolution transformation, which first performs a dimensionality reduction operation and then a dimensionality increase operation on the channel dimension, and finally extracts an enhanced feature representation;

[0118] The adaptive edge enhancement module effectively extracts and enhances edge features by combining depthwise separable convolution with the Sobel algorithm of 8-directional templates.

[0119] Decoder and loss function: Features are obtained from the random weighted spatial attention module, the dynamic semantic fusion module, and the adaptive edge enhancement module, respectively. swsam 、f dsfm and f aeem The resolutions of these features are 64×11×11, 128×44×44, and 128×88×88, respectively. These features are processed by the decoder to obtain the predicted image

[0120] After 4 times upsampling, the resolution is restored to the same as the input image to obtain the saliency map

[0121] P o Perform sigmoid operation to keep its value between [0,1] and obtain the saliency map P s ∈[0,1] 1×352×352 In the training phase, we use the combination of intersection-over-union (IoU) loss and binary cross entropy (BCE) loss as the total loss function of ADNet. The total loss function L total The calculation method is as follows:

[0122] L total =L iou (P s ,G)+L bce (P o ,G)

[0123] Among them, G∈{0,1} 1×352×352 is the true label, L iou is the IoU loss, L bce is the BCE loss, L iou and L bce The calculation formula is as follows:

[0124]

[0125] Among them, P is the predicted image, G is the true label, N is the total number of pixels, and P i is the predicted value of the i-th pixel, G i ∈{0,1} is the true label value corresponding to the i-th pixel.

[0126] Comparative experiment

[0127] In this study, we compare the proposed model ADNet with seven other advanced methods, including MINet, F3Net, ICON, CorrNet, ACCoNet, SeaNet and TSCNet. CorrNet, ACCoNet, SeaNet and TSCNet are detection methods specially designed for remote sensing images, which have significant effects in processing ORSI-SOD tasks and are important components of the methods we compare.

[0128] Quantitative comparison

[0129] As shown in Table 1, ADNet is quantitatively compared with other advanced methods in multiple indicators. On the EORSSD dataset, ADNet ranks first in MAE and second in F β 、E φ and S α The overall performance is excellent. Specifically, ADNet is 1.64% lower than TSCNet in MAE. β 、E φ and S α The indicators are slightly inferior to TSCNet, but ADNet's excellent performance is achieved with a relatively moderate model parameter volume of 27.7M and floating point operation number of 23.8G, which is 73.5M and 93.6G lower than TSCNet, a decrease of 72.6% and 79.7%, which means that the computational cost of ADNet is much lower than TSCNet, but it can still be close to TSCNet in performance. Compared with other advanced models, ADNet shows excellent performance and exceeds these models in all indicators;

[0130] Table 1 Quantitative comparison of ADNet with 7 other advanced methods on the datasets EORSSD and FORSSD

[0131]

[0132] Note: ↑ The larger the indicator, the better. ↓ It means that the smaller the indicator is, the better. The three best results are marked with horizontal line, dotted line and wavy line respectively.

[0133] On the FORSSD dataset, ADNet showed excellent performance. β 、E φ , S α Compared with the second best model TSCNet, ADNet ranks first in F β 、E φ and S αThe indicators were improved by 3.20%, 1.40% and 1.65% respectively, and the indicators were significantly reduced by 26.14%. The FORSSD dataset contains a large number of remote sensing images processed by Gaussian noise, random contrast and random brightness, which makes the image background more complex and contains a variety of interference information. On the FORSSD dataset, ADNet's performance is more advantageous than that of the EORSSD dataset, and is better than TSCNet in overall effect. This result shows that ADNet has stronger discrimination and target segmentation capabilities when processing scene images with complex backgrounds and multi-scale targets, can effectively handle background noise, and carefully segment salient targets;

[0134] Figure 4 The PR curves of ADNet and other methods on the EORSSD and FORSSD datasets are shown. The PR curve of ADNet is close to the upper right corner, surpassing all other compared methods, indicating that our ADNet designed specifically for ORSI-SOD performs well in terms of precision and recall, which is highly consistent with the quantitative results in Table 1.

[0135] In general, our model has achieved excellent performance among all the compared methods. ADNet not only excels in performance, but also has significant advantages in model efficiency. Compared with the top models with large parameters and high computational complexity, ADNet has greatly reduced the complexity of the model while maintaining excellent performance. This makes ADNet achieve the best balance between performance and model complexity, and becomes an efficient and accurate salient object detection method when processing complex remote sensing image tasks.

[0136] Qualitative comparison

[0137] exist Figure 5 In, we show the qualitative comparison of ADNet with 7 other representative state-of-the-art methods on the EORSSD dataset, covering a variety of typical and challenging ORSI scenarios from the EORSSD set;

[0138] The first row shows multiple aircraft targets with different directions. ADNet successfully detects and accurately segments all significant targets and maintains the complete outline. Other methods have obvious shortcomings, such as TSCNet and SeaNet. Some aircraft wings are missing or incomplete, indicating that they are insufficient in dealing with multi-target tasks. The second row is a scene of a straight road. ADNet completely and accurately detects the entire road, while TSCNet and ICON mistakenly detect some road shadows as targets. In addition, SeaNet, ACCoNet and F3Net failed to fully detect the entire road and only presented partial fragments. CorrNet ​​and MINet even failed to detect significant road targets. The third row shows a ship target. The detection result of ADNet is basically complete. The other methods all have the problem of missing some areas of the ship target, which shows the superior ability of ADNet to retain the target shape. The fourth and fifth rows show the detection results of curved roads. ADNet can clearly detect the complete outline of the road. In contrast, other methods have significant defects, presenting intermittent detection results and unable to correctly outline the overall shape of the road. The sixth row is a scene of a circular roof. ADNet accurately highlights the most prominent part in the detection and effectively eliminates the interference of other buildings, while SeaNet and CorrNet ​​are affected by the surrounding buildings, resulting in redundant target misdetection. TSCNet, ACCoNet, ICON, F3Net and MINet even mistakenly detect another roof as a significant target, showing their limitations in dealing with complex scenes.

[0139] From the above qualitative comparison, it can be seen that ADNet performs superiorly on the EORSSD dataset and can more accurately detect complex and diverse salient targets. This is due to ADNet's powerful target positioning and detail retention capabilities, which effectively avoids background interference, target missing and edge blur.

[0140] exist Figure 6 In, we show the qualitative comparison of ADNet with 7 other representative state-of-the-art methods on the FORSSD dataset, covering a variety of challenging ORSI scenarios from the FORSSD set;

[0141] The first row shows the curved river target. ADNet successfully and completely detects the full picture of the river, maintaining the continuity and smoothness of the river curve, while other methods show defects to varying degrees. The detection results of SeaNet are not smooth enough relative to the true label, and there are irregular jumps on the edges of the curve. ACCoNet, CorrNet, ICON, F3Net and MINet mistakenly detect additional targets in areas outside the river, resulting in many irrelevant areas in the salient image. The second and third rows show the detection results of low-contrast rivers and ponds. In low-contrast scenes, ADNet can still accurately detect targets, demonstrating its advantages in processing low-contrast images. In contrast, other methods perform poorly in such scenes. For the river in the second row, TSCNet and ICON mistakenly regard other areas outside the river as salient targets, resulting in false detections. ACCoNet and CorrNet ​​cannot fully detect the full picture of the river, and there is a phenomenon of target loss. SeaNet, F3Net and MINet t even failed to detect any salient targets, resulting in missing results. In the detection of the pond in the third row, TSCNet completely failed to detect the pond, but instead misidentified other areas as salient targets. Although SeaNet and F3Net partially detected the pond, the detection results were incomplete. ACCoNet, CorrNet, ICON and MINet all failed to detect the pond, resulting in serious target omissions. The fourth and fifth rows show small salient targets with irregular shapes. ADNet can clearly detect these targets and accurately outline their complete outlines, reflecting its good ability to capture small targets. In contrast, the performance of other methods is unsatisfactory. Either there is unnecessary background interference in the detection results, or the detection results are incomplete. The sixth row shows the greenhouse target in the farmland. ADNet can not only accurately detect the salient target greenhouse, but also effectively eliminate the interference of the road next to it, ensuring the purity of the detection results. Other methods fail to distinguish the greenhouse from the road next to it, resulting in the inclusion of unnecessary road targets in the detection results and the lack of precision of the salient area.

[0142] From the above qualitative comparison, it can be seen that ADNet performs significantly better than other methods on the FORSSD dataset. It not only demonstrates excellent target detection capabilities under complex backgrounds, but also can accurately segment salient targets in low-contrast images. This performance is due to ADNet's powerful multi-scale feature extraction capabilities and effective attention mechanism, which can better cope with challenges in different scenarios and avoid the common problems of missed detection and false detection of other methods, especially in complex and detail-rich scenes.

[0143] Ablation experiment

[0144] In this section, we conduct a series of ablation experiments on ADNet on the EORSSD dataset to evaluate the effectiveness of several key modules in ADNet, namely the adaptive edge enhancement module, the dynamic semantic fusion module, and the random weighted spatial attention module. We measure the performance of each model from four evaluation indicators, including F β 、E φ , S α and MAE, where F β 、E φ and S α Higher values ​​indicate better model performance, while lower MAE values ​​indicate smaller errors and more accurate models;

[0145] Table 2 Ablation experiment results of ADNet on EORSSD dataset

[0146]

[0147] Note: √ means adding modules. ↑ The larger the indicator, the better. ↓ The smaller the indicator, the better, and the best result is marked in bold.

[0148] As shown in Table 2, when only the Baseline model is used, the four indicators of the model on the EORSSD dataset are: F β =0.8636, E φ =0.9428, S α =0.9223 and MAE=0.0086. The performance of this baseline model provides a reference for the subsequent experiments of adding modules. When adding individual modules gradually on the basis of the baseline, the model performance has been improved to varying degrees, indicating that each module has played a positive role in improving the detection performance. Among them, the performance of the model is the best when the random weighted spatial attention module is added, reaching F β =0.8672, E φ =0.9554, S α =0.9252 and MAE =0.0077, compared with the results of using only the Baseline model, F β Improved by 0.42%, E φ Increased by 1.34%, S α The performance is improved by 0.31% and the MAE is reduced by 10.47%, which shows that the random weighted spatial attention module significantly improves the detection accuracy and consistency of salient objects by capturing local spatial information. When the two modules are added to the baseline model, the performance is further improved, especially when the adaptive edge enhancement module and the random weighted spatial attention module are combined, the model performance reaches a new height: F β =0.8714, Eφ =0.9655, S α =0.9295, MAE = 0.0065, compared with the Baseline, F β Improved by 0.90%, E φ Improved by 2.41%, S α The accuracy was improved by 0.78% and the MAE was reduced by 24.42%, which shows that the adaptive edge enhancement module further improves the edge segmentation accuracy of the target by extracting and enhancing edge features, while the random weighted spatial attention module continues to strengthen the local features of the model. Finally, when the three core modules of the adaptive edge enhancement module, the dynamic semantic fusion module and the random weighted spatial attention module are all added to the model, the model achieves the best performance: F β =0.8728, E φ =0.9675, S α =0.9314, MAE = 0.0060, compared with the Baseline, F β Improved by 1.07%, E φ Improved by 2.62%, S α The accuracy was improved by 0.99% and the MAE was significantly reduced by 30.23%. This result shows that through the synergy of the adaptive edge enhancement module, the dynamic semantic fusion module and the random weighted spatial attention module, the model has achieved all-round performance improvement in detecting salient targets in optical remote sensing images.

[0149] From these quantitative analyses, it can be seen that the adaptive edge enhancement module, dynamic semantic fusion module and random weighted spatial attention module are crucial to improving the model performance. The adaptive edge enhancement module helps the model better handle complex target boundaries by enhancing the edge details of salient targets; the dynamic semantic fusion module improves the model's detection ability for targets of different scales and optimizes the model's semantic understanding ability by fusing multi-scale semantic information; the random weighted spatial attention module further improves the model's attention to local features through the spatial attention mechanism. The effective coordination of these modules enables the model to detect salient targets more accurately when dealing with complex optical remote sensing scenes, verifying its effectiveness and robustness in practical applications.

[0150] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A method for detecting salient objects in optical remote sensing images, characterized in that: It includes feature extractor PVT, three core modules and decoder; Feature extractor PVT: PVT is used as the encoder of ADNet, and its input image size is set to 3×352×352. PVT consists of four different levels of Transformer encoder blocks, denoted as F i (i∈{1,2,3,4}); The three core modules include random weighted spatial attention module, dynamic semantic fusion module and adaptive edge enhancement module, and the steps are as follows: Step 1: Input the low-level feature f1 into the adaptive edge enhancement module, and use the edge information extraction and the Sobel algorithm of the 8-directional template to enhance the expression of details and edge information to generate feature f aeem ; Step 2: Input the intermediate-level features f2 and f3 into the dynamic semantic fusion module, which generates feature f by dynamically matching semantic information of different scales and using a multi-scale channel attention mechanism to improve cross-scale semantic consistency. dsfm ; Step 3: Input the high-level feature f4 into the random weighted spatial attention module, which uses the spatial attention mechanism and random weighting mechanism to improve the model's attention to key spatial areas and generate feature f swsam ; The parameters k2 and k3 of the dynamic semantic fusion module are the adaptive convolution kernels dynamically generated by SKC; The features f generated by the above modules are aeem 、f dsfm and f swsam The saliency features are input into the decoder, which generates the final saliency map by upsampling and fusing these features.

2. The method for detecting salient objects in optical remote sensing images according to claim 1, characterized in that: The steps of feature extractor PVT are: S1: PVT extracts multi-scale features from the input image and generates four-level basic global features; S2: Channel normalization is performed on these features, and the result is expressed as f i (i∈{1,2,3,4}); S3: Design different modules to handle f i .

3. The method for detecting salient objects in optical remote sensing images according to claim 1, characterized in that: The random weighted spatial attention module randomly shuffles the input feature f4 according to the channel dimension. The feature after the shuffle operation is represented as f s , and divide it into 4 subsets, each subset corresponds to a part of local features, denoted as f i s (i∈{1,2,3,4}); Use the spatial attention mechanism to process these sub-features separately and generate a spatial attention map a i (i∈{1,2,3,4}), where a i ∈(0,1), then, for a i Perform weighted summation, the formula is: Among them, w i (i∈{1,2,3,4}) is a learnable weight parameter; to a w Perform convolution and sigmoid operations to obtain a detail-enhanced spatial attention map a s , the formula is: a s =sigmoid(conv(a w )) Among them, conv(·) is a normal convolutional layer, sigmoid(·) is the sigmoid activation function; Using attention map a s Enhance the local details of high-level features and combine them with f s Fusion is performed to generate the output feature f of the randomly weighted spatial attention module swsam , the formula is: in, is element-wise multiplication, is element-wise addition.

4. The method for detecting salient objects in optical remote sensing images according to claim 1, characterized in that: Dynamic semantic fusion module: f swsam After the semantic knowledge compression module, the convolution kernels k2 and k3 are dynamically generated, and the formula is: k i =AP(DSConv(f swsam )) Where i∈{2,3}, DSConv(·) is a depth-wise separable convolution, and AP(·) is an average pooling layer; f2 and f3 are subjected to dynamic depth convolution with three different dilation rates (r1=1, r2=2, r3=3) and fused, where the convolution kernels of DDConv are k2 and k3 generated by SKC; The fused features are convolved point by point, and the formula is: Among them, i∈{2,3}, r t ∈{1,2,3} is the expansion rate of DDConv; right Perform depth-separable convolution to obtain features right First upsample and then use depth-separable convolution to get the features This paper designs a multi-scale channel attention fusion module to enhance the expressiveness of feature fusion and improve the model's attention to the target area. The module introduces multi-scale information and combines the channel attention mechanism to achieve fine fusion of features of different scales. The module cascades and splices the enhanced features in the channel dimension to generate richer and more discernible feature representations. The formula is: Among them, Concat(·) means concatenating multiple input features in the channel dimension, and MSCAFM(·) is a multi-scale channel attention fusion module.

5. The method for detecting salient objects in optical remote sensing images according to claim 1, characterized in that: The adaptive edge enhancement module uses the depth-separable convolution operation to perform preliminary processing on the feature f1 to obtain the feature The local edge features are extracted by average pooling operation, and the difference between the input feature and its pooled value is calculated to obtain the feature The channel attention mechanism is introduced into the adaptive edge enhancement module. By calculating the adaptive weight, the formula is: in, is element-wise multiplication, is element-by-element addition, CA(·) is the channel attention mechanism; in particular, the adaptive edge enhancement module uses the Sobel algorithm of the 8-directional template to further extract the edge information of the image. The calculation formula is shown as follows: Among them, s i (i∈{1,2,3,4,5,6,7,8}) is the convolution result corresponding to the i-th direction template, and its formula is shown as follows: s i =Conv(f1,x i ) Among them, Conv(·) is a 3×3 convolution operation, x i (i∈{1,2,3,4,5,6,7,8}) is the convolution weight parameter corresponding to the i-th direction template.

6. The method for detecting salient objects in optical remote sensing images according to claim 1, characterized in that: Decoder and loss function: Features are obtained from the random weighted spatial attention module, the dynamic semantic fusion module, and the adaptive edge enhancement module, respectively. swsam 、f dsfm and f aeem The resolutions of these features are 64×11×11, 128×44×44, and 128×88×88, respectively. These features are processed by the decoder to obtain the predicted image After 4 times upsampling, the resolution is restored to the same as the input image to obtain the saliency map P o Perform sigmoid operation to keep its value between (0,1) and obtain the saliency map P s ∈(0,1) 1×352×352 In the training phase, we use the combination of intersection-over-union (IoU) loss and binary cross entropy (BCE) loss as the total loss function of ADNet. The total loss function L total The calculation method is as follows: L total =L iou (P s ,G)+L bce (P o ,G) Among them, G∈{0,1} 1×352×352 is the true label, L iou is the IoU loss, L bce is the BCE loss, L iou and L bce The calculation formula is as follows: Among them, P is the predicted image, G is the true label, N is the total number of pixels, and P i is the predicted value of the i-th pixel, G i ∈{0,1} is the true label value corresponding to the i-th pixel.

Images