A method for salient object detection in optical remote sensing images

Abstract

The present invention discloses a method for detecting salient targets in optical remote sensing images. The method extracts features of different scales of an optical remote sensing image to be detected, and then obtains edge features and main features of different scales through edge characterization and main body characterization. After main body decoding and edge decoding operations, fused main features and edge features are obtained, and interactive features of different scales are obtained through an interactive encoder. Main body decoding and edge decoding operations are performed here. After the fused features are connected, convolution and upsampling are performed to obtain the final salient target detection result. The present invention uses label decoupling to decouple the optical remote sensing image to obtain main labels and edge labels, respectively extract and strengthen main features and edge features, and supervise the results. And by iterating interactive information, a more accurate salient target detection result is obtained, while weakening the influence of cluttered background, the edge of the image is refined, and a better detection effect is achieved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a method for detecting salient targets in optical remote sensing images. Background Art

[0002] Salient object detection is to quickly and effectively extract the salient object area in the scene based on salient features such as spatial domain and frequency domain. In computer vision, salient object detection usually detects the most salient object first and then segments the accurate area of ​​the object. In recent years, salient object detection has been widely used in image and video segmentation, image relocation, image foreground annotation, thumbnail creation and image quality assessment, etc. In the field of computer vision, great progress has been made in the detection of salient objects in natural images; in contrast, salient object detection in optical remote sensing images still faces great challenges. Due to the significant differences between the scale, illumination and imaging direction of optical remote sensing images and natural images, it is difficult to directly transfer the salient object detection of natural images to the salient object detection of optical remote sensing images. Therefore, how to effectively improve the efficiency and accuracy of scene analysis and rapid object detection of large amounts of optical remote sensing data with cluttered backgrounds is crucial for further exploration of optical remote sensing images.

[0003] Compared with other tasks in computer vision, the development history of salient object detection in optical remote sensing images as a new research task is relatively short. In recent years, many technologies have been introduced into salient object detection in optical remote sensing images, such as multi-scale fusion, edge guidance, and complementary loss. For example, LVNet fuses multi-resolution inputs in a nested structure to perceive objects of different sizes, but the model has poor effect on edge refinement of images; PDFNet integrates five-scale features from five branches for comprehensive detection, but it is difficult to suppress the influence of cluttered background on the detection results; EMFINet uses three optical remote sensing images of different resolutions as input, and uses edge supervision to generate features with edge-aware constraints, and introduces a hybrid loss to infer salient objects with shape boundaries. Although the EMFINet model has a good detection effect, the computational complexity is high. These CNN-based salient object detection methods for optical remote sensing images have proposed effective solutions, but there is still room for improvement. Summary of the invention

[0004] The purpose of this application is to provide a method for detecting salient targets in optical remote sensing images, which supervises the model by decomposing salient labels into main maps and edge maps, and uses local content to enhance features and perform feature interaction, making full use of complementary information between branches. This salient target detection method can effectively distinguish foreground and background, not only overcoming the influence of cluttered background, but also refining the edges of the image.

[0005] In order to achieve the above purpose, the technical solution of this application is as follows:

[0006] A method for detecting salient objects in an optical remote sensing image, comprising:

[0007] Inputting the optical remote sensing image to be detected into the feature extraction module to extract features of different scales;

[0008] Input the features of different scales into the edge characterization module and the main characterization module corresponding to each scale respectively, and obtain the corresponding edge features of different scales and main features of different scales;

[0009] The main features and edge features of different scales are input into the main decoder and the edge decoder for decoding operations respectively, so as to obtain the main features and edge features after the first fusion;

[0010] The main features and edge features after the first fusion are connected to obtain the first main edge features, and the first main edge features are input into the interaction encoder to obtain interaction features of different scales;

[0011] The interactive features of different scales are input into the main decoder and the edge decoder, and the main decoder and the edge decoder decode again to output the main features and edge features after the second fusion, and the main features and edge features after the second fusion are connected to obtain the second main edge features;

[0012] The second subject edge feature is convolved and upsampled to obtain the final subject edge map as the final salient object detection result.

[0013] Furthermore, the features of different scales are respectively input into the edge characterization module and the main body characterization module corresponding to each scale to obtain the corresponding edge features of different scales and main body features of different scales, wherein the edge feature acquisition process is as follows:

[0014] Features of different scales are respectively compressed into refined features with the same number of channels and the original spatial scale by passing them through a layer of 1×1 convolution and a layer of 3×3 convolution.

[0015] Perform channel attention processing on the refined features to reduce redundant information in the features and obtain purified features;

[0016] Perform spatial attention processing on the purified features to obtain edge feature maps;

[0017] The purified features and edge feature maps are element-wise multiplied, and then two layers of 3×3 convolution are performed to obtain edge features of different scales;

[0018] The main feature acquisition process is as follows:

[0019] Features of different scales are respectively compressed into refined features with the same number of channels and the original spatial scale by passing them through a layer of 1×1 convolution and a layer of 3×3 convolution.

[0020] Perform channel attention processing on the refined features to reduce redundant information in the features and obtain purified features;

[0021] Perform spatial attention processing on the purified features to obtain the foreground feature map;

[0022] The purified features and foreground feature maps are element-wise multiplied, and then passed through two layers of 3×3 convolution to obtain main features of different scales.

[0023] Furthermore, the main features and edge features of different scales are respectively input into the main decoder and the edge decoder for decoding operations to obtain the main features and edge features after the first fusion, wherein the main decoder converts the input features of the previous and next scales into the same scale by upsampling, and then adds them, and refines the features through a layer of convolution; the edge decoder converts the input features of the previous and next scales into the same scale by upsampling, and then adds them, and refines the features through a layer of convolution.

[0024] Furthermore, the first subject edge feature is input into the interactive encoder to obtain interactive features of different scales, wherein the interactive encoder includes four layers, each layer of the interactive encoder is a layer of 3×3 convolution, and the first subject edge feature is input into the interactive encoder and passes through each layer of 3×3 convolution in turn to obtain interactive features of different scales.

[0025] Furthermore, the interactive features of different scales are input into the main decoder and the edge decoder, and the main decoder and the edge decoder decode again to output the main features and edge features after the second fusion, wherein the main decoder converts the input features of the previous and next scales into the same scale by upsampling, and then adds them to the input interactive features, and refines the features through a layer of convolution; the edge decoder converts the input features of the previous and next scales into the same scale by upsampling, and then adds them to the input interactive features, and refines the features through a layer of convolution.

[0026] Furthermore, the optical remote sensing image salient object detection method further includes:

[0027] During training, the distance transformation method is used to decouple the original labels of the training samples to obtain the main labels and edge labels;

[0028] Then, the main decoder and the edge decoder are decoded to obtain the main features and edge features after the first fusion, and then the main features and edge features are convolved once by 3×3, and then up-sampled to obtain the main image and edge image respectively;

[0029] According to the obtained main map and feature map, as well as the main label and edge label, the main map loss and edge map loss are calculated;

[0030] Based on the final subject edge map and the original label, the subject edge loss is calculated;

[0031] The joint loss including the main graph loss, edge graph loss and main edge loss is calculated, and back propagation is performed to update the network parameters.

[0032] Furthermore, the method of using distance transformation to decouple the original labels of the training samples to obtain the main labels and edge labels includes:

[0033] Get the original label of the training sample, whose label is a binary image I;

[0034] The binary image I is divided into two groups: foreground I fg and Background I bg , for each pixel p, I(p) is the value corresponding to each pixel, I fg (p) = 1, I bg (p) = 0;

[0035] Calculate the minimum distance between the pixels of the foreground and background. If pixel p is in the background, set its minimum distance to zero. If pixel p is in the foreground, find its nearest pixel q in the background through distance transformation, and then use the distance formula to calculate the distance between pixels p and q.

[0036] The calculated minimum distance between pixels is used as the pixel of the image to generate a new image I′;

[0037] Normalize the pixel values ​​in image I′ and map the pixel values ​​to [0,1];

[0038] Multiply the image I′ with the original image I to remove background interference and obtain the decoupled new label. The calculation formula is as follows:

[0039] BL=I×I′, DL=I×(1-I′)

[0040] Among them, BL is the main label and DL is the edge label.

[0041] This application proposes a method for detecting salient targets in optical remote sensing images. Label decoupling is used to decouple optical remote sensing images to obtain subject labels and edge labels. The subject features and edge features are respectively extracted and enhanced through a branch network. The obtained subject feature map and edge feature map supervise the results. And by iteratively interacting with information, more accurate salient target detection results are obtained. The technical solution of this application overcomes the shortcomings of the prior art, weakens the influence of cluttered background, refines the edges of the image, and achieves better detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of the method for detecting salient objects in optical remote sensing images of this application;

[0043] Figure 2 This is a network structure diagram of an embodiment of the present application;

[0044] Figure 3 This is a schematic diagram of an edge characterization module in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a main body depicting module of an embodiment of the present application;

[0046] Figure 5 Schematic diagram of the channel attention CA module;

[0047] Figure 6 Schematic diagram of the spatial attention SA module. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The present application provides a method for detecting salient targets in optical remote sensing images, which is based on label decoupling and feature interaction, and fully utilizes the advantages of a branch network and a multi-scale information interaction fusion method to achieve accurate salient target detection in optical remote sensing images.

[0050] In one embodiment, Figure 1 As shown in FIG. 1 , a method for detecting salient targets in optical remote sensing images is proposed, including:

[0051] Step S1: input the optical remote sensing image to be detected into a feature extraction module to extract features of different scales.

[0052] In this embodiment, the optical remote sensing image to be detected is unified in size, for example, unified to an optical remote sensing image of 256×256×3, and then

[0053] ResNet-50 is used as the feature extraction module to extract features of the input 256×256×3 optical remote sensing image, and features of five scales are extracted. The five scales are 128×128×64, 64×64×256, 32×32×512, 16×16×1024, and 8×8×2048.

[0054] In this example, ResNet-50 is used to extract features from the input 256×256×3 optical remote sensing image in order to extract features of different scales. The high-level network has a larger receptive field and a strong ability to represent semantic information, which can help us accurately detect targets. The low-level network has a smaller receptive field and a strong ability to represent geometric detail information, which helps us refine edge details.

[0055] like Figure 2 As shown, ResNet-50 outputs features of five scales, namely 128×128×64, 64×64×256, 32×32×512, 16×16×1024, and 8×8×2048. Figure 2 Medium E 1 、E 2 、E 3 、E 4 and E 5 .

[0056] Step S2: Input the features of different scales into the edge characterization module and the main characterization module corresponding to each scale, and obtain the corresponding edge features of different scales and main features of different scales.

[0057] In this example, Figure 2 As shown, the feature outputs of five different scales are respectively input into the edge characterization module and the main body characterization module corresponding to each scale to obtain the corresponding edge features of different scales and main features of different scales.

[0058] The edge characterization module of this embodiment is as follows Figure 3 As shown in the figure, for features of different scales, channel compression is first performed to facilitate subsequent feature fusion, edge features are extracted using channel attention and spatial attention, and two layers of convolution are used for refinement. The operation is as follows:

[0059] Step 211: Features of different scales are respectively compressed into refined features with the same number of channels and the original spatial scale by passing them through a layer of 1×1 convolution and a layer of 3×3 convolution.

[0060] In this embodiment, features of different scales are Through a layer of 1×1 convolution and a layer of 3×3 convolution, the features are refined and compressed to 64 channels while maintaining the original spatial scale. The output feature scales are 128×128×64, 64×64×64, 32×32×64, 16×16×64, and 8×8×64, respectively.

[0061] Step 212: Perform channel attention processing on the refined features to reduce redundant information in the features and obtain purified features.

[0062] After refinement, the features Channel attention processing is performed to reduce redundant information in features and purify features. The calculation formula is as follows:

[0063]

[0064] in, is the purified feature, CA(·) is the channel attention, and ⊙ is the channel multiplication.

[0065] Step 213: Perform spatial attention processing on the purified features to obtain an edge feature map.

[0066] The purified features Through spatial attention processing, we get the edge feature map and use The calculation formula is as follows:

[0067]

[0068] Among them, SA(·) is spatial attention. Figure 3 D in the figure represents the extraction of edge features.

[0069] Step 214: perform element-wise multiplication on the purified features and the edge feature map, and then perform two layers of 3×3 convolution to obtain edge features of different scales.

[0070] This embodiment has the following features and Perform element multiplication to highlight the features of the salient areas. The calculation formula is as follows:

[0071]

[0072] in, is element-wise multiplication.

[0073] The features that will be obtained in the end Through two layers of 3×3 convolution, the features are further refined to generate edge features of different scales.

[0074] It should be noted that the channel attention module CA is Figure 5As shown, the spatial attention module SA is Figure 6 As shown, these are relatively mature technologies in this technical field and will not be described in detail here.

[0075] In this embodiment, Figure 4 As shown in the figure, for features of different scales, the channels are compressed first to facilitate subsequent feature fusion, the main features are extracted using channel attention and spatial attention, and two layers of convolution are used for refinement. The operation is as follows:

[0076] Step 221: The features of different scales are respectively compressed into refined features with the same number of channels and the original spatial scale by passing the features of different scales through a layer of 1×1 convolution and a layer of 3×3 convolution.

[0077] In this embodiment, the obtained features Through a layer of 1×1 convolution and a layer of 3×3 convolution, the features are refined and compressed to 64 channels while maintaining the original spatial scale. The output feature scales are 128×128×64, 64×64×64, 32×32×64, 16×16×64, and 8×8×64, respectively.

[0078] Step 222: Perform channel attention processing on the refined features to reduce redundant information in the features and obtain purified features.

[0079] After refinement, the features Perform channel attention processing to reduce redundant information in features, purify features, and obtain features

[0080] Step 223: Perform spatial attention processing on the purified features to obtain a foreground feature map.

[0081] The purified features Through spatial attention processing, we get the foreground feature map and use The calculation formula is as follows:

[0082]

[0083] Where SA(·) is the spatial attention. Figure 4 As shown, F represents the extraction of subject features.

[0084] Step 224: perform element-wise multiplication of the purified features and the foreground feature map, and then pass through two layers of 3×3 convolution to obtain subject features of different scales.

[0085] The characteristics and Perform element multiplication to highlight the features of the salient areas. The calculation formula is as follows:

[0086]

[0087] in, is element multiplication; the features obtained are Through two layers of 3×3 volumes, the features are further refined to generate main features of different scales

[0088] Step S3: inputting the main features and edge features of different scales into the main decoder and edge decoder for decoding operation respectively, so as to obtain the main features and edge features after the first fusion.

[0089] In this example, the main feature Input main decoder, such as Figure 2 As shown in the figure, the main decoder converts the input features of the previous and next scales into the same scale through upsampling, then adds them together and refines the features through a layer of convolution.

[0090] In this embodiment, the main decoder performs a decoding operation, and the operation process is as follows:

[0091] Step S311, upsampling the features with a scale of 8×8×64 output by the subject characterization module, adding the upsampled features to the features with a scale of 16×16×64 output by the subject characterization module, and then refining the features through a layer of convolution to obtain the fused 16×16×64 subject features;

[0092] Step S312, upsampling the fused features with a scale of 16×16×64, adding the upsampled features to the features with a scale of 32×32×64 output by the subject characterization module, and then refining the features through a layer of convolution to obtain fused subject features of 32×32×64;

[0093] Step S313, upsampling the fused features with a scale of 32×32×64, adding the upsampled features to the features with a scale of 64×64×64 output by the subject characterization module, and then refining the features through a layer of convolution to obtain fused subject features with a scale of 64×64×64;

[0094] Step S314, up-sample the fused features with a scale of 64×64×64, add the up-sampled features to the features with a scale of 128×128×64 output by the subject characterization module, and then refine the features through a layer of convolution to obtain the fused subject features of 128×128×64.

[0095] In this example, the edge features Input edge decoder, such as Figure 2As shown in the figure, the edge decoder converts the input features of the previous and next scales into the same scale by upsampling, then adds them together and refines the features through a layer of convolution.

[0096] In this embodiment, the edge decoder performs a decoding operation, and the operation process is as follows:

[0097] Step S311, upsampling the features with a scale of 8×8×64 output by the edge characterization module, adding the upsampled features to the features with a scale of 16×16×64 output by the edge characterization module, and then refining the features through a layer of convolution to obtain fused edge features of 16×16×64;

[0098] Step S312, upsampling the fused features with a scale of 16×16×64, adding the upsampled features to the features with a scale of 32×32×64 output by the edge characterization module, and then refining the features through a layer of convolution to obtain fused edge features of 32×32×64;

[0099] Step S313, upsampling the fused features with a scale of 32×32×64, adding the upsampled features to the features with a scale of 64×64×64 output by the edge characterization module, and then refining the features through a layer of convolution to obtain fused edge features of 64×64×64;

[0100] Step S314, up-sample the fused features with a scale of 64×64×64, add the up-sampled features to the features with a scale of 128×128×64 output by the edge characterization module, and then refine the features through a layer of convolution to obtain the fused edge features of 128×128×64.

[0101] Step S4: Connect the main features and edge features after the first fusion to obtain a first main edge feature, and input the first main edge feature into each scale interaction encoder to obtain interaction features of different scales.

[0102] Generally speaking, after the main decoder and the edge decoder are fused, the main features and edge features of the scale of 128×128×64 are connected to obtain preliminary main edge features, that is, the first main edge features. At this time, the output preliminary main edge features are passed through two layers of 3×3 convolution and upsampled to obtain the main edge map of the first iteration output. However, generally speaking, the main edge map of the first iteration is quite different from the actual label, so this application enhances the edge features of the output results through the information interaction module.

[0103] like Figure 2As shown, the interactive encoder of this embodiment includes four layers, each of which is a 3×3 convolution layer. The first main edge feature is input into the interactive encoder and sequentially passes through each layer of 3×3 convolution to obtain interactive features of different scales. The interactive features of different scales output by each layer of interactive encoder are returned to the corresponding main decoder and edge decoder, further promoting the interaction of information.

[0104] Step S5: input the interactive features of different scales into the main decoder and the edge decoder, the main decoder and the edge decoder decode again to output the main features and edge features after the second fusion, and connect the main features and edge features after the second fusion to obtain the second main edge features.

[0105] like Figure 2 As shown, unlike the decoding operation in step S3, the interactive features are incorporated here. Figure 2 In the main decoder and edge decoder, the dotted part refers to the input interaction features, that is, after upsampling the input features of the previous and next scales to the same scale and then adding them, the interaction features are also added. Thus, the main features and edge features after the second fusion are obtained.

[0106] Then, the main features and edge features fused by the main decoder and the edge decoder are connected to obtain the second main edge features.

[0107] For example, for the body decoder, include:

[0108] The features with a scale of 8×8×64 output by the main characterization module are upsampled, and the upsampled features are added to the features with a scale of 16×16×64 output by the main characterization module. The addition also includes 16×16×64 interactive features. Then, a layer of convolution is performed to refine the features to obtain the fused 16×16×64 main features.

[0109] The fused features with a scale of 16×16×64 are upsampled, and the upsampled features are added to the features with a scale of 32×32×64 output by the main characterization module. The addition also includes 32×32×64 interactive features. Then, a layer of convolution is performed to refine the features to obtain the fused main features of 32×32×64.

[0110] The fused features with a scale of 32×32×64 are upsampled, and the upsampled features are added to the features with a scale of 64×64×64 output by the main characterization module. The addition also includes 64×64×64 interactive features. Then, a layer of convolution is performed to refine the features to obtain the fused main features of 64×64×64.

[0111] The fused features with a scale of 64×64×64 are upsampled and added to the features with a scale of 128×128×64 output by the main characterization module. The addition also includes 128×128×64 interactive features. Then, a layer of convolution is performed to refine the features and obtain the fused main features of 128×128×64.

[0112] For the edge decoder, similar operations are performed and will not be described in detail here.

[0113] Step S6: Convolve and upsample the second subject edge feature to obtain a final subject edge map as the final salient object detection result.

[0114] In this step, the second subject edge feature is passed through two layers of 3×3 convolution and up-sampled to obtain the final output subject edge map. The output subject edge map is the final salient object detection result.

[0115] In this example, the interactive encoder is used to fuse the main features and edge features, and then fed back to the main encoder and edge encoder for secondary iteration. The output of the secondary iteration will have clearer edge features and be more consistent with the actual labels.

[0116] This embodiment uses a multi-branch fusion network to separate the main body and the edge for multi-scale feature extraction and fusion, which is conducive to the edge characterization of salient targets. In the example, a label decoupling method is introduced, which decouples the optical remote sensing image label and uses the distance transformation method to decouple the original label into the main body label and the edge label. The decoupled label is conducive to the supervision and evaluation of the model.

[0117] This embodiment designs a main body characterization module and a detail characterization module, mainly using spatial attention and channel attention, clearly characterizing the main features and edge features, which is conducive to highlighting the main features and edges. In this module, a jump connection is used to well integrate texture information and semantic information.

[0118] In another embodiment, in order to better refine the edge of the optical remote sensing image, the main body and edge features of the detection target can be processed and supervised separately. That is, the optical remote sensing image salient target detection method of the present application also includes:

[0119] During training, the distance transformation method is used to decouple the original labels of the training samples to obtain the main labels and edge labels;

[0120] Then, the main decoder and the edge decoder are decoded to obtain the main features and edge features after the first fusion, and then the main features and edge features are convolved once by 3×3, and then up-sampled to obtain the main image and edge image respectively;

[0121] According to the obtained main map and feature map, as well as the main label and edge label, the main map loss and edge map loss are calculated;

[0122] Based on the final subject edge map and the original label, the subject edge loss is calculated;

[0123] The joint loss including the main graph loss, edge graph loss and main edge loss is calculated, and back propagation is performed to update the network parameters.

[0124] In this embodiment, the original labels of the training samples are decoupled into main labels and edge labels. The label decoupling method provided in this embodiment specifically includes:

[0125] Get the original label of the training sample, whose label is a binary image I;

[0126] The binary image I is divided into two groups: foreground I fg and Background I bg , for each pixel p, I(p) is the value I corresponding to each pixel fg (p) = 1, I bg (p) = 0;

[0127] Calculate the minimum distance between the pixels of the foreground and background. If pixel p is in the background, set its minimum distance to zero. If pixel p is in the foreground, find its nearest pixel q in the background through distance transformation, and then use the distance formula to calculate the distance between pixels p and q.

[0128] The calculated minimum distance between pixels is used as the pixel of the image to generate a new image I′;

[0129] Normalize the pixel values ​​in image I′ and map the pixel values ​​to [0,1];

[0130] Multiply the image I′ with the original image I to remove background interference and obtain the decoupled new label. The calculation formula is as follows:

[0131] BL=I×I′, DL=I×(1-I′)

[0132] Among them, BL is the main label and DL is the edge label.

[0133] The label decoupling method of this embodiment aims to solve the problem of edge prediction and improve the efficiency and accuracy of salient target detection. The original label is decoupled into the main label and the edge label. The edge label helps the model learn better edge features, and the main label reduces the dispersion from near-edge pixels to center pixels. Since the background of optical remote sensing images is cluttered, pixels close to the edge are difficult to detect. Therefore, the main and edge features of the detection target can be processed and supervised separately to improve the accuracy of salient target detection.

[0134] In this embodiment, the main image loss is calculated based on the obtained main image and feature image, as well as the main label and edge label. and edge graph loss Specifically, binary cross entropy (BCE) is used to calculate the loss between the main graph and the edge graph and the decoupled main label and edge label. BCE is a widely used loss in classification. The calculation formula is as follows:

[0135]

[0136] Among them, g(x,y)∈[0,1] is the decoupled main label or edge label, and p(x,y)∈[0,1] is the main map or edge map. The loss is calculated separately to obtain and Among them l (k) is the loss at the kth iteration, is the main image loss, is the edge map loss.

[0137] This embodiment calculates the subject edge loss based on the final subject edge map and the original label. Specifically, it is calculated using IoU loss. IoU loss is mainly used to measure the overall similarity of two images. The calculation formula is as follows:

[0138]

[0139] Among them, g(x,y)∈[0,1] is the original label of the training sample, and p(x,y)∈[0,1] is the final subject edge map, which is the salient target detection result finally output by the network.

[0140] Finally, the sum of the above three losses is used as the joint loss, and directional propagation is performed to update the parameters of the trained network to complete the training of the network. The training of neural networks is already a relatively mature technology in this field and will not be described here.

[0141] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting salient objects in optical remote sensing images, characterized in that: The optical remote sensing image salient object detection method comprises: Inputting the optical remote sensing image to be detected into the feature extraction module to extract features of different scales; Input the features of different scales into the edge characterization module and the main characterization module corresponding to each scale respectively, and obtain the corresponding edge features of different scales and main features of different scales; The main features and edge features of different scales are input into the main decoder and the edge decoder for decoding operations respectively, so as to obtain the main features and edge features after the first fusion; The main features and edge features after the first fusion are connected to obtain the first main edge features, and the first main edge features are input into the interaction encoder to obtain interaction features of different scales; The interactive features of different scales are input into the main decoder and the edge decoder, and the main decoder and the edge decoder decode again to output the main features and edge features after the second fusion, and the main features and edge features after the second fusion are connected to obtain the second main edge features; Convolve and upsample the second subject edge feature to obtain the final subject edge map as the final salient object detection result; The features of different scales are respectively input into the edge characterization module and the main body characterization module corresponding to each scale to obtain the corresponding edge features of different scales and main body features of different scales, wherein the edge feature acquisition process is as follows: Features of different scales are respectively compressed into refined features with the same number of channels and the original spatial scale by passing them through a layer of 1×1 convolution and a layer of 3×3 convolution. Perform channel attention processing on the refined features to reduce redundant information in the features and obtain purified features; Perform spatial attention processing on the purified features to obtain edge feature maps; The purified features and edge feature maps are element-wise multiplied, and then two layers of 3×3 convolution are performed to obtain edge features of different scales; The main feature acquisition process is as follows: Features of different scales are respectively compressed into refined features with the same number of channels and the original spatial scale by passing them through a layer of 1×1 convolution and a layer of 3×3 convolution. Perform channel attention processing on the refined features to reduce redundant information in the features and obtain purified features; Perform spatial attention processing on the purified features to obtain the foreground feature map; The purified features and foreground feature maps are element-wise multiplied, and then passed through two layers of 3×3 convolution to obtain main features of different scales; The main features and edge features of different scales are respectively input into the main decoder and the edge decoder for decoding operations to obtain the main features and edge features after the first fusion, wherein the main decoder converts the input features of the previous and next scales into the same scale by upsampling, then adds them, and refines the features through a layer of convolution; the edge decoder converts the input features of the previous and next scales into the same scale by upsampling, then adds them, and refines the features through a layer of convolution; The first subject edge feature is input into the interactive encoder to obtain interactive features of different scales, wherein the interactive encoder includes four layers, each layer of the interactive encoder is a layer of 3×3 convolution, and the first subject edge feature is input into the interactive encoder and sequentially passes through each layer of 3×3 convolution to obtain interactive features of different scales; The interactive features of different scales are input into the main decoder and the edge decoder, and the main decoder and the edge decoder perform decoding operations again to output the main features and edge features after the second fusion, wherein the main decoder converts the input features of the previous and next scales into the same scale by upsampling, and then adds them to the input interactive features, and refines the features through a layer of convolution; the edge decoder converts the input features of the previous and next scales into the same scale by upsampling, and then adds them to the input interactive features, and refines the features through a layer of convolution.

2. The optical remote sensing image salient object detection method according to claim 1, characterized in that: The optical remote sensing image salient object detection method further includes: During training, the distance transformation method is used to decouple the original labels of the training samples to obtain the main labels and edge labels; Then, the main decoder and the edge decoder are decoded to obtain the main features and edge features after the first fusion, and then the main features and edge features are convolved once by 3×3, and then up-sampled to obtain the main image and edge image respectively; According to the obtained main map and feature map, as well as the main label and edge label, the main map loss and edge map loss are calculated; Calculate the subject edge loss based on the final subject edge map and the original label; The joint loss including the main graph loss, edge graph loss and main edge loss is calculated, and back propagation is performed to update the network parameters.

3. The optical remote sensing image salient object detection method according to claim 2, characterized in that: The method of using distance transformation to decouple the original labels of the training samples to obtain the main labels and edge labels includes: Get the original label of the training sample, whose label is a binary image I; The binary image I is divided into two groups: foreground I fg and Background I bg , for each pixel p, I(p) is the value corresponding to each pixel, I fg (p) = 1, I bg (p) = 0; Calculate the minimum distance between the pixels of the foreground and background. If pixel p is in the background, set its minimum distance to zero. If pixel p is in the foreground, find its nearest pixel q in the background through distance transformation, and then use the distance formula to calculate the distance between pixels p and q. The calculated minimum distance between pixels is used as the pixel of the image to generate a new image I′; Normalize the pixel values ​​in image I′ and map the pixel values ​​to [0, 1]; Multiply the image I′ with the original image I to remove background interference and obtain the decoupled new label. The calculation formula is as follows: BL=I×I′, DL=I×(1-I′) Among them, BL is the main label and DL is the edge label.

Images