Salient target detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a saliency target detection method and device, electronic equipment and a storage medium. The saliency target detection method comprises the steps of obtaining a target optical remote sensing image; detecting a salient target in the target optical remote sensing image by using the target network model; wherein the target network model comprises a feature extraction part, a multi-level semantic enhancement part, a boundary guiding part and a fusion part; the feature extraction part is used for extracting multi-level features of the target optical remote sensing image; the multi-level semantic enhancement part is used for obtaining semantic attention information features based on high-level features in the multi-level features; the boundary guiding part is used for performing boundary guiding based on the multi-level features to obtain boundary guiding features; and the fusion part is used for fusing the semantic attention information features and the boundary guide features to obtain a detection result of the saliency target. By applying the technical scheme provided by the invention, the detection efficiency of the salient target in the target optical remote sensing image can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and in particular to a method, device, electronic device and storage medium for detecting a significant target. Background Art

[0002] Remote sensing technology, in a broad sense, refers to a method of obtaining target state information by indirect means (without physical contact with the object). Optical remote sensing images have been widely used in many fields, such as land planning, disaster monitoring, military reconnaissance, etc., because they can provide large-scale, high-resolution, rich content, and intuitive image information. Significant target detection in optical remote sensing images has practical value and can assist various downstream visual applications in remote sensing scenes, such as image fusion, scene classification, and object detection.

[0003] Salient object detection aims to extract visually attractive objects or regions from the entire field of view, with the goal of mimicking the inherent visual attention mechanism of humans. Faced with large-scale and challenging optical remote sensing images, only some local areas, such as man-made objects, islands, river systems, etc., may attract human attention. Salient object detection in optical remote sensing images focuses on detecting these prominent objects or regions in the scene.

[0004] Optical remote sensing images are usually obtained by taking aerial photos from satellites or airplanes. The shooting process is easily affected by environmental factors such as shooting altitude, shooting time and light intensity, resulting in complex image backgrounds, large object scale variations, and varying degrees of noise interference. These factors make it unreliable to directly apply natural image salient object detection methods to optical remote sensing images for salient object detection, and the detection efficiency is low.

[0005] Therefore, how to detect significant targets in optical remote sensing images and improve detection efficiency is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0006] The purpose of this application is to provide a significant target detection method, device, electronic device and storage medium to achieve significant target detection in optical remote sensing images and improve detection efficiency.

[0007] In order to solve the above technical problems, this application provides the following technical solutions:

[0008] In a first aspect, a salient object detection method is provided, comprising:

[0009] Obtaining target optical remote sensing images;

[0010] Detecting salient targets in the target optical remote sensing image using a target network model;

[0011] The target network model includes a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part, and a fusion part;

[0012] The feature extraction part is used to extract multi-level features of the target optical remote sensing image;

[0013] The multi-level semantic enhancement part is used to generate global semantic information based on high-level features in the multi-level features, and use the global semantic information to perform semantic refinement to obtain semantic attention information features;

[0014] The boundary guiding part is used to perform boundary guiding based on the multi-level features to obtain boundary guiding features;

[0015] The fusion part is used to fuse the semantic attention information feature and the boundary guidance feature to obtain a detection result of a salient target.

[0016] Optionally, the feature extraction part includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a fifth convolution block stacked in sequence, and the boundary guidance part includes a first boundary guidance module, a second boundary guidance module and a third boundary guidance module;

[0017] The third convolution block is also connected to the multi-level semantic enhancement part and the first boundary guidance module respectively, the fourth convolution block is also connected to the multi-level semantic enhancement part and the second boundary guidance module respectively, and the fifth convolution block is also connected to the multi-level semantic enhancement part and the third boundary guidance module respectively.

[0018] Optionally, the multi-level semantic enhancement part is specifically used for:

[0019] Upsampling the features obtained from the fifth convolutional block, and obtaining first attention map features using a spatial attention mechanism;

[0020] Performing multi-scale progressive fusion processing on the features obtained from the third convolution block, the fourth convolution block, and the fifth convolution block to obtain a second attention map feature;

[0021] The first attention map feature and the second attention map feature are fused to obtain a semantic attention information feature.

[0022] Optionally, the multi-level semantic enhancement part is specifically used for:

[0023] Upsampling the features obtained from the fifth convolution block to obtain upsampled features, where the spatial resolution of the upsampled features is the same as the spatial resolution of the features output by the third convolution block;

[0024] Using average pooling and maximum pooling of the upsampled features to form two single-channel mappings, and performing feature concatenation along the channel direction to generate dual-channel features;

[0025] Based on the convolution operation on the dual-channel features, a first attention map feature is obtained.

[0026] Optionally, the multi-level semantic enhancement part is specifically used for:

[0027] Using an extended convolution layer with a first expansion rate, the feature obtained from the third convolution block is expanded to obtain a first extended feature;

[0028] Using an extended convolution layer with a second expansion rate, the feature obtained from the fourth convolution block is expanded to obtain a second extended feature;

[0029] Using an extended convolution layer with a third expansion rate, the features obtained from the fifth convolution block are expanded to obtain third extended features;

[0030] The third extended feature is upsampled, fused with the second extended feature, and then fused with the first extended feature to obtain a second attention map feature.

[0031] Optionally, the second convolution block is connected to the first boundary guiding module, the second boundary guiding module and the third boundary guiding module respectively.

[0032] Optionally, the target boundary guidance module included in the boundary guidance part is specifically used for:

[0033] Performing a first convolution operation on the features obtained from the second convolution block to obtain boundary features;

[0034] Upsampling the features obtained from the target convolution block to obtain a first mask, wherein the target convolution block is a convolution block other than the second convolution block connected to the target boundary guidance module;

[0035] Merging the first mask with the boundary feature to obtain a first guiding feature;

[0036] Performing a second convolution operation on the boundary feature to obtain a second mask;

[0037] fusing the second mask with the feature obtained from the target convolution block to obtain a second guided feature;

[0038] The first guide feature and the second guide feature are merged to obtain a boundary guide feature corresponding to the target boundary guide module;

[0039] The target boundary guiding module is the first boundary guiding module, the second boundary guiding module, or the third boundary guiding module.

[0040] Optionally, the fusion part is specifically used for:

[0041] Fusing the semantic attention information feature with the boundary guidance feature corresponding to the third boundary guidance module to obtain a first fused feature;

[0042] Fusing the first fused feature with the boundary guidance feature corresponding to the second boundary guidance module to obtain a second fused feature;

[0043] Fusing the second fused feature with the boundary guidance feature corresponding to the first boundary guidance module to obtain a third fused feature;

[0044] Based on the third fusion feature, a detection result of a salient object is obtained.

[0045] Optionally, the target network model is pre-trained based on a loss function, wherein the loss function includes boundary loss, intersection-over-union loss and binary cross entropy loss.

[0046] In a second aspect, a salient object detection device is provided, comprising:

[0047] An acquisition module, used for acquiring a target optical remote sensing image;

[0048] A detection module, used to detect salient targets in the target optical remote sensing image using a target network model;

[0049] The target network model includes a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part, and a fusion part;

[0050] The feature extraction part is used to extract multi-level features of the target optical remote sensing image;

[0051] The multi-level semantic enhancement part is used to generate global semantic information based on high-level features in the multi-level features, and use the global semantic information to perform semantic refinement to obtain semantic attention information features;

[0052] The boundary guiding part is used to perform boundary guiding based on the multi-level features to obtain boundary guiding features;

[0053] The fusion part is used to fuse the semantic attention information feature and the boundary guidance feature to obtain a detection result of a salient target.

[0054] In a third aspect, an electronic device is provided, including:

[0055] Memory for storing computer programs;

[0056] A processor is used to implement the steps of the salient object detection method as described in the first aspect when executing the computer program.

[0057] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the salient object detection method as described in the first aspect are implemented.

[0058] In a fifth aspect, a computer program product is provided, which includes computer instructions, which are stored in a computer-readable storage medium and are suitable for being read and executed by a processor, so that a computer device having the processor performs the steps of the salient target detection method as described in the first aspect.

[0059] By applying the technical solution provided in the embodiment of the present application, after obtaining the target optical remote sensing image, the target network model is used to detect the salient targets in the target optical remote sensing image, and the target network model includes a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part and a fusion part, wherein the multi-level semantic enhancement part is used to generate global semantic information based on the high-level features in the multi-level features of the target optical remote sensing image extracted by the feature extraction part, and use the global semantic information to perform semantic refinement to obtain semantic attention information features, the boundary guidance part is used to perform boundary guidance based on the multi-level features to obtain boundary guidance features, and the fusion part is used to fuse the semantic attention information features and the boundary guidance features to obtain the detection results of the salient targets. Through the multi-level semantic enhancement part of the target network model, the scale change and cluttered background interference of the salient targets in the optical remote sensing image can be effectively solved, and through the boundary guidance part of the target network model, the problems of incomplete salient target detection structure and sensitivity to noise can be effectively solved, thereby improving the detection efficiency of the salient targets in the target optical remote sensing image.

[0060] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 A schematic diagram of a system composition architecture applicable to an embodiment of the present application;

[0063] Figure 2 This is a flowchart of a method for detecting a significant object in an embodiment of the present application;

[0064] Figure 3 A structural diagram of a target network model in an embodiment of the present application;

[0065] Figure 4 Another structural diagram of the target network model in the embodiment of the present application;

[0066] Figure 5 This is a schematic diagram of the structure of the multi-level semantic enhancement part in the embodiment of the present application;

[0067] Figure 6 This is a schematic diagram of the structure of the target boundary guidance module in the embodiment of the present application;

[0068] Figure 7 This is a schematic diagram of a detection result in an embodiment of the present application.

[0069] Figure 8 Schematic diagram of the structure of a salient object detection device in an embodiment of the present application;

[0070] Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of the present application.

[0072] The terms "first", "second", etc. of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of one type, and the number of objects is not limited, for example, the first object can be one or more. In addition, "or" in the present application represents at least one of the connected objects. For example, the protection scope of "A or B" covers at least three schemes, that is, scheme one: including A and excluding B; scheme two: including B and excluding A; scheme three: including both A and B. In addition, the terms "A and / or B", "at least one of A and B", and "at least one of A or B" also cover at least the above three schemes respectively. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0073] The core of this application is to provide a significant target detection method, which can be applied to various scenarios where significant target detection is required for optical remote sensing images, such as image processing, image fusion, scene classification, object detection and the like.

[0074] For ease of understanding, the following first introduces the system composition architecture to which the technical solution of this application is applicable. Figure 1 As shown, the application server is connected to the optical remote sensing image acquisition device and the processing server respectively, and a target network model is deployed on the processing server. The application server can obtain the target optical remote sensing image from the optical remote sensing image acquisition device, and after obtaining the target optical remote sensing image, the target optical remote sensing image can be sent to the processing server, and the target network model deployed on the processing server is used to detect the salient target in the target optical remote sensing image.

[0075] Alternatively, the application server is connected to the optical remote sensing image acquisition device, and a target network model is deployed on the application server. The application server can obtain the target optical remote sensing image from the optical remote sensing image acquisition device, and after obtaining the target optical remote sensing image, the application server can detect the salient target in the target optical remote sensing image using the target network model deployed on itself.

[0076] It should be noted that the above description is based on an example in which the application server or processing server is an independent server. However, it is understandable that in actual applications, the application server or processing server can also be replaced by a server cluster, or a distributed cluster composed of multiple servers. Figure 1 In the embodiment, the application server may be replaced by an application platform composed of multiple application servers, or the processing server may be replaced by a processing platform composed of multiple processing servers.

[0077] At present, with the development of remote sensing earth observation technology, the acquisition capability of optical remote sensing images has been continuously improved, and the amount of optical remote sensing image data generated every day has been increasing. In addition, due to the development of acquisition equipment, the collected optical remote sensing images have higher resolution and smaller resolution error. Therefore, how to quickly process, interpret and extract remote sensing information faces great challenges.

[0078] In order to meet this challenge, we can consider combining deep learning technology with remote sensing image interpretation technology, and use the advantages of deep learning such as high adaptability, wide coverage and fast computing speed to extract remote sensing information. Especially in pixel-level tasks, deep learning technology can provide higher accuracy and richer information. A pixel-level task can provide information for multiple downstream tasks at the same time, which can reduce the cost of algorithm design to a certain extent and facilitate downstream tasks.

[0079] The purpose of salient target detection in optical remote sensing images is to simulate the human visual attention mechanism and automatically capture targets and areas that attract human attention in complex large-scene aerial images. It has practical value and can not only be used as a preprocessing technology to assist various downstream visual applications in remote sensing scenes, such as image fusion, scene classification, and object detection, but also improve the automation and intelligence level of optical remote sensing image processing.

[0080] The technical solution provided in the embodiments of the present application can improve the performance of salient target detection in optical remote sensing images by designing an algorithm model to meet the growing processing needs of optical remote sensing images.

[0081] See also Figure 2 FIG. 1 is a flowchart of an implementation of a significant object detection method provided in an embodiment of the present application. The method may include the following steps:

[0082] S210: Obtaining a target optical remote sensing image.

[0083] The target optical remote sensing image may be any optical remote sensing image to be subjected to significant target detection, and may be obtained by image acquisition, image transmission, and the like.

[0084] S220: Detect salient targets in the target optical remote sensing image using the target network model.

[0085] In an embodiment of the present application, a target network model may be obtained by pre-training.

[0086] like Figure 3As shown, the target network model may include a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part, and a fusion part. The feature extraction part is respectively connected to the hierarchical semantic enhancement part and the boundary guidance part, and the fusion part is respectively connected to the multi-level semantic enhancement part and the boundary guidance part.

[0087] The feature extraction part is used to extract multi-level features of the target optical remote sensing image. The multi-level features can include low-level features and high-level features, and there can be multiple low-level features and multiple high-level features respectively.

[0088] The multi-level semantic enhancement part is used to generate global semantic information based on high-level features in the multi-level features, and use the global semantic information to perform semantic refinement to obtain semantic attention information features.

[0089] The boundary guidance part is used to perform boundary guidance based on multi-level features to obtain boundary guidance features.

[0090] The fusion part is used to fuse the semantic attention information features and the boundary guidance features to obtain the detection results of salient targets.

[0091] After obtaining the target optical remote sensing image, the target optical remote sensing image can be input into the target network model, and the target network model is used to detect salient targets, and the detection result of the salient targets is obtained according to the output of the target network model.

[0092] Optionally, after obtaining the target optical remote sensing image, the target optical remote sensing image may be preprocessed, such as cropping, scaling, normalization, etc., to adapt it to the input requirements of the target network model.

[0093] By applying the technical solution provided in the embodiment of the present application, after obtaining the target optical remote sensing image, the target network model is used to detect the salient targets in the target optical remote sensing image, and the target network model includes a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part and a fusion part, wherein the multi-level semantic enhancement part is used to generate global semantic information based on the high-level features in the multi-level features of the target optical remote sensing image extracted by the feature extraction part, and use the global semantic information to perform semantic refinement to obtain semantic attention information features, the boundary guidance part is used to perform boundary guidance based on the multi-level features to obtain boundary guidance features, and the fusion part is used to fuse the semantic attention information features and the boundary guidance features to obtain the detection results of the salient targets. Through the multi-level semantic enhancement part of the target network model, the scale change and cluttered background interference of the salient targets in the optical remote sensing image can be effectively solved, and through the boundary guidance part of the target network model, the problems of incomplete salient target detection structure and sensitivity to noise can be effectively solved, thereby improving the detection efficiency of the salient targets in the target optical remote sensing image.

[0094] In some embodiments of the present application, the feature extraction part includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block stacked in sequence, and the boundary guidance part includes a first boundary guidance module, a second boundary guidance module, and a third boundary guidance module;

[0095] The third convolution block is also connected to the multi-level semantic enhancement part and the first boundary guidance module respectively, the fourth convolution block is also connected to the multi-level semantic enhancement part and the second boundary guidance module respectively, and the fifth convolution block is also connected to the multi-level semantic enhancement part and the third boundary guidance module respectively.

[0096] In an embodiment of the present application, the feature extraction part includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a fifth convolution block stacked in sequence. These five convolution blocks are connected in sequence, and the output feature of the previous convolution block is used as the input of the next convolution block.

[0097] like Figure 4 As shown, the feature extraction part includes convolution blocks Conv1, Conv2, Conv3, Conv4, and Conv5. The target optical remote sensing image is input into Conv1 to obtain feature X1 output by Conv1, feature X1 is input into Conv2 to obtain feature X2 output by Conv2, feature X2 is input into Conv3 to obtain feature X3 output by Conv3, feature X3 is input into Conv4 to obtain feature X4 output by Conv4, and feature X4 is input into Conv5 to obtain feature X5 output by Conv5.

[0098] The features output by the third convolution block, the fourth convolution block, and the fifth convolution block can be considered as high-level features, and the features output by the first convolution block and the second convolution block can be considered as low-level features.

[0099] The boundary guiding part includes a first boundary guiding module, a second boundary guiding module and a third boundary guiding module.

[0100] The third convolution block is also connected to the multi-level semantic enhancement part and the first boundary guidance module respectively. That is, in addition to being connected to the second convolution block and the fourth convolution block, the third convolution block is also connected to the multi-level semantic enhancement part and the first boundary guidance module respectively, and the features output by the third convolution block are respectively input into the fourth convolution block, the multi-level semantic enhancement part and the first boundary guidance module.

[0101] The fourth convolution block is also connected to the multi-level semantic enhancement part and the second boundary guidance module respectively. That is, in addition to being connected to the third convolution block and the fifth convolution block, the fourth convolution block is also connected to the multi-level semantic enhancement part and the second boundary guidance module respectively, and the features output by the fourth convolution block are respectively input into the fifth convolution block, the multi-level semantic enhancement part and the second boundary guidance module.

[0102] The fifth convolution block is also connected to the multi-level semantic enhancement part and the third boundary guidance module respectively. That is, in addition to being connected to the fourth convolution block, the fifth convolution block is also connected to the multi-level semantic enhancement part and the third boundary guidance module respectively, and the features output by the fifth convolution block are respectively input into the multi-level semantic enhancement part and the third boundary guidance module.

[0103] like Figure 4 In the figure, the boundary guiding part includes a first boundary guiding module, a second boundary guiding module and a third boundary guiding module. In addition to being connected to Conv2 and Conv4, Conv3 is also connected to the multi-level semantic enhancement part and the first boundary guiding module, respectively. In addition to being connected to Conv3 and Conv5, Conv4 is also connected to the multi-level semantic enhancement part and the second boundary guiding module, respectively. In addition to being connected to Conv4, Conv5 is also connected to the multi-level semantic enhancement part and the third boundary guiding module, respectively.

[0104] It is understandable that semantic understanding is a feasible solution to the problems of wide coverage, complex background, and object scale changes in optical remote sensing images. High-level features have rich semantic information, which can not only build the relationship between different scattered areas, but also build the relationship between complex background and target area. Therefore, the embodiment of the present application deploys a multi-level semantic enhancement part to aggregate high-level features with a complementary multi-branch strategy to generate semantic attention information features to highlight important areas. That is, starting from different spatial resolutions and the information content of different high-level features, a multi-branch structure is designed to comprehensively utilize the rich semantic information of the last three convolutional blocks.

[0105] In addition, low-level features have greater spatial resolution and contain rich detailed information such as boundaries, which is conducive to accurately refining the boundaries of significant targets. In order to highlight the important boundaries in feature decoding, the embodiment of the present application deploys a boundary guidance part, which introduces boundary guidance of low-level features from the perspective of feature fusion.

[0106] In some embodiments of the present application, the multi-level semantic enhancement part is specifically used for:

[0107] Upsample the features obtained from the fifth convolutional block and use the spatial attention mechanism to obtain the first attention map features;

[0108] Perform multi-scale progressive fusion processing on the features obtained from the third convolution block, the fourth convolution block and the fifth convolution block to obtain the second attention map features;

[0109] The first attention map features and the second attention map features are fused to obtain the semantic attention information features.

[0110] In the embodiment of the present application, considering that the top-level features contain the richest channel semantic features, but their spatial resolution is the lowest, the most intuitive upsampling filtering method is used to retain the accuracy of the top-level semantic information. The multi-level semantic enhancement part can upsample the features obtained from the fifth convolutional block, such as upsampling four times, to retain the accurate highest-level semantic information, and then use the spatial attention mechanism to obtain the first attention map features. Figure 5 As shown, the feature X5 obtained from the fifth convolutional block is upsampled, and the first attention map feature att is obtained using the spatial attention mechanism. s .

[0111] In addition, the embodiment of the present application proposes a multi-scale progressive fusion strategy, which considers the information of three high-level features at the same time, and finally unifies the resolution at the third level. This can more comprehensively understand the complementary relationship between different high-level features and reduce spatial resolution sampling distortion. That is, the multi-level semantic enhancement part performs multi-scale progressive fusion processing on the features obtained from the third convolution block, the fourth convolution block, and the fifth convolution block to obtain the second attention map feature.

[0112] The first attention map features and the second attention map features are fused to obtain the semantic attention information features.

[0113] By processing high-level features using multi-level semantic enhancement parts, semantic attention information features can be accurately obtained.

[0114] In some embodiments of the present application, the multi-level semantic enhancement part is specifically used for:

[0115] Upsampling the features obtained from the fifth convolution block to obtain upsampled features, where the spatial resolution of the upsampled features is the same as the spatial resolution of the features output by the third convolution block;

[0116] Use the average pooling and maximum pooling of the upsampled features to form two single-channel mappings, and perform feature concatenation along the channel direction to generate dual-channel features;

[0117] Based on the convolution operation on the dual-channel features, the first attention map features are obtained.

[0118] In an embodiment of the present application, the multi-level semantic enhancement part can upsample the features obtained from the fifth convolution block, such as performing 4-fold upsampling, to obtain upsampled features, and the spatial resolution of the upsampled features is the same as the spatial resolution of the features output by the third convolution block, that is, the spatial resolution of the features output by the fifth convolution block is restored to the resolution of the features output by the third convolution block.

[0119] However, because the top-level features contain a large amount of redundant information, the embodiment of the present application adopts a spatial attention mechanism to determine the most important position in the feature and obtain the first attention map feature.

[0120] In order to achieve spatial attention, two single-channel maps are formed by using average pooling and maximum pooling of upsampled features, and features are concatenated along the channels to generate dual-channel features. Then, based on the convolution operation of the dual-channel features, the first attention map features are obtained. For example, a convolution layer with a filter size of 3×3 is applied to transform a dual-channel feature into a two-dimensional first attention map feature.

[0121] First attention map feature att s It can be expressed by the following formula:

[0122] att s =sigmoid(f 3×3 ([avgPool(up4(X5));maxPool(up4(X5))]));

[0123] Among them, [;] represents feature concatenation along the channel direction, up4() represents 4 times upsampling, avgPool() represents average pooling, maxPool() represents maximum pooling, sigmoid represents sigmoid function, and f m×m represents a convolution operation with a filter size of m×m.

[0124] The first attention map features are obtained through the above operations, which can retain high-level semantic information.

[0125] In some embodiments of the present application, the multi-level semantic enhancement part is specifically used for:

[0126] Expanding the features obtained from the third convolution block using an expanded convolution layer with a first expansion rate to obtain a first expanded feature;

[0127] Using an extended convolution layer with a second expansion rate, the features obtained from the fourth convolution block are expanded to obtain second extended features;

[0128] Expanding the features obtained from the fifth convolution block using an expanded convolution layer with a third expansion rate to obtain a third expanded feature;

[0129] The third extended feature is upsampled, fused with the second extended feature, and then fused with the first extended feature to obtain the second attention map feature.

[0130] It is understandable that in the embodiment of the present application, although the features output by the third convolution block, the features output by the fourth convolution block, and the features output by the fifth convolution block are all regarded as high-level features, there are differences between the three. Therefore, in order to obtain more comprehensive semantic information, it is necessary to effectively fuse them. In addition, in order to allow the target network model to obtain robust and stable detection results for different regions, the embodiment of the present application first enhances the features obtained from the third convolution block, the features obtained from the fourth convolution block, and the features obtained from the fifth convolution block before fusion to perceive a larger receptive field.

[0131] In an embodiment of the present application, first, extended convolution layers for achieving different expansion rates are deployed corresponding to the third convolution block, the fourth convolution block, and the fifth convolution block, respectively. Then, the features obtained from the fifth convolution block are gradually upsampled, and the deep-level features are fused with the shallow-level features to achieve sufficient high-level feature aggregation, and the second attention map features are obtained through the convolution layer.

[0132] Specifically, the feature obtained from the third convolution block can be expanded using an extended convolution layer with a first expansion rate to obtain a first extended feature, the feature obtained from the fourth convolution block can be expanded using an extended convolution layer with a second expansion rate to obtain a second extended feature, the feature obtained from the fifth convolution block can be expanded using an extended convolution layer with a third expansion rate to obtain a third extended feature, and then the third extended feature can be upsampled, fused with the second extended feature, and then fused with the first extended feature to obtain a second attention map feature.

[0133] For example, Figure 5 As shown, the feature X3 obtained from Conv3 is expanded using an expanded convolution layer with an expansion rate of d=8 to obtain an expanded feature M3, the feature X4 obtained from Conv4 is expanded using an expanded convolution layer with an expansion rate of d=4 to obtain an expanded feature M4, and the feature X5 obtained from Conv5 is expanded using an expanded convolution layer with an expansion rate of d=2 to obtain an expanded feature M5, as shown below:

[0134] M3=sigmoid(f 3×3 d＝8 (X3));

[0135] M4=sigmoid(f 3×3 d＝4 (X4));

[0136] M5=sigmoid(f 3×3d＝2 (X5)).

[0137] In order to minimize the resolution blurring and distortion caused by upsampling during the fusion process, a progressive fusion strategy is adopted. First, M5 is upsampled and then connected to M4, and a 3×3 convolutional layer is used to generate a fusion feature M with the same spatial resolution as X4. 4,5 Similarly, the fusion feature M 4,5 After further upsampling, it is fused with M3 to obtain the global semantic feature M 3,4,5 After that, a 1×1 convolutional layer is used to generate the second attention map feature att m . As shown below:

[0138] att m =sigmoid(f 1×1 (f 3×3 ([up2(f 3×3 ([up2(M5),M4]),M3)]))).

[0139] Through the above operations, accurate second attention map features can be obtained.

[0140] Get the first attention map feature att s and the second attention map feature att m After that, the first attention map feature att s and the second attention map feature att m Fusion is performed to obtain the semantic attention information feature att e , as shown below:

[0141]

[0142] in, Represents the sum of corresponding elements.

[0143] The semantic attention information features are obtained through multi-level semantic enhancement, which helps to highlight important areas.

[0144] In some embodiments of the present application, the second convolution block is connected to the first boundary guiding module, the second boundary guiding module and the third boundary guiding module respectively.

[0145] In an embodiment of the present application, in order to enhance the expressiveness of boundary information, the features output by the second convolution block may be fed into a boundary guiding part so that the boundary guiding part extracts boundary information and generates a single-channel mask of a boundary map.

[0146] Specifically, the second convolution block can be connected to the first boundary guide module, the second boundary guide module and the third boundary guide module respectively, and the features output by the second convolution block are respectively input into the first boundary guide module, the second boundary guide module and the third boundary guide module, such as Figure 4 shown.

[0147] It is understandable that low-level features have greater spatial resolution and contain rich detailed information such as boundaries, which is conducive to accurately refining the boundaries of salient objects. The features output by the second convolutional block are fed to the boundary guidance part as low-level features containing rich detailed information. The introduction of boundary guidance of low-level features can highlight important boundaries in feature decoding.

[0148] In some embodiments of the present application, the target boundary guidance module included in the boundary guidance part is specifically used to:

[0149] Perform the first convolution operation on the features obtained from the second convolution block to obtain boundary features;

[0150] Upsampling the features obtained from the target convolution block to obtain a first mask, where the target convolution block is a convolution block other than the second convolution block connected to the target boundary guidance module;

[0151] The first mask is merged with the boundary feature to obtain a first guide feature;

[0152] Perform a second convolution operation on the boundary features to obtain a second mask;

[0153] Fusion of the second mask with the features obtained from the target convolutional block to obtain a second guided feature;

[0154] The first guide feature and the second guide feature are merged to obtain a boundary guide feature corresponding to the target boundary guide module;

[0155] The target boundary guiding module is the first boundary guiding module, the second boundary guiding module, or the third boundary guiding module.

[0156] In the embodiment of the present application, the first boundary guiding module, the second boundary guiding module, or the third boundary guiding module may be used as a target boundary guiding module.

[0157] The target boundary guidance module is a symmetrical structure that combines the corresponding encoder features (features obtained from the target convolution block) with the boundary information of the encoder features corresponding to the second convolution block. Instead of simply combining these features through concatenation or addition operations, feature guidance and refinement are performed through multiplication in a symmetrical manner. The boundary features obtained based on the second convolution block and the corresponding encoder features are used as basic features for each other, and other features are used for guidance. This symmetric strategy can ensure the complementarity and sufficiency of feature learning, thereby maximizing the interference suppression ability of multiplication fusion.

[0158] The target convolution block is a convolution block other than the second convolution block connected to the target boundary guidance module.

[0159] like Figure 6 As shown, the target boundary guidance module performs a first convolution operation on the features obtained from the second convolution block to obtain boundary features, so that the features obtained from the second convolution block can be compressed to the same number of channels as the features obtained from the target convolution block. For example, the features obtained from the second convolution block can be fed into a 1×1 convolution layer.

[0160] The target boundary guidance module upsamples the features obtained from the target convolution block to obtain a first mask. For example, a 3×3 convolution layer upsampling operation is performed on the features obtained from the target convolution block.

[0161] The first mask is fused with the boundary feature, the boundary feature is refined, and the first guide feature is obtained.

[0162] For example, the first guide feature X n,xb As shown below:

[0163] X n,xb =ReLU(W n ⊙f 1×1 (X b ));

[0164] Among them, W n =up(f 3×3 (X n ), ReLU represents the ReLU activation function, ⊙ represents the product of the corresponding elements, X b Denotes the feature X2 obtained from the second convolutional block, W n represents the first mask obtained by upsampling the features obtained from the target convolutional block, X n represents the features obtained from the target convolutional block. If the target boundary guidance module is the first boundary guidance module, then the target convolutional block is the third convolutional block, X n If the target boundary guidance module is the second boundary guidance module, the target convolution block is the fourth convolution block, X nIf the target boundary guidance module is the third boundary guidance module, then the target convolution block is the fifth convolution block, X n For X5.

[0165] Symmetrically to the above method, the encoder features can be modified using a mask derived from the boundary features. Specifically, the target boundary guidance module can perform a second convolution operation on the boundary features to obtain a second mask. For example, an additional 3×3 convolution operation is performed on the boundary features to obtain the second mask. The second mask is fused with the features obtained from the target convolution block to obtain the second guidance feature.

[0166] For example, the second guide feature X n,bx As shown below:

[0167] X n,bx =up(X n ⊙f 3×3 (X b )).

[0168] By fusing the first guide feature and the second guide feature, the boundary guide feature corresponding to the target guide module can be obtained, such as X n,B .

[0169] The above operations are performed on the first boundary guiding module, the second boundary guiding module, and the third boundary guiding module respectively, and corresponding boundary guiding features can be obtained respectively.

[0170] In some embodiments of the present application, the fusion part is specifically used for:

[0171] Fusing the semantic attention information feature with the boundary guidance feature corresponding to the third boundary guidance module to obtain a first fused feature;

[0172] Fusing the first fused feature with the boundary guidance feature corresponding to the second boundary guidance module to obtain a second fused feature;

[0173] Fusing the second fused feature with the boundary guidance feature corresponding to the first boundary guidance module to obtain a third fused feature;

[0174] Based on the third fusion feature, the detection result of the salient object is obtained.

[0175] In the embodiments of the present application, Figure 4As shown, the multi-level semantic enhancement part generates global semantic information based on the high-level features in the multi-level features extracted by the feature extraction part, and uses the global semantic information for semantic refinement. After obtaining the semantic attention information feature, and after obtaining the boundary guidance features corresponding to the first boundary guidance module, the second boundary guidance module and the third boundary guidance module respectively, the boundary guidance feature corresponding to the third boundary guidance module can be fused with the semantic attention information feature to obtain the first fused feature. Then, the first fused feature is fused with the boundary guidance feature corresponding to the second boundary guidance module to obtain the second fused feature. Then, the second fused feature is fused with the boundary guidance feature corresponding to the first boundary guidance module to obtain the third fused feature. Finally, based on the third fused feature, the detection result of the salient target is obtained.

[0176] The fusion process can be understood as a splicing process. After obtaining the third fusion feature, the third fusion feature can be input into the 3×3 convolution layer and compressed to the original number of channels.

[0177] By fusing the semantic attention information features with the boundary guidance features, accurate salient object detection results can be obtained.

[0178] In some embodiments of the present application, the target network model is pre-trained based on a loss function, wherein the loss function includes boundary loss, intersection-over-union loss, and binary cross entropy loss.

[0179] In an embodiment of the present application, the target network model can be pre-trained based on the loss function. The target network initial model has the same structure as the target network model introduced above. Deep supervision can be used to design a hierarchical loss function on different scales, such as the output of the first fusion feature G5, the second fusion feature G4, and the third fusion feature G3.

[0180] In addition, the embodiment of the present application uses the standard binary cross entropy on the boundary as the loss function, and designs semantic loss on the semantic attention information feature through weighted intersection over Union (IoU) loss and weighted binary cross entropy (BCE) loss to improve the network's recognition ability of edge pixels, refine edge features, improve segmentation accuracy, and alleviate the problem of data imbalance.

[0181] The total loss can be defined as:

[0182]

[0183] Among them, l bce represents boundary loss;

[0184] l k,iouRepresents weighted IoU loss, which is a measure of the accuracy of the segmentation model, which measures the overlap between the model output and the true label:

[0185]

[0186] Where N represents the total number of pixels, g i represents the true value of the i-th pixel, represents the predicted value of the i-th pixel, ω i Represents the weight coefficient of the i-th pixel.

[0187] l k,bce It represents weighted BCE loss, which is a classification loss function that assigns different weight coefficients to different categories, improves the model's attention to key categories, and improves segmentation performance.

[0188]

[0189] The target network model is pre-trained based on the above loss function, which can improve the accuracy of the target network model in detecting salient targets.

[0190] After detecting salient targets in target optical remote sensing images using the target network model, the performance of the target network model can be evaluated, and the model parameters or structure can be adjusted and optimized based on the evaluation results to improve the accuracy and efficiency of salient target detection.

[0191] Figure 7 The visualization example of the technical solution provided by the embodiment of the present application is shown. The first column is a red, green, blue (RGB) image, the second column is a true image of the significant target detection result, and the third column is the significant target detection result obtained by applying the technical solution provided by the embodiment of the present application. It can be seen from the results that the technical solution provided by the embodiment of the present application can completely and accurately extract significant targets, and can well suppress interference.

[0192] The embodiment of the present application adopts an encoder-decoder architecture and proposes a method for detecting salient objects in optical remote sensing images with boundary-guided semantic learning. The overall framework can be divided into an encoder stage and a decoder stage, where the encoder stage corresponds to the processing of the feature extraction part, and the decoder stage corresponds to the processing of the multi-level semantic enhancement part, the boundary guidance part, and the fusion part.

[0193] The feature extraction part in the embodiment of the present application can be a backbone coding extractor, such as Res2Net, which is composed of five sequentially stacked convolution blocks to obtain multi-level features {X1, X2, X3,, X4, X5}.

[0194] Considering the important role of semantic relations and boundary constraints in the saliency target task, the embodiment of the present application uses a multi-level semantic enhancement part and a boundary guidance part to highlight the global semantics and clear boundaries in the decoding process.

[0195] The high-level features of the last three convolution blocks contain rich semantic information, while the corresponding low-level features of the first two convolution blocks contain more detailed boundary information due to their higher spatial resolution. In order to comprehensively utilize the rich semantic information of the last three convolution blocks, the embodiment of the present application adopts a complementary multi-level approach to design a multi-level semantic enhancement part, generate global semantic information, and use this information for semantic refinement.

[0196] Clear boundaries are crucial for salient object detection in optical remote sensing images, so making full use of boundary information is the key to obtaining competitive results. Therefore, in addition to the commonly used boundary supervision constraints, the embodiment of the present application also incorporates boundary information into the decoding process by designing a boundary guidance part to achieve more in-depth and comprehensive boundary optimization and guidance.

[0197] In addition, in order to enhance the expressiveness of boundary information, the features of the second convolutional block are fed into the boundary-guided part to extract boundary information, thereby generating a single-channel mask of the boundary map.

[0198] Furthermore, explicit supervision can be performed between the generated boundary map and the boundary truth obtained by a boundary extractor, such as Canny, to ensure the effectiveness of learning.

[0199] Finally, the features obtained in the fusion part are used to generate the final prediction of the salient objects through an additional sigmoid activation function.

[0200] Corresponding to the above method embodiment, the embodiment of the present application further provides a salient object detection device. The salient object detection device described below and the salient object detection method described above can refer to each other.

[0201] See also Figure 8 As shown, the salient object detection device 800 includes the following modules:

[0202] An acquisition module 810 is used to obtain a target optical remote sensing image;

[0203] A detection module 820 is used to detect salient targets in the target optical remote sensing image using the target network model;

[0204] Among them, the target network model includes feature extraction part, multi-level semantic enhancement part, boundary guidance part and fusion part;

[0205] The feature extraction part is used to extract multi-level features of the target optical remote sensing image;

[0206] The multi-level semantic enhancement part is used to generate global semantic information based on high-level features in the multi-level features, and use the global semantic information to perform semantic refinement to obtain semantic attention information features;

[0207] The boundary guidance part is used to guide the boundary based on multi-level features to obtain boundary guidance features;

[0208] The fusion part is used to fuse the semantic attention information features and the boundary guidance features to obtain the detection results of salient targets.

[0209] After obtaining the target optical remote sensing image by using the device provided in the embodiment of the present application, the target network model is used to detect the salient targets in the target optical remote sensing image, and the target network model includes a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part and a fusion part, wherein the multi-level semantic enhancement part is used to generate global semantic information based on the high-level features in the multi-level features of the target optical remote sensing image extracted by the feature extraction part, and use the global semantic information to perform semantic refinement to obtain semantic attention information features, the boundary guidance part is used to perform boundary guidance based on the multi-level features to obtain boundary guidance features, and the fusion part is used to fuse the semantic attention information features and the boundary guidance features to obtain the detection result of the salient target. Through the multi-level semantic enhancement part of the target network model, the scale change and cluttered background interference of the salient targets in the optical remote sensing image can be effectively solved, and through the boundary guidance part of the target network model, the problems of incomplete salient target detection structure and sensitivity to noise can be effectively solved, thereby improving the detection efficiency of the salient targets in the target optical remote sensing image.

[0210] In some embodiments of the present application, the feature extraction part includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, and a fifth convolution block stacked in sequence, and the boundary guidance part includes a first boundary guidance module, a second boundary guidance module, and a third boundary guidance module;

[0211] The third convolution block is also connected to the multi-level semantic enhancement part and the first boundary guidance module respectively, the fourth convolution block is also connected to the multi-level semantic enhancement part and the second boundary guidance module respectively, and the fifth convolution block is also connected to the multi-level semantic enhancement part and the third boundary guidance module respectively.

[0212] In some embodiments of the present application, the multi-level semantic enhancement part is specifically used for:

[0213] Upsample the features obtained from the fifth convolutional block and use the spatial attention mechanism to obtain the first attention map features;

[0214] Perform multi-scale progressive fusion processing on the features obtained from the third convolution block, the fourth convolution block and the fifth convolution block to obtain the second attention map features;

[0215] The first attention map features and the second attention map features are fused to obtain the semantic attention information features.

[0216] In some embodiments of the present application, the multi-level semantic enhancement part is specifically used for:

[0217] Upsampling the features obtained from the fifth convolution block to obtain upsampled features, where the spatial resolution of the upsampled features is the same as the spatial resolution of the features output by the third convolution block;

[0218] Use the average pooling and maximum pooling of the upsampled features to form two single-channel mappings, and perform feature concatenation along the channel direction to generate dual-channel features;

[0219] Based on the convolution operation on the dual-channel features, the first attention map features are obtained.

[0220] In some embodiments of the present application, the multi-level semantic enhancement part is specifically used for:

[0221] Expanding the features obtained from the third convolution block using an expanded convolution layer with a first expansion rate to obtain a first expanded feature;

[0222] Using an extended convolution layer with a second expansion rate, the features obtained from the fourth convolution block are expanded to obtain second extended features;

[0223] Expanding the features obtained from the fifth convolution block using an expanded convolution layer with a third expansion rate to obtain a third expanded feature;

[0224] The third extended feature is upsampled, fused with the second extended feature, and then fused with the first extended feature to obtain the second attention map feature.

[0225] In some embodiments of the present application, the second convolution block is connected to the first boundary guiding module, the second boundary guiding module and the third boundary guiding module respectively.

[0226] In some embodiments of the present application, the target boundary guidance module included in the boundary guidance part is specifically used to:

[0227] Perform the first convolution operation on the features obtained from the second convolution block to obtain boundary features;

[0228] Upsampling the features obtained from the target convolution block to obtain a first mask, where the target convolution block is a convolution block other than the second convolution block connected to the target boundary guidance module;

[0229] The first mask is merged with the boundary feature to obtain a first guide feature;

[0230] Perform a second convolution operation on the boundary features to obtain a second mask;

[0231] Fusion of the second mask with the features obtained from the target convolutional block to obtain a second guided feature;

[0232] The first guide feature and the second guide feature are merged to obtain a boundary guide feature corresponding to the target boundary guide module;

[0233] The target boundary guiding module is the first boundary guiding module, the second boundary guiding module, or the third boundary guiding module.

[0234] In some embodiments of the present application, the fusion part is specifically used for:

[0235] Fusing the semantic attention information feature with the boundary guidance feature corresponding to the third boundary guidance module to obtain a first fused feature;

[0236] Fusing the first fused feature with the boundary guidance feature corresponding to the second boundary guidance module to obtain a second fused feature;

[0237] Fusing the second fused feature with the boundary guidance feature corresponding to the first boundary guidance module to obtain a third fused feature;

[0238] Based on the third fusion feature, the detection result of the salient object is obtained.

[0239] In some embodiments of the present application, the target network model is pre-trained based on a loss function, wherein the loss function includes boundary loss, intersection-over-union loss, and binary cross entropy loss.

[0240] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0241] Corresponding to the above method embodiment, the present application embodiment further provides an electronic device, including:

[0242] Memory for storing computer programs;

[0243] A processor is used to implement the steps of the above-mentioned significant object detection method when executing a computer program.

[0244] like Fig. 9, which is a schematic diagram of the composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through the communication bus 13.

[0245] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0246] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in an embodiment of the salient object detection method.

[0247] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0248] Obtaining target optical remote sensing images;

[0249] Using the target network model, the salient targets in the target optical remote sensing image are detected;

[0250] Among them, the target network model includes feature extraction part, multi-level semantic enhancement part, boundary guidance part and fusion part;

[0251] The feature extraction part is used to extract multi-level features of the target optical remote sensing image;

[0252] The multi-level semantic enhancement part is used to generate global semantic information based on high-level features in the multi-level features, and use the global semantic information to perform semantic refinement to obtain semantic attention information features;

[0253] The boundary guidance part is used to guide the boundary based on multi-level features to obtain boundary guidance features;

[0254] The fusion part is used to fuse the semantic attention information features and the boundary guidance features to obtain the detection results of salient targets.

[0255] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0256] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0257] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0258] Of course, it should be noted that Fig. 9 The structure shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Fig. 9 More or fewer components than shown, or combinations of certain components.

[0259] Corresponding to the above method embodiment, the embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned salient target detection method are implemented.

[0260] In addition, it should be noted that: the embodiment of the present application also provides a computer program product or a computer program, which may include computer instructions, and the computer instructions may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor can execute the computer instructions so that the computer device executes the description of the significant target detection method in the corresponding embodiment of the foregoing text, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0261] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0262] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0263] Through the description of the above implementation methods, those skilled in the art can also clearly understand that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0264] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be directly implemented using hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art, including several instructions to execute the methods described in the various embodiments of the present application.

[0265] The embodiments of the present application are described above in conjunction with the accompanying drawings, and the description of the above embodiments is only used to help understand the technical solution and its core idea of ​​the present application. It should be pointed out that the present application is not limited to the above-mentioned specific implementation methods, which are merely illustrative and not restrictive. For ordinary technicians in this field, many forms of implementation methods can be made without departing from the scope of protection of the purpose of the present application and the claims, and the present application can also be improved and modified in a number of ways, and these implementation methods, improvements and modifications are all within the scope of protection of the present application.

Claims

1. A method for detecting a salient object, characterized in that: include: Obtaining target optical remote sensing images; Detecting salient targets in the target optical remote sensing image using a target network model; The target network model includes a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part, and a fusion part; The feature extraction part is used to extract multi-level features of the target optical remote sensing image; The multi-level semantic enhancement part is used to generate global semantic information based on high-level features in the multi-level features, and use the global semantic information to perform semantic refinement to obtain semantic attention information features; The boundary guiding part is used to perform boundary guiding based on the multi-level features to obtain boundary guiding features; The fusion part is used to fuse the semantic attention information feature and the boundary guidance feature to obtain a detection result of a salient target.

2. The method according to claim 1, characterized in that The feature extraction part includes a first convolution block, a second convolution block, a third convolution block, a fourth convolution block and a fifth convolution block stacked in sequence, and the boundary guidance part includes a first boundary guidance module, a second boundary guidance module and a third boundary guidance module; The third convolution block is also connected to the multi-level semantic enhancement part and the first boundary guidance module respectively, the fourth convolution block is also connected to the multi-level semantic enhancement part and the second boundary guidance module respectively, and the fifth convolution block is also connected to the multi-level semantic enhancement part and the third boundary guidance module respectively.

3. The method according to claim 2, characterized in that The multi-level semantic enhancement part is specifically used for: Upsampling the features obtained from the fifth convolutional block, and obtaining first attention map features using a spatial attention mechanism; Performing multi-scale progressive fusion processing on the features obtained from the third convolution block, the fourth convolution block, and the fifth convolution block to obtain a second attention map feature; The first attention map feature and the second attention map feature are fused to obtain a semantic attention information feature.

4. The method according to claim 3, characterized in that The multi-level semantic enhancement part is specifically used for: Upsampling the features obtained from the fifth convolution block to obtain upsampled features, where the spatial resolution of the upsampled features is the same as the spatial resolution of the features output by the third convolution block; Using average pooling and maximum pooling of the upsampled features to form two single-channel mappings, and performing feature concatenation along the channel direction to generate dual-channel features; Based on the convolution operation on the dual-channel features, a first attention map feature is obtained.

5. The method according to claim 3, characterized in that: The multi-level semantic enhancement part is specifically used for: Using an extended convolution layer with a first expansion rate, the feature obtained from the third convolution block is expanded to obtain a first extended feature; Using an extended convolution layer with a second expansion rate, the feature obtained from the fourth convolution block is expanded to obtain a second extended feature; Using an extended convolution layer with a third expansion rate, the features obtained from the fifth convolution block are expanded to obtain third extended features; The third extended feature is upsampled, fused with the second extended feature, and then fused with the first extended feature to obtain a second attention map feature.

6. The method according to claim 2, characterized in that The second convolution block is connected to the first boundary guiding module, the second boundary guiding module and the third boundary guiding module respectively.

7. The method according to claim 6, characterized in that The target boundary guidance module included in the boundary guidance part is specifically used for: Performing a first convolution operation on the features obtained from the second convolution block to obtain boundary features; Upsampling the features obtained from the target convolution block to obtain a first mask, wherein the target convolution block is a convolution block other than the second convolution block connected to the target boundary guidance module; Merging the first mask with the boundary feature to obtain a first guiding feature; Performing a second convolution operation on the boundary feature to obtain a second mask; fusing the second mask with the feature obtained from the target convolution block to obtain a second guided feature; The first guide feature and the second guide feature are merged to obtain a boundary guide feature corresponding to the target boundary guide module; The target boundary guiding module is the first boundary guiding module, the second boundary guiding module, or the third boundary guiding module.

8. The method according to claim 7, characterized in that The fusion part is specifically used for: Fusing the semantic attention information feature with the boundary guidance feature corresponding to the third boundary guidance module to obtain a first fused feature; Fusing the first fused feature with the boundary guidance feature corresponding to the second boundary guidance module to obtain a second fused feature; Fusing the second fused feature with the boundary guidance feature corresponding to the first boundary guidance module to obtain a third fused feature; Based on the third fusion feature, a detection result of a salient object is obtained.

9. The method according to any one of claims 1 to 8, characterized in that The target network model is pre-trained based on a loss function, wherein the loss function includes a boundary loss, an intersection-over-union loss, and a binary cross entropy loss.

10. A salient object detection device, characterized in that: include: An acquisition module, used for acquiring a target optical remote sensing image; A detection module, used to detect salient targets in the target optical remote sensing image using a target network model; The target network model includes a feature extraction part, a multi-level semantic enhancement part, a boundary guidance part, and a fusion part; The feature extraction part is used to extract multi-level features of the target optical remote sensing image; The multi-level semantic enhancement part is used to generate global semantic information based on high-level features in the multi-level features, and use the global semantic information to perform semantic refinement to obtain semantic attention information features; The boundary guiding part is used to perform boundary guiding based on the multi-level features to obtain boundary guiding features; The fusion part is used to fuse the semantic attention information feature and the boundary guidance feature to obtain a detection result of a salient target.

11. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the salient object detection method as described in any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the salient object detection method as described in any one of claims 1 to 9 are implemented.

Images