Optical remote sensing image saliency detection method and system based on contrast enhancement and hierarchical attention fusion, storage medium and electronic equipment

Abstract

The invention discloses an optical remote sensing image saliency detection method and system based on contrast enhancement and hierarchical attention fusion, a storage medium and electronic equipment, and the method comprises the following steps: firstly, increasing the number of samples through data augmentation, improving the generalization capability of a model through data enhancement, and then inputting an enhanced image into CEHAFNet for coding; the contrast enhancement module is used for enhancing the coding features of the first four layers; the decoder combines the global semantics with the enhanced features, and gradually decodes and recovers the significant features; then, the saliency header predicts saliency features of all layers, the saliency features are integrated through a hierarchical attention fusion module, a final saliency map is generated, and the difference between the saliency map and a label is calculated through self-adaptive mixed loss; after training is completed, model parameters are stored; and finally, inputting a to-be-tested sample into the model, and calculating to output a saliency map. According to the method, saliency detection can be more effectively carried out on the optical remote sensing image under the conditions of complex backgrounds, salient targets, similar backgrounds and the like.

Description

Technical Field

[0001] The present invention relates to the field of optical remote sensing image saliency detection, and in particular to an optical remote sensing image saliency detection method, system, storage medium and electronic device based on contrast enhancement and hierarchical attention fusion. Background Art

[0002] The main purpose of saliency detection in optical remote sensing images is to help achieve rapid positioning and identification of specific targets such as complex terrain, vegetation, water bodies, buildings, etc. [1], and to provide important data for various target identification and scene analysis, which has practical application value. However, some complex problems in optical remote sensing images, such as shadow interference, similarity between salient targets and background, still pose great challenges to the detection effect. By solving the difficulties such as shadow interference and similarity between targets and background, the analysis accuracy of optical remote sensing images can be improved, and the model's recognition ability for complex backgrounds and low-contrast targets can be enhanced, which will bring more application potential to the field of optical remote sensing image analysis and promote the popularization and application of remote sensing technology in various practical scenarios.

[0003] In recent years, saliency detection methods for optical remote sensing images have been continuously developed and have achieved outstanding results. According to different guidance methods, these methods can be roughly divided into two categories: semantic guidance [2] [3] [4] and context guidance [5] [6]. LV-Net [2] extracts multi-scale features through a dual-stream pyramid module, and gradually integrates detail and semantic features using encoder-decoder modules and nested connections, automatically selecting discriminative features to suppress cluttered backgrounds; BAFSNet [3] designs a semantic-guided fusion module (SGFM) to fuse low-level detail features and high-level semantic features, and uses the spatial location information of the semantic graph in the whole stage to retain the location information of the target object and suppress background noise; SARNet [4] aggregates high-level semantic features through a semantic-guided decoder (SGD), and guides the aggregation of low-level features through top-down feedback to locate objects in complex backgrounds. The semantic-guided method focuses on obtaining high-level semantic information of the scene, such as target category and approximate shape, and improves positioning accuracy through global features. However, semantic guidance methods have certain limitations in grasping the details of small objects or complex backgrounds [4]. ACCoNet proposed an Adjacent Context Coordination Module (ACCoM) to coordinate adjacent context features and improve the representation of salient objects by learning context information; DAFNet [5] introduced Global Context-aware Attention (GCA) to encode remote pixel dependencies, using global context information to suppress background interference and improve the accuracy of salient object detection; the Global Context Relation Learning Module (GCRL) for salient object detection in optical remote sensing images [6] was proposed to capture the contextual relationship between different spatial positions in optical remote sensing images and use it to guide feature aggregation at each stage. The context-guided method focuses on the relationship between the target and the surrounding area, and enhances the detection of salient objects through local features between neighboring pixels or regions. However, when the background and target features are similar or the contrast is low, the local features are easily interfered by noise, resulting in difficulty in distinguishing the target and background, resulting in false detection or missed detection. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system, storage medium and electronic device for optical remote sensing image saliency detection based on contrast enhancement and hierarchical attention fusion, which can accurately and effectively suppress the influence of shadows in complex backgrounds and achieve accurate distinction of salient targets in low contrast and when the salient target is close to the background.

[0005] The technical solution adopted by the present invention is:

[0006] A saliency detection method for optical remote sensing images based on contrast enhancement and hierarchical attention fusion includes the following steps:

[0007] Step 1, preprocessing the optical remote sensing image samples so that the optical remote sensing image samples are amplified and enhanced in sequence;

[0008] Step 2: Establish a saliency detection model CEHAFNet (Contrast Enhanced HierarchicalAttention Fusion Network); Step 2 specifically includes the following steps: Create a CEHAFNet encoder-decoder architecture, including a contrast enhancement module (Contrast Enhanced Module, CEM), a decoder ( Decoder ), saliency head, hierarchical attention fusion module (HAFM); the encoder and decoder are both five layers, the encoder is a ResNet50 pre-trained model and 5 CBR blocks, the decoder consists of decoding Decode, upsampling Up and fusion F, the deepest decoder Decoder5 does not contain fusion F, there is a contrast enhancement module between the corresponding layers of the encoder and decoder of the first four layers; the contrast enhancement module enhances the context features of the first four layers; the decoder decodes the global semantic features of the last layer and fuses them with the enhanced context features of the first four layers layer by layer and decodes them again to identify salient objects and salient areas; the saliency head generates a predicted saliency map for the salient features of each layer; the hierarchical attention fusion module is used to aggregate salient features of different layers;

[0009] Step 3: Train the saliency detection model CEHAFNet using the enhanced optical remote sensing image samples obtained in step 1 to obtain the optimal saliency detection model CEHAFNet;

[0010] Step 4: Use the optimal saliency detection model CEHAFNet to detect the optical remote sensing image to be detected and obtain the corresponding saliency map.

[0011] In the step 1, amplifying the optical remote sensing image samples specifically includes the following steps: enhancing the training sets of ORSSD and EORSSD by rotating at angles of 90°, 180° and 270°, and mirror-reversing all samples in the training sets; in this way, the training sets of ORSSD and EORSSD contain 4800 and 11200 samples respectively, achieving an eight-fold amplification effect.

[0012] In the step 1, data enhancement of the optical remote sensing image data specifically includes the following steps: scaling the optical remote sensing image samples obtained after amplification to a uniform size, and then randomly rotating and cropping them in sequence to enhance them, and obtain data as training samples.

[0013] The step 3 specifically includes the following steps:

[0014] Step 3.1: Input the training samples in step 1 into the saliency detection model CEHAFNet created in step 2;

[0015] Step 3.2: The input training sample is first extracted by truncating the last three layers of the pre-trained model of ResNet50 to obtain a multi-scale feature map, and then the encoder's multi-level features E1, E2, E3, E4, and E5 are obtained by the CBR block;

[0016] Step 3.3, input E4, E3, E2, and E1 into the contrast enhancement module in sequence to obtain enhanced context features CE4, CE3, CE2, and CE1;

[0017] Step 3.4: Input the global semantic feature map E5 obtained in step 3.2 into the decoder for decoding to obtain D5, fuse D5 with the enhanced context feature CE4 in step 3.3 through F to obtain F4, and then obtain the salient feature D4 of the fourth layer through Decode and Up; fuse the salient feature D4 of the fourth layer with CE3 through F and then through Decode and Up to obtain the salient feature D3 of the third layer; fuse D3 with CE2 through F and then through Decode and Up to obtain the salient feature D2 of the second layer; fuse D2 with CE1 through F and then through Decode and Up to obtain the salient feature D1 of the first layer;

[0018] Step 3.5: Predict the predicted saliency maps S1, S2, S3, S4, S5 corresponding to the salient features D1, D2, D3, D4, D5 through the saliency head, and aggregate the salient features D1, D2, D3, D4, D5 through the hierarchical attention fusion module to obtain the final predicted saliency map S0;

[0019] Step 3.6: Introduce a multi-level hybrid loss function with adaptive weights at the pixel level, region level, and image level to calculate

[0020] The loss between the predicted saliency maps S0, S1, S2, S3, S4, S5 obtained in step 3.5 and the corresponding label maps;

[0021] Step 3.7: After the training is completed, all parameter information of the trained model is saved.

[0022] The multi-level features E1, E2, E3, and E4 obtained in step 3.3 are processed using a contrast enhancement module, which specifically includes the following steps: The contrast enhancement module is divided into two parts: a contrast ratio attention module (CRA) and a CBR block. The CBR block extracts the global context information of the input features, and the CRA focuses on the local details of the input features.

[0023] First, use CRB with a convolution kernel size of 1 to extract the global context information of the current layer:

[0024] x_global i =CBR(E i ),i=1,2,3,4

[0025] The number of channels of multi-level features E1, E2, E3, and E4 is consistent with the input;

[0026] Then the CRA submodule enhances local details as follows: First extract E i The local feature x local_i , the difference between the local response and its average value is obtained by pooling the local features, that is, the contrast feature; the contrast feature x that needs to be paid attention to is learned after being activated by convolution, batch normalization and Tanh activation function. contrast_i , and then change the number of channels through a two-dimensional convolution with a convolution kernel of 1, and convert x contrast_i Mapped to the output dimension and predicting the contrast score of the region of interest,

[0027] Then, Sigmoid is used to map the obtained contrast attention score to [0, 1] to obtain the final score E cra_score_i ;

[0028] Multiply the input multi-level features E1, E2, E3, E4 with their respective contrast attention scores to obtain the enhanced local features x_local i , and then fuse it with the global context information through element-by-element addition to obtain the output CE1, CE2, CE3, and CE4 of the contrast enhancement module, so that the final output contains both global context information and fine-grained contrast features, effectively improving the detection ability of targets in complex backgrounds and suppressing background noise;

[0029] x local_i =Conv 3×3 (E i ),i=1,2,3,4

[0030] x local_avg_i =AvgPool(x local_i )

[0031] xcontrast_i = Tanh(BatchNorm(Conv 3×3 (x local_i -x local_avg_i )))

[0032] E cra_score_i =σ(Conv 1×1 (x contrast_i ))

[0033] x_local i =E i ×E cra_score_i

[0034] CE i =x_global i +x_local i

[0035] Where i = 1, 2, 3, 4, the number of channels of E1, E2, E3, and E4 is consistent with the input; CE i represents the output CE1, CE2, CE3, and CE4 of the contrast enhancement module; CBR represents ReLU (BatchNorm (Conv (f))) with a convolution kernel size of 3, where f is an arbitrary feature and Conv 3×3 Represents a two-dimensional convolution with a convolution kernel of 3, Conv 1×1 Represents a two-dimensional convolution with a convolution kernel of 1, AvgPool represents two-dimensional pooling, BatchNorm represents batch normalization, Tanh represents the Tanh activation function, and σ represents Sigmoid.

[0036] The hierarchical attention fusion module described in step 3.5 aggregates the salient features D1, D2, D3, D4, and D5 to obtain the final predicted salient map S0; the specific process is: the hierarchical attention aggregation module receives the salient features D1, D2, D3, D4, and D5 of all levels in step 3.4,

[0037] First, adjust D1, D2, D3, D4, and D5 to the size of D1 through bilinear interpolation.

[0038] Then, the number of channels of each feature map is unified into the feature map D with a target channel number of 32 through 1×1 convolution. i , in order to facilitate further integration;

[0039] The dimensions of each processed feature map are [B, C, H, W]. By stacking the feature maps of each layer, a new dimension L is generated after the first dimension to represent the level, thus forming a tensor D of [B, L, C, H, W] dimensions containing multiple levels of information. stack ;

[0040] Next, reshape it to the shape of [B, L, C, H×W], flatten the feature map of each layer to retain the spatial information, and aggregate the features of different levels through 1×1 convolution;

[0041] Then, the new features formed by aggregation are subjected to global average pooling and maximum pooling operations respectively. The pooling results are processed by the fully connected layer and the ReLU activation function. The two features are added together and the attention weight is generated through the Softmax function. The attention weight is expanded to [B, 1, 1, 1, 1] through the reshape operation, so that the weighted attention′ is consistent with the superimposed feature map D stack The dimensions of the , so that they can act correctly on each level and channel during subsequent weighting;

[0042] Finally, the final saliency map is generated by weighted summation in the L dimension and a 1×1 convolution layer;

[0043] D i ′=Conv 1×1 (interpolate(D i ))

[0044] D stack =Stack(D i ′, dim=1)

[0045] D′ stack =Reshape(D stack )

[0046] attention=Softmax(ReLU(fc1(AdaptAP(D′ stack )))+ReLU(fc1(AdaptMP(D′ stack ))))

[0047] attention′=Reshape(attention)

[0048] S0=Conv 1×1 (Sum(attention′×D stack ))

[0049] Among them, i = 1, 2, 3, 4, 5; B represents the batchsize dimension, L represents the level dimension, C represents the channel dimension, H and W represent the height and width dimensions; it represents a bilinear interpolation operation, Stack represents superimposing the feature map after dimension B and generating a new dimension; AdaptAP and AdaptMP represent adaptive averaging and adaptive maximum pooling respectively; Sum represents summing along the L dimension.

[0050] The multi-level adaptive hybrid loss function described in step 3.6 is specifically: multi-level hybrid loss L(s) = Σα with adaptive weights at the pixel level, region level, and image level k f k (s); f k represents pixel-level, region-level, and image-level losses, α k Represents the weight corresponding to the corresponding level of loss;

[0051] Pixel level loss Calculate the difference between the predicted pixel and the true pixel;

[0052] Regional level losses Measure the similarity between the predicted area and the true area;

[0053] Image-level loss Evaluate the overall overlap between the predicted image and the true image; The model can automatically adjust the weights of each loss function according to real-time performance statistics during training, forming a weight-adaptive hybrid loss function L; where G x,y represents the true label at pixel (x, y), P x,y is the probability value of the predicted pixel, N is the total number of pixels; they are the average values ​​of the predicted image and the true value image in the region, σ x and σ y is the standard deviation of the predicted image and the true image in the region, σ xy is the covariance between the predicted graph and the true value graph, C1 = 0.01 2 , C2=0.03 2 is a small constant added to avoid the denominator being zero; k is the loss function f k is an approximation of the recent rate of change of , and β is a tunable hyperparameter with a default value of 0.1.

[0054] The optical remote sensing image saliency detection system based on contrast enhancement and hierarchical attention fusion includes:

[0055] The encoder consists of a pre-trained model with the last three layers of ResNet50 truncated and CBR;

[0056] Contrast enhancement module, used to enhance contextual features;

[0057] The decoder is composed of decoding Decode, upsampling Up, and fusion F;

[0058] Saliency head, which generates predicted saliency maps for salient features at each layer;

[0059] Hierarchical attention fusion module, used to aggregate salient features from all layers;

[0060] Adaptive weighted hybrid loss function for deep supervision;

[0061] First, an encoder based on ResNet50 and CBR is used to obtain feature maps of five levels. Then, the contrast enhancement module is used to enhance the local context and supplement the local details. The decoder is then used to fuse deep semantics and local details to extract salient objects at the same time. Finally, pixel-level, region-level, and image-level hybrid losses with adaptive weights are introduced to improve the detection effect in complex backgrounds, shadows, and when salient objects are similar to the background. By combining global semantics with contextual features in this way, the final result of optical remote sensing image saliency detection retains deep global semantic information while containing low-level detail information, thereby more accurately and effectively suppressing shadows and distinguishing salient objects that are similar to the background.

[0062] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the device where the computer-readable storage medium is located executes the optical remote sensing image saliency detection method based on contrast enhancement and hierarchical attention fusion.

[0063] An electronic device comprises: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the optical remote sensing image saliency detection method based on contrast enhancement and hierarchical attention fusion is implemented.

[0064] The present invention takes saliency detection of optical remote sensing images as the application background. Aiming at the problems that the models trained by the existing saliency detection methods are affected by shadows and the detection effect is poor when the salient target and the background are similar, a saliency detection method, system, storage medium and electronic device for optical remote sensing images based on contrast enhancement and hierarchical attention fusion are proposed, which improves the perception of salient targets under the conditions of shadow interference and the similarity of salient targets and backgrounds. Specifically, the encoder constructed by the pre-trained model based on truncated ResNet50's last three layers and CBR obtains the feature maps of five levels; the local context is enhanced and the local details are supplemented by the contrast enhancement module; the decoder fuses the deep semantics and local details to extract the salient targets at the same time; the pixel-level, region-level and image-level mixed losses with adaptive weights are introduced to improve the detection effect under the conditions of complex backgrounds, shadows and the similarity of salient targets and backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 It is a schematic diagram of the process of the present invention.

[0067] Figure 2 This is a diagram of the backbone network structure of the present invention.

[0068] Figure 3 4 is a structural diagram of the contrast enhancement module of the present invention.

[0069] Figure 4 It is a structural diagram of the decoder of the present invention.

[0070] Figure 5 It is a structural diagram of the hierarchical attention fusion module of the present invention.

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] like Figure 1 As shown, the present invention comprises the following steps:

[0073] Step 1, preprocess the optical remote sensing image samples so that the optical remote sensing image samples are amplified and enhanced in turn; in actual use, the data set in this application is amplified: the data set must include remote sensing images of different categories and environments to avoid single images and weak network generalization ability; secondly, the data set is amplified to make up for the problem of small number of data set images to reduce the risk of overfitting. The data sets used in the embodiment of the present invention are ORSSD and EORSSD. The ORSSD training set contains 600 images and corresponding true value maps, and the test set contains 200 images and corresponding true value maps; the EORSSD training set contains 1400 images and corresponding true value maps, and the test set contains 600 images and corresponding true value maps. The training sets of ORSSD and EORSSD are enhanced by rotating at 90°, 180° and 270°, and all samples in the training set are mirror-flipped. In this way, the training set of ORSSD contains 4800 samples, and the training set of EORSSD contains 11200 examples, achieving an eight-fold amplification effect.

[0074] In the step 1, data enhancement of the optical remote sensing image data specifically includes the following steps: scaling the optical remote sensing image samples obtained after amplification to a uniform size, and then randomly rotating and cropping them in sequence to enhance the data to obtain data as training samples. When used specifically, all samples are uniformly scaled to a size of 256×256 by preprocessing the training set, and a 224×224 size image is randomly cropped from the scaled image.

[0075] Step 2, establish a saliency detection model CEHAFNet; the step 2 specifically includes the following steps: create a CEHAFNet of an encoder-decoder architecture, including a contrast enhancement module, a decoder, a saliency head, and a hierarchical attention fusion module; wherein the encoder and the decoder are both five layers, the encoder is a ResNet50 pre-trained model and 5 CBR blocks, the decoder consists of three parts: decoding Decode, upsampling Up, and fusion F, wherein the deepest decoder Decoder5 does not contain fusion F, and there is a contrast enhancement module between the corresponding layers of the encoder and decoder of the first four layers; the contrast enhancement module enhances the contextual features of the first four layers; the decoder decodes the global semantic features of the last layer and fuses them with the enhanced contextual features of the first four layers layer by layer and decodes them again to identify salient targets and salient areas; the saliency head generates a predicted saliency map for the salient targets of each layer; the hierarchical attention fusion module is used to aggregate salient features of different layers;

[0076] In actual use, create a CEHAFNet network model (such as Figure 2 ), is the encoder-decoder architecture, including the contrast enhancement module (such as Figure 3 ), decoder (such as Figure 4 ), salient head, hierarchical attention fusion module (such as Figure 5 ). The encoder and decoder are both five layers. The encoder is a pre-trained model with the last three layers of ResNet50 cut off and 5 CBR blocks with 3 convolution kernels. The decoder consists of Decode, Up, and F (the decoder Decoder5 of the last layer does not contain F, and the input F i The output is E5, and the output is D5). There is a contrast enhancement module between the corresponding layers of the encoder and decoder of the first four layers. The contrast enhancement module enhances the context features of the first four layers; the decoder decodes the encoded features of the last layer and fuses them with the enhanced context features of the first four layers layer by layer and then decodes them again, preliminarily identifying the salient features D1, D2, D3, D4, and D5 of each layer; the salient head predicts the salient maps S1, S2, S3, S4, and S5 of each layer, and the hierarchical attention fusion module aggregates D1, D2, D3, D4, and D5 and outputs the final salient map S0.

[0077] Step 3: Train the saliency detection model CEHAFNet using the enhanced optical remote sensing image samples obtained in step 1 to obtain the optimal saliency detection model CEHAFNet;

[0078] The step 3 specifically includes the following steps:

[0079] Step 3.1: Input the training samples in step 1 into the saliency detection model CEHAFNet created in step 2;

[0080] Step 3.2: The input training sample is first extracted by truncating the last three layers of the pre-trained model of ResNet50 to obtain a multi-scale feature map, and then the multi-level features E1, E2, E3, E4, and E5 of the encoder are obtained through the CBR block; specifically, the optical remote sensing image is first encoded through the ResNet50 pre-trained model and 5 CBR blocks to obtain 5 feature maps E1, E2, E3, E4, and E5. The specific method is as follows: the last average pooling, full connection, and softmax layers of ResNet50 are truncated, and then ResNet50 is divided into 5 blocks according to the resolution size of the feature map; each block is then processed with a CBR with a convolution kernel size of 3 to obtain the encoder output E1, E2, E3, E4, and E5. Among them, the sizes of the feature maps output after 5 encodings are 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original input size, and the number of channels are 64, 256, 512, 1024 and 2048 respectively; CBR stands for ReLU(BatchNorm(Conv(f))), f is any input feature, and Conv is a two-dimensional convolution with a convolution kernel of 3.

[0081] Step 3.3, input E4, E3, E2, and E1 into the contrast enhancement module in sequence to obtain enhanced context features CE4, CE3, CE2, and CE1; process the multi-level features E1, E2, E3, and E4 obtained in step 3.3 using the contrast enhancement module, which specifically includes the following steps: The contrast enhancement module is divided into two parts: a contrast attention module CRA and a CBR block, the CBR block extracts the global context information of the input features, and the CRA focuses on the local details of the input features;

[0082] First, use CRB with a convolution kernel size of 1 to extract the global context information of the current layer:

[0083] x_global i =CBR(E i ),i=1,2,3,4

[0084] The number of channels of multi-level features E1, E2, E3, and E4 is consistent with the input;

[0085] Then the CRA submodule enhances local details as follows: First, extract the local features x of Ei local_i , the difference between the local response and its average value is obtained by pooling the local features, i.e., the contrast feature [7]; the contrast feature x that needs to be paid attention to is learned after being activated by convolution, batch normalization, and Tanh activation function. contrast_i , and then change the number of channels through a two-dimensional convolution with a convolution kernel of 1, and convert x contrast_i Mapped to the output dimension and predicting the contrast score of the region of interest,

[0086] Then, Sigmoid is used to map the obtained contrast attention score to [0, 1] to obtain the final score E cra_score_i ;

[0087] Multiply the input multi-level features E1, E2, E3, E4 with their respective contrast attention scores to obtain the enhanced local features x_local i , and then fuse it with the global context information through element-by-element addition to obtain the output CE1, CE2, CE3, and CE4 of the contrast enhancement module, so that the final output contains both global context information and fine-grained contrast features, effectively improving the detection ability of targets in complex backgrounds and suppressing background noise;

[0088] x local_i =Conv 3×3 (E i ),i=1,2,3,4

[0089] x local_avg_i =AvgPool(x local_i )

[0090] x contrast_i = Tanh(BatchNorm(Conv 3×3 (x local_i -x local_avg_i )))

[0091] E cra_score_i =σ(Conv 1×1 (x contrast_i ))

[0092] x_local i =E i ×E cra_score_i

[0093] CE i =x_global i +x_local i

[0094] Among them, i=1,2,3,4, Ei Indicates E1, E2, E3, and E4. Their channel numbers and inputs are consistent; CE i represents the output CE1, CE2, CE3, and CE4 of the contrast enhancement module; CBR represents ReLU (BatchNorm (Conv (f))) with a convolution kernel size of 3, where f is an arbitrary feature and Conv 3×3 Represents a two-dimensional convolution with a convolution kernel of 3, Conv 1×1 Represents a two-dimensional convolution with a convolution kernel of 1, AvgPool represents a two-dimensional pooling with a pooling kernel of 3 and a stride of 1, BatchNorm represents batch normalization, Tanh represents the Tanh activation function, and σ represents Sigmoid.

[0095] Step 3.4: Input the global semantic feature map E5 obtained in step 3.2 into the decoder for decoding to obtain D5, fuse D5 with the enhanced context feature CE4 in step 3.3 through F to obtain F4, and then obtain the salient feature D4 of the fourth layer through Decode and Up; fuse the salient feature D4 of the fourth layer with CE3 through F and then through Decode and Up to obtain the salient feature D3 of the third layer; fuse D3 with CE2 through F and then through Decode and Up to obtain the salient feature D2 of the second layer; fuse D2 with CE1 through F and then through Decode and Up to obtain the salient feature D1 of the first layer;

[0096] Step 3.5: predict the predicted saliency maps S1, S2, S3, S4, S5 corresponding to the salient features D1, D2, D3, D4, D5 through the saliency head, and obtain the final predicted saliency map S0 after aggregating the salient features D1, D2, D3, D4, D5 through the hierarchical attention fusion module; the final predicted saliency map S0 is obtained after aggregating the salient features D1, D2, D3, D4, D5 through the hierarchical attention fusion module described in step 3.5; the specific process is:

[0097] The hierarchical attention aggregation module receives the salient features D1, D2, D3, D4, and D5 of all levels in step 3.4, and first adjusts D1, D2, D3, D4, and D5 to the size of D1 through bilinear interpolation.

[0098] Then, the number of channels of each feature map is unified into the feature map D with a target channel number of 32 through 1×1 convolution. i , in order to facilitate further integration;

[0099] The dimensions of each processed feature map are [B, C, H, W]. By stacking the feature maps of each layer, a new dimension L is generated after the first dimension to represent the level, thus forming a tensor D of [B, L, C, H, W] dimensions containing multiple levels of information. stack ;

[0100] Next, reshape it to the shape of [B, L, C, H×W], flatten the feature map of each layer to retain the spatial information, and aggregate the features of different levels through 1×1 convolution;

[0101] Then, the new features formed by aggregation are subjected to global average pooling and maximum pooling operations respectively. The pooling results are processed by the fully connected layer and the ReLU activation function. The two features are added together and the attention weight is generated through the Softmax function. The attention weight is expanded to [B, 1, 1, 1, 1] through the reshape operation, so that the weighted attention′ is consistent with the superimposed feature map D stack The dimensions of the , so that they can act correctly on each level and channel during subsequent weighting;

[0102] Finally, the final saliency map is generated by weighted summation in the L dimension and a 1×1 convolution layer;

[0103] D i ′=Conv 1×1 (interpolate(D i ))

[0104] D stack =Stack(D i ′, dim=1)

[0105] D s ' tack =Reshape(D stack )

[0106] attention=Softmax(ReLU(fc1(AdaptAP(D s ' tack )))+ReLU(fc1(AdaptMP(D s ' tack ))))

[0107] attention′=Reshape(attention)

[0108] S0=Conv 1×1 (Sum(attention′×D stack ))

[0109] Among them, i = 1, 2, 3, 4, 5; B represents the batchsize dimension, L represents the level dimension, C represents the channel dimension, H and W represent the height and width dimensions; it represents a bilinear interpolation operation, Stack represents superimposing the feature map after dimension B and generating a new dimension; AdaptAP and AdaptMP represent adaptive averaging and adaptive maximum pooling respectively; Sum represents summing along the L dimension.

[0110] In actual use, the saliency map prediction is divided into two parts. First, D1, D2, D3, D4, and D5 obtained in G are passed through the saliency head (such as Figure 1 ) outputs the saliency maps S1, S2, S3, S4, S5 of each layer, and then inputs D1, D2, D3, D4, D5 into the hierarchical attention aggregation module to obtain the final saliency map S0.

[0111] SalHead(f)=σ(CBR(f))

[0112] S i =SalHead(D i ),i=1,2,3,4,5

[0113] Among them, f represents the salient features D1, D2, D3, D4, and D5; CBR represents ReLU (BatchNorm (Conv (f))) with a convolution kernel size of 3. The hierarchical attention aggregation module performs unified channel mapping and attention weighted fusion on the inputs of multiple different feature layers to extract features that are conducive to saliency detection.

[0114] Step 3.6: Introduce a multi-level mixed loss function with adaptive weights at the pixel level, region level, and image level, and calculate the loss between the predicted saliency maps S0, S1, S2, S3, S4, S5 obtained in step 3.5 and the corresponding label maps; the multi-level adaptive mixed loss function described in step 3.6 is specifically: multi-level mixed loss L(s) = Σα with adaptive weights at the pixel level, region level, and image level k f k (s); f k represents pixel-level, region-level, and image-level losses, α k Represents the weight corresponding to the corresponding level of loss;

[0115] Pixel level loss Calculate the difference between the predicted pixel and the true pixel;

[0116] Regional level losses Measure the similarity between the predicted area and the true area;

[0117] Image-level loss Evaluate the overall overlap between the predicted image and the true image; The model can automatically adjust the weights of each loss function according to real-time performance statistics during training, forming a weight-adaptive hybrid loss function L; where G x,y represents the true label at pixel (x, y), P x,y is the probability value of the predicted pixel, N is the total number of pixels; they are the average values ​​of the predicted image and the true value image in the region, σ x and σ y is the standard deviation of the predicted image and the true image in the region, σ xy is the covariance between the predicted graph and the true value graph, C1 = 0.01 2 , C2=0.03 2 is a small constant added to avoid the denominator being zero; k is the loss function f k is an approximation of the recent rate of change of , and β is a tunable hyperparameter with a default value of 0.1.

[0118] In actual use, the pixel level loss L pixel The BCE loss

[11] is used to calculate the difference between the predicted pixels and the actual pixels, and the region-level loss L region SSIM is used as the loss function

[12] to measure the similarity between the predicted region and the true region. The image-level loss L image The IoU loss function is used to evaluate the degree of overlap between the predicted image and the real image as a whole.

[0119] Based on Softadapt

[13] , the weight of each part of the loss is dynamically calculated, so that the model can automatically adjust the weight of each loss function according to the real-time performance statistics during the training process, forming a weight-adaptive hybrid loss function L, which is used to calculate the total loss L between the predicted saliency maps S0, S1, S2, S3, S4, S5 and the corresponding true value maps. total For each loss f k , whose weight is α k , f k Represents pixel-level, region-level, and image-level loss functions, k=1,2,3.

[0120]

[0121] L(s)=∑α k f k (s)

[0122]

[0123] Where s represents the predicted saliency maps S0, S1, S2, S3, S4, S5; G x,y represents the true label at pixel (x, y), P x,yis the probability value of the predicted pixel, N is the total number of pixels; they are the average values ​​of the predicted image and the true value image in the region, σ x and σ y is the standard deviation of the predicted image and the true image in the region, σ xy is the covariance between the predicted graph and the true value graph, C1 = 0.01 2 , C2=0.03 2

[14] is a small constant added to avoid the denominator being zero; s k is the loss function f k is an approximation of the recent rate of change of , and β is a tunable hyperparameter with a default value of 0.1

[13] .

[0124] Step 3.7: After the training is completed, all parameter information of the trained model is saved.

[0125] In practical application, the present application also involves a decoding stage, in which E5 and CE1, CE2, CE3, and CE4 obtained in the above steps are input into a decoder. First, the global semantic feature E5 is input into Decoder5 (such as Figure 5 , Decoder5 does not contain F, input F i is E5, the output is D5), and then upsampled by Up (such as Figure 5 ) to obtain the preliminary salient target D5 (in order not to lose spatial information, the transposed convolution is used to restore the resolution, and the number of channels between adjacent levels is transformed, and then Conv 3×3 The upsampled features are further refined to prevent checkerboard artifacts [8] from affecting detection).

[0126] CA(f)=σ(fc2(ReLU(fc1(AvgPool(f)))))+fc2(ReLU(fc1(MaxPool(f)))))

[0127] SA(f)=σ(Conv 7×7 (cat(Avg(f),Max(f))))

[0128] Up(f) = ReLU(BatchNorm(Conv 3×3 (ReLU(BatchNorm(ConvTranspose(f))))))

[0129] D5=Decode(E5)

[0130] Among them, f represents any input feature; σ represents Sigmoid; fc1 represents a two-dimensional convolution with a kernel size of 1, which reduces the dimension to 1 / 16 of the input dimension (which can be adjusted by ratio); fc2 represents a two-dimensional convolution with a kernel size of 1, which increases the dimension to the original input dimension; ReLU represents the ReLU activation function; AvgPool represents average pooling; MaxPool represents maximum pooling; Avg and Max represent the average and maximum values ​​along the channel dimension, CA represents channel attention; SA represents spatial attention; Up represents upsampling; Conv 7×7 Denotes a two-dimensional convolution with a kernel size of 7; cat denotes concatenation in the channel dimension; ConvTranspose denotes a transposed convolution with a kernel size of 2 and a stride of 2. Decode contains two paths: the residual path and the main path. The specific operation process is as follows: Since the use of channel attention will lead to the loss of some semantic information [9], the residual path composed of convolution, batch normalization and ReLU activation is used to supplement the lost semantic information x residual ; In the main path, conv1 and bn1 extract and normalize the input features, combine CA to perform channel attention

[10] weighting, and then add it to the semantic information of the residual path to obtain the global semantic features x of the salient target semantic , and then pass it through the spatial attention SA, ReLU activation to perform spatial attention enhancement, focusing on the position x of the salient target location Then, it passes through conv2, bn2, ReLU for further feature extraction, and then passes through the dropout layer to reduce the possibility of overfitting, so as to obtain the output x of Decode d This series of steps enables the module to maintain global semantic information while capturing the details of the target, thereby improving the detection accuracy of salient targets.

[0131] x residual =ReLU(BatchNorm(Conv 3×3 (x)))

[0132] x semantic =CA(x)×x+x residual

[0133] x location =ReLU(SA(x semantic )×x)

[0134] x d =Decode(x)=Dropout(ReLU(BatchNorm(Conv 3×3 (x location ))))

[0135] Where x represents E5, F(CE i ,D i+1 ), i=1,2,3,4. D5 is fused with CE4 in step F through F to obtain F4, and after Decode and Up, the salient feature D4 of the fourth layer is obtained; D4 is fused with CE3 through F and then Decode and Up to obtain the salient feature D3 of the third layer; D3 is fused with CE2 through F and then Decode and Up to obtain the salient feature D2 of the second layer; D2 is fused with CE1 through F and then Decode and Up to obtain the salient feature D1 of the first layer. That is, except for D5, the enhanced context feature CE of the current layer is cyclically fused. i With deep distinctive features D i+1 After fusion, decoding and upsampling are performed to obtain significant features D4, D3, D2, and D1:

[0136] F(CE i ,D i+1 )=ReLU(BatchNorm(Conv 3×3 (Conv 1×1 (cat(CE i ,D i+1 ))))+D i+1 )

[0137] D i =Up(Decode(F(CE i ,D i+1 ))),i=1,2,3,4

[0138] After the training is completed, all parameter information of the trained model is saved;

[0139] Step 4: Detect the optical remote sensing image to be detected by using the optimal saliency detection model CEHAFNet to obtain the corresponding saliency map. That is, input the sample to be tested into the saliency detection model saved in step I, and output the saliency map after calculation.

[0140] The present invention also includes an optical remote sensing image saliency detection system based on contrast enhancement and hierarchical attention fusion, including an encoder, which is composed of a pre-trained model of truncated ResNet50's last three layers and CBR; a contrast enhancement module, which is used to enhance context features; a decoder, which is composed of decoding Decode, upsampling Up, and fusion F; a hierarchical attention fusion module, which is used to aggregate salient features of all layers; and an adaptive weighted hybrid loss function, which is used for deep supervision.

[0141] In the specific work, we first obtain five levels of feature maps through an encoder built based on ResNet50 and CBR; then use the contrast enhancement module to enhance the local context and supplement the local details; then use the decoder to fuse deep semantics and local details to extract salient targets at the same time; finally, introduce pixel-level, region-level, and image-level hybrid losses with adaptive weights to improve the detection effect in complex backgrounds, shadows, and when salient targets are similar to the background; by combining global semantics with contextual features in this way, the final result of optical remote sensing image saliency detection retains deep global semantic information while containing low-level detail information, thereby more accurately and effectively suppressing shadows and distinguishing salient targets that are similar to the background.

[0142] The invention of this patent aims to improve the effect of saliency detection of optical remote sensing images in the face of complex situations such as shadow interference, similarity between salient targets and background, etc. To this end, this patent enhances local details in the image through a contrast enhancement module, making it easier to distinguish when it is similar to the background; through a hierarchical attention fusion mechanism, features from different levels are effectively fused, and weights of features at different levels are adaptively allocated, thereby highlighting the focus on salient targets and reducing the interference of background information. At the same time, through layer-by-layer weighted fusion processing, the target is enhanced at multiple scales to improve the accuracy of detection; an adaptive multi-level mixed loss function is designed so that the model can balance the detection effect of salient targets at different levels, thereby enhancing the detection effect of the model under complex conditions such as shadows and similar backgrounds.

[0143] The embodiments of the present application provide a method, system, storage medium and electronic device for saliency detection of optical remote sensing images based on contrast enhancement and hierarchical attention fusion, which can implement the steps and processes of any of the above-mentioned embodiments of the method for saliency detection of optical remote sensing images based on contrast enhancement and hierarchical attention fusion, and achieve the same technical effects, which will not be described one by one here.

[0144] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the device where the computer-readable storage medium is located executes the optical remote sensing image saliency detection method based on contrast enhancement and hierarchical attention fusion as described above. The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory.

[0145] An electronic device comprises: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the optical remote sensing image saliency detection method based on contrast enhancement and hierarchical attention fusion as described above is implemented.

[0146] If the module / unit integrated in the electronic device described in this application is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0147] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0148] This patent designed two sets of experiments: the first set of experiments was conducted on the optical remote sensing image saliency detection dataset ORSSD dataset; the second set of experiments was conducted on the EORSSD dataset. In order to verify the effectiveness of the proposed CEHAFNet, the following five most advanced saliency detection methods were selected for comparison with the method of the present invention, and they were briefly introduced.

[0149] The LVNet network consists of two key modules: a dual-stream pyramid module (L-shaped module) and an encoder-decoder module with nested connections (V-shaped module). The L-shaped module hierarchically extracts a set of complementary information, which helps to perceive the diversity and local details of salient objects. The V-shaped module gradually integrates the detailed features of the encoder and the semantic features of the decoder through nested connections, suppressing complex backgrounds and highlighting salient objects. BASNet

[14] captures multi-scale features through its encoder-decoder architecture, improves boundary accuracy using a residual refinement module, and uses a fixed-weight hybrid loss function to optimize on pixels, regions, and the entire image, thereby accurately detecting salient objects in complex backgrounds. DAFNet uses a global feature aggregation module (Global Feature Aggregation, GFA) to aggregate global contextual information to achieve feature alignment and mutual reinforcement, which helps to maintain feature consistency in salient regions globally, thereby reducing the interference of complex backgrounds; Cascaded Pyramid Attention (CPA) is used to construct a cascaded pyramid framework to handle objects of different scales, and gradually refines the attention map from coarse to fine, capturing salient objects at different scales, including small objects in complex backgrounds. MJRBM

[15] uses a hierarchical attention module to extract multi-scale regional features, then generates boundary features by combining local cues and global information, and finally embeds the boundary features into the multi-scale regional features; it designs a joint learning scheme based on bidirectional feature conversion to optimize boundary and regional features at the same time to improve the detection accuracy in complex backgrounds. ERPNet

[16] generates precise edge information of salient objects through an edge extraction module, while the feature fusion module uses edge information to sharpen the position clues of the object and repeatedly corrects the location of the salient object during the decoding process. ERPNet provides clearly defined boundaries while maintaining the integrity of the salient object, thereby effectively highlighting the salient object and suppressing the interference of the complex background.

[0150] Table Ⅰ is a comparative experiment on the ORSSD dataset, and Table Ⅱ is a comparative experiment on the EORSSD dataset. The widely used evaluation indicators include S-measure (S α ,α=0.5)

[17] ; mean absolute error (MAE); adaptive, average and maximum F-measure (F β ,β 2=0.3)

[18] ; adaptive, average and maximum E-measure

[19] are used to comprehensively measure the performance of our CEHAFNet and other comparison methods. Specifically, S-measure measures the structural similarity of region perception and object perception at the same time, and evaluates the consistency between the predicted results and the true target in shape and space; MAE is used to evaluate the average error between the predicted results and the true label at the pixel level, reflecting the overall prediction accuracy; F-measure is a weighted harmonic average that combines precision and recall, including adaptive (adp), average (mean) and maximum (max) F-measure, to comprehensively consider the performance of the model under different thresholds; E-measure includes adaptive (adp), average (mean) and maximum (max) E-measure, which is used to measure the consistency and stability of the model locally and globally; the calculation of these indicators is as follows:

[0151] S α =α×S o +(1-α)×S r , α=0.5

[0152]

[0153] Among them, S o is the object perception of structural similarity, S r is the region perception of structural similarity; setting β 2 Different weight values ​​are used to emphasize the importance of recall or precision (β 2 =0.3); W and H represent the width and height of the image, S and P represent the normalized predicted saliency map and true value map, (x, y) represents the pixel coordinates; ξ s It is an enhanced alignment matrix. Among these performance indicators, only MAE has a lower value, the better the performance, while the other indicators have a higher value, the better the performance.

[0154] Table I Comparative experiments on the ORSSD dataset

[0155]

[0156]

[0157] Table II Comparative experiments on the EORSSD dataset

[0158]

[0159] From the data in the above two tables, we can see that CEHAFNet performs better than other methods in saliency detection on ORSSD and EORSSD datasets. Specifically, on the ORSSD dataset, CEHAFNet performs better than other methods in saliency detection on ORSSD and EORSSD datasets. α 、MAE、adpF β 、meanF β 、adpE ξ 、meanE ξ In terms of evaluation indicators, its values ​​are significantly better than other methods, especially adpF β , an improvement of 2.07%; on the EORSSD dataset, CEHAFNet also shows superior performance. α 、MAE、adpF β 、meanF β 、maxF β 、adpE ξ 、meanE ξ In terms of evaluation indicators, its values ​​are significantly better than other methods, especially in meanF β and meanE ξ In terms of performance, the performance of CEHAFNet has been improved by 2% and 1.92% respectively, indicating that CEHAFNet can more accurately capture salient targets when dealing with complex scenes. CEHAFNet roughly locates the salient target area by extracting global semantic features; refines the target layer by layer based on contrast-enhanced contextual features to improve the fine-grained detection accuracy; uses hierarchical attention to aggregate semantic contextual information from different layers, thereby enhancing the model's perception of salient targets in complex backgrounds; this patent also introduces pixel-level, region-level, and image-level hybrid losses with adaptive weights to improve the detection effect in complex backgrounds, shadows, and when the salient target and background are similar. These experimental results verify the effectiveness and advancement of CEHAFNet in saliency detection of remote sensing images.

[0160] The references in the invention patent are as follows:

[0161] [1]Li G, Liu Z, Zeng D, et al. Adjacent context coordination network for salient object detection in optical remote sensing images [J]. IEEE Transactions on Cybernetics, 2022, 53(1): 526-538.

[0162] [2]Li C,Cong R,Hou J,et al.Nested network with two-stream pyramid forsalient object detection in optical remote sensing images[J].IEEETransactions on Geoscience and Remote Sensing,2019,57(11):9156-9166.

[0163] [3]Gu Y,Xu H,Quan Y,et al.ORSI salient object detection viabidimensional attention and full-stage semantic guidance[J].IEEE Transactionson Geoscience and Remote Sensing,2023,61:1-13.

[0164] [4]Huang Z,Chen H,Liu B,et al.Semantic-guided attention refinementnetwork for salient object detection in optical remote sensing images[J].Remote Sensing,2021,13(11):2163.

[0165] [5]Zhang Q,Cong R,Li C,et al.Dense attention fluid network forsalient object detection in optical remote sensing images[J].IEEETransactions on Image Processing,2020,30:1305-1317.

[0166] [6]Li J,Li C,Zheng X,et al.Global context relation-guided featureaggregation network for salient object detection in optical remote sensingimages[J].Remote Sensing,2024,16(16):2978.

[0167] [7]Luo Z,Mishra A,Achkar A,et al.Non-local deep features for salientobject detection[C] / / Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition.2017:6609-6617.

[0168] [8]Zhang P,Wang D,Lu H,et al.Learning uncertain convolutionalfeatures for accurate saliency detection[C] / / Proceedings of the IEEEInternational Conference on Computer Vision.2017:212-221.

[0169] [9]Lin Y,Sun H,Liu N,et al.Attention guided network for salientobject detection in optical remote sensing images[C] / / InternationalConference on Artificial Neural Networks.Cham:Springer InternationalPublishing,2022:25-36.

[0170]

[10] Woo S,Park J,Lee J Y,et al.CBAM:Convolutional block attentionmodule[C] / / Proceedings of the European Conference on Computer Vision(ECCV).2018:3-19.

[0171]

[11] De Boer P T,Kroese D P,Mannor S,et al.A tutorial on the cross-entropy method[J].Annals of Operations Research,2005,134:19-67.

[0172]

[12] Wang Z,Simoncelli E P,Bovik A C.Multiscale structural similarityfor image quality assessment[C] / / The Thrity-Seventh Asilomar Conference onSignals,Systems&Computers,2003.IEEE,2003,2:1398-1402.

[0173]

[13] Heydari AA,Thompson C A,Mehmood A.Softadapt:Techniques foradaptive loss weighting of neural networks with multi-part loss functions[J].arXiv:1912.12355,2019.

[0174]

[14] Qin X,Zhang Z,Huang C,et al.BASNet:Boundary-aware salient objectdetection[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision andPattern Recognition.2019:7479-7489.

[0175]

[15] Tu Z,Wang C,Li C,et al.ORSI salient object detection viamultiscale joint region and boundary model[J].IEEE Transactions on Geoscienceand Remote Sensing,2021,60:1-13.

[0176]

[16] Zhou X,Shen K,Weng L,et al.Edge-guided recurrent positioningnetwork for salient object detection in optical remote sensing images[J].IEEETransactions on Cybernetics,2022,53(1):539-552.

[0177]

[17] Fan D P,Cheng M M,Liu Y,et al.Structure-measure:A new way toevaluate foreground maps[C] / / Proceedings of the IEEE International Conferenceon Computer Vision.2017:4548-4557.

[0178]

[18] Achanta R,Hemami S,Estrada F,et al.Frequency-tuned salient regiondetection[C] / / 2009 IEEE Conference on Computer Vision and PatternRecognition.IEEE,2009:1597-1604.

[0179]

[19] Fan D P,Gong C,Cao Y,et al.Enhanced-alignment measure for binaryforeground map evaluation[C] / / Proceedings of the 27th International JointConference on Artificial Intelligence.2018:698-704.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0181] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0182] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

[0183] In the description of the present invention, it should be noted that directional words, such as the terms "center", "lateral", "longitudinal", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like, indicating directions and positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and cannot be understood as limiting the specific protection scope of the present invention.

[0184] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged in appropriate circumstances, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0185] Note that the above are only preferred embodiments of the present invention and the principles of the application technology. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the specific embodiments described herein, and may include more other effective embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A saliency detection method for optical remote sensing images based on contrast enhancement and hierarchical attention fusion, characterized by: It includes the following steps: Step 1, preprocessing the optical remote sensing image samples so that the optical remote sensing image samples are amplified and enhanced in sequence; Step 2: Establish a saliency detection model CEHAFNet; the step 2 specifically includes the following steps: create a CEHAFNet of an encoder-decoder architecture, including an encoder, a contrast enhancement module, a decoder, a saliency head, and a hierarchical attention fusion module; The encoder and decoder both have five layers. The encoder is a ResNet50 pre-trained model with 5 CBR blocks. The decoder consists of three parts: decoding Decode, upsampling Up, and fusion F. The deepest decoder Decoder5 does not contain fusion F. There is a contrast enhancement module between the corresponding layers of the encoder and decoder of the first four layers. The contrast enhancement module enhances the context features of the first four layers; the decoder decodes the global semantic features of the last layer and fuses them with the enhanced context features of the first four layers layer by layer and decodes them again to identify salient objects and salient areas; the salient head generates a predicted salient map for the salient features of each layer; the hierarchical attention fusion module is used to aggregate salient features of different layers; Step 3: Train the saliency detection model CEHAFNet using the enhanced optical remote sensing image samples obtained in step 1 to obtain the optimal saliency detection model CEHAFNet; Step 4: Use the optimal saliency detection model CEHAFNet to detect the optical remote sensing image to be detected and obtain the corresponding saliency map.

2. The method for saliency detection of optical remote sensing images based on contrast enhancement and hierarchical attention fusion according to claim 1, characterized in that: In the step 1, amplifying the optical remote sensing image samples specifically includes the following steps: enhancing the training sets of ORSSD and EORSSD by rotating at angles of 90°, 180° and 270°, and mirror-reversing all samples in the training sets; in this way, the training sets of ORSSD and EORSSD contain 4800 and 11200 samples respectively, achieving an eight-fold amplification effect.

3. The method for saliency detection of optical remote sensing images based on contrast enhancement and hierarchical attention fusion according to claim 2, characterized in that: In the step 1, data enhancement of the optical remote sensing image data specifically includes the following steps: scaling the optical remote sensing image samples obtained after amplification to a uniform size, and then randomly rotating and cropping them in sequence to enhance them, and obtain data as training samples.

4. The optical remote sensing image saliency detection method based on contrast enhancement and hierarchical attention fusion according to claim 1, characterized in that: The step 3 specifically includes the following steps: Step 3.1: Input the training samples in step 1 into the saliency detection model CEHAFNet created in step 2; Step 3.2: The input training sample is first extracted by truncating the last three layers of the pre-trained model of ResNet50 to obtain a multi-scale feature map, and then the encoder's multi-level features E1, E2, E3, E4, and E5 are obtained by the CBR block; Step 3.3, input E4, E3, E2, and E1 into the contrast enhancement module in sequence to obtain enhanced context features CE4, CE3, CE2, and CE1; Step 3.4: Input the global semantic feature map E5 obtained in step 3.2 into the decoder for decoding to obtain D5, fuse D5 with the enhanced context feature CE4 in step 3.3 through F to obtain F4, and then obtain the salient feature D4 of the fourth layer through Decode and Up; fuse the salient feature D4 of the fourth layer with CE3 through F and then through Decode and Up to obtain the salient feature D3 of the third layer; fuse D3 with CE2 through F and then through Decode and Up to obtain the salient feature D2 of the second layer; fuse D2 with CE1 through F and then through Decode and Up to obtain the salient feature D1 of the first layer; Step 3.5: Predict the predicted saliency maps S1, S2, S3, S4, S5 corresponding to the salient features D1, D2, D3, D4, D5 through the saliency head, and aggregate the salient features D1, D2, D3, D4, D5 through the hierarchical attention fusion module to obtain the final predicted saliency map S0; Step 3.6, introduce a multi-level hybrid loss function with adaptive weights at the pixel level, region level, and image level, and calculate the loss between the predicted saliency maps S0, S1, S2, S3, S4, S5 obtained in step 3.5 and the corresponding label maps; Step 3.7: After the training is completed, all parameter information of the trained model is saved.

5. The method for saliency detection of optical remote sensing images based on contrast enhancement and hierarchical attention fusion according to claim 4, characterized in that: The multi-level features E1, E2, E3, and E4 obtained in step 3.3 are processed using a contrast enhancement module, which specifically includes the following steps: the contrast enhancement module is divided into two parts: a contrast attention module CRA and a CBR block, the CBR block extracts the global context information of the input features, and the CRA focuses on the local details of the input features; First, use CRB with a convolution kernel size of 1 to extract the global context information of the current layer: x_global i =CBR(E i ),i=1,2,3,4 The number of channels of multi-level features E1, E2, E3, and E4 is consistent with the input; Then the CRA submodule enhances local details as follows: First, extract the local features x of Ei local_i , the difference between the local response and its average value is obtained by pooling the local features, that is, the contrast feature; the contrast feature x that needs to be paid attention to is learned after being activated by convolution, batch normalization and Tanh activation function. contrast_i , and then change the number of channels through a two-dimensional convolution with a convolution kernel of 1, and convert x contrast_i Mapped to the output dimension and predicting the contrast score of the region of interest, Then, the obtained contrast attention score is mapped to [0, 1] through Sigmoid to obtain the final score E cra_score_i ; Multiply the input multi-level features E1, E2, E3, E4 with their respective contrast attention scores to obtain the enhanced local features x_local i , and then fuse it with the global context information through element-by-element addition to obtain the output CE1, CE2, CE3, and CE4 of the contrast enhancement module, so that the final output contains both global context information and fine-grained contrast features, effectively improving the detection ability of targets in complex backgrounds and suppressing background noise; x local_i =Conv3×3(E i ),i=1,2,3,4 x local_avg_i =AvgPool(x local_i ) x contrast_i =Tanh(BatchNorm(Conv 3×3 (x local_i -x local_avg_i ))) AND cra_score_i =σ(Conv 1×1 (x contrast_i )) x_local i =E i ×E cra_score_i CE i =x_global i +x_local i Where i = 1, 2, 3, 4, the number of channels of E1, E2, E3, and E4 is consistent with the input; CE i represents the output CE1, CE2, CE3, and CE4 of the contrast enhancement module; CBR represents ReLU (BatchNorm (Conv (f))) with a convolution kernel size of 3, where f is an arbitrary feature and Conv 3×3 Represents a two-dimensional convolution with a convolution kernel of 3, Conv 1×1 Represents a two-dimensional convolution with a convolution kernel of 1, AvgPool represents two-dimensional pooling, BatchNorm represents batch normalization, Tanh represents the Tanh activation function, and σ represents Sigmoid.

6. The method for saliency detection of optical remote sensing images based on contrast enhancement and hierarchical attention fusion using adaptive hybrid loss according to claim 4, characterized in that: The hierarchical attention fusion module described in step 3.5 aggregates the salient features D1, D2, D3, D4, and D5 to obtain the final predicted salient map S0; the specific process is: the hierarchical attention aggregation module receives the salient features D1, D2, D3, D4, and D5 of all levels in step 3.4, First, adjust D1, D2, D3, D4, and D5 to the size of D1 through bilinear interpolation. Then, the number of channels of each feature map is unified into a feature map D with a target channel number of 32 through 1×1 convolution. i , in order to facilitate further integration; The dimensions of each processed feature map are [B, C, H, W]. By stacking the feature maps of each layer, a new dimension L is generated after the first dimension to represent the level, thus forming a tensor D of [B, L, C, H, W] dimensions containing multiple levels of information. stack ; Next, reshape it to the shape of [B, L, C, H×W], flatten the feature map of each layer to retain the spatial information, and aggregate the features of different levels through 1×1 convolution; Then, the new features formed by aggregation are subjected to global average pooling and maximum pooling operations respectively. The pooling results are processed by the fully connected layer and the ReLU activation function. The two features are added together and the attention weight is generated through the Softmax function. The attention weight is expanded to [B, 1, 1, 1, 1] through the reshape operation, so that the weighted attention′ is consistent with the superimposed feature map D stack The dimensions of the , so that they can act correctly on each level and channel during subsequent weighting; Finally, the final saliency map is generated by weighted summation in the L dimension and a 1×1 convolution layer; D i ′=Conv 1×1 (interpolate(D i )) D stack =Stack(D i ′,dim=1) D s ′ tack =Reshape(D stack ) attention=Softmax(ReLU(fc1(AdaptAP(D s ′ tack )))+ReLU(fc1(AdaptMP(D s ′ tack )))) attention′=Reshape(attention) S0=Conv 1×1 (Sum(attention′×D stack )) Among them, i = 1, 2, 3, 4, 5; B represents the batchsize dimension, L represents the level dimension, C represents the channel dimension, H and W represent the height and width dimensions; it represents a bilinear interpolation operation, Stack represents superimposing the feature map after dimension B and generating a new dimension; AdaptAP and AdaptMP represent adaptive averaging and adaptive maximum pooling respectively; Sum represents summing along the L dimension.

7. The method for saliency detection of optical remote sensing images based on contrast enhancement and hierarchical attention fusion according to claim 4, characterized in that: The multi-level adaptive hybrid loss function described in step 3.6 is specifically: multi-level hybrid loss L(s) = Σα with adaptive weights at the pixel level, region level, and image level k f k (s); f k represents pixel-level, region-level, and image-level losses, α k Represents the weight corresponding to the loss of the corresponding level; pixel-level loss Calculate the difference between the predicted pixel and the true pixel; Regional level losses Measure the similarity between the predicted area and the true area; Image-level loss Evaluate the overall overlap between the predicted image and the true image; The model can automatically adjust the weights of each loss function according to real-time performance statistics during training, forming a weight-adaptive hybrid loss function L; where G x,y represents the true label at pixel (x, y), P x,y is the probability value of the predicted pixel, N is the total number of pixels; they are the average values ​​of the predicted image and the true value image in the region, σ x and σ y is the standard deviation of the predicted image and the true image in the region, σ xy is the covariance between the predicted graph and the true value graph, C1 = 0.01 2 , C2=0.03 2 is a small constant added to avoid the denominator being zero; k is the loss function f k is an approximation of the recent rate of change of , and β is a tunable hyperparameter with a default value of 0.

1.

8. An optical remote sensing image saliency detection system based on contrast enhancement and hierarchical attention fusion, characterized by: include The encoder is composed of a pre-trained model with the last three layers of ResNet50 truncated and CBR; Contrast enhancement module, used to enhance contextual features; The decoder is composed of decoding Decode, upsampling Up, and fusion F; Saliency head, which generates predicted saliency maps for salient features at each layer; Hierarchical attention fusion module, used to aggregate salient features from all layers; Adaptive weighted hybrid loss function for deep supervision; First, an encoder based on ResNet50 and CBR is used to obtain feature maps of five levels. Then, the contrast enhancement module is used to enhance the local context and supplement the local details. The decoder is then used to fuse deep semantics and local details to extract salient objects at the same time. Finally, pixel-level, region-level, and image-level hybrid losses with adaptive weights are introduced to improve the detection effect in complex backgrounds, shadows, and when salient objects are similar to the background. By combining global semantics with contextual features in this way, the final result of optical remote sensing image saliency detection retains deep global semantic information while containing low-level detail information, thereby more accurately and effectively suppressing shadows and distinguishing salient objects that are similar to the background.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the device where the computer-readable storage medium is located executes the optical remote sensing image saliency detection method based on contrast enhancement and hierarchical attention fusion as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the optical remote sensing image saliency detection method based on contrast enhancement and hierarchical attention fusion as described in any one of claims 1 to 7 is implemented.

Images