A method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement
By employing a salient target detection method for optical remote sensing images with reliable gating of feature transfer and foreground/background residual refinement, the problems of false background response and poor target structure restoration are solved, achieving more stable cross-domain semantic feature transfer and target detection results.
Patent Information
- Application Number
- CN202611207973.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-11
- Publication Date
- 2026-11-10
AI Technical Summary
Existing optical remote sensing image salient target detection techniques are unable to effectively suppress false background responses, have poor structural restoration of weak and slender targets, and general pre-trained semantic features are difficult to stably adapt and transfer to remote sensing image detection tasks.
We employ a method that combines feature transfer reliability gating and foreground/background residual refinement. Through steps such as hierarchical feature extraction, freezing dense semantic prior feature extraction and alignment, feature transfer reliability gating, pre-decoding feature construction, pre-decoding and coarse saliency prediction, and foreground/background residual refinement, we construct a progressive collaborative architecture to manage the reliability of cross-domain semantic transfer and correct prediction errors.
The model's detection stability and generalization ability for low-contrast targets, small-sized targets, slender linear features, and multi-target mixed scenes have been improved, and its ability to suppress complex backgrounds and the integrity of target structures have been enhanced.
Smart Images

Figure CN123071850A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of digital image processing and computer vision technology, specifically to a method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement. Background Technology
[0002] Salience target detection is a pixel-level visual prediction task used to separate visually or semantically prominent target regions from images. It can provide visual priors for subsequent remote sensing intelligent interpretation tasks such as target detection, semantic segmentation, change detection, scene understanding, and information retrieval.
[0003] Optical remote sensing images are typically acquired from satellites, drones, or aerial platforms using a top-down perspective, covering a wide area and containing a rich variety of land features. A single image may simultaneously contain large areas of land features, small targets, regular man-made structures, and slender targets such as roads, rivers, and port shorelines. Due to the influence of imaging scale, shooting height, land feature distribution, and background texture, the scale, shape, orientation, contrast, and boundary features of salient targets vary significantly. Salient target detection faces the following technical challenges: (1) False background responses are difficult to suppress. Complex backgrounds such as roads, water bodies, building roofs, and farmland textures often have strong edges, high contrast, or regular structures, which easily obtain high responses and are misjudged as prominent targets.
[0004] (2) The structural restoration effect of small and slender targets is poor. Small targets such as ships, aircraft and vehicles are prone to semantic feature weakening during downsampling; slender targets such as roads, port shorelines and long buildings are prone to response breaks, holes and blurred boundaries.
[0005] (3) General pre-trained semantic features are difficult to stably adapt and transfer to remote sensing image saliency detection tasks. The training materials of existing visual pre-trained models are mainly natural images. Optical remote sensing images are taken from above, with huge differences in target scale, dense texture of ground objects, and special spatial arrangement. There is a significant domain shift between the two in terms of visual appearance and semantic distribution. Directly transferring pre-trained features will send semantic responses and data biases that do not match the remote sensing scene into the decoding stage, and at the same time weaken the edges, textures and small ground object details carried in the lower layers of the image.
[0006] Current salient target detection algorithms for optical remote sensing images are mainly divided into two categories: single-stage and two-stage. Single-stage algorithms can directly output a saliency map with only one forward inference, which is more computationally efficient. However, when faced with complex backgrounds, low-contrast weak targets, and slender features, it is difficult to suppress background noise and restore the complete structure of the target. Existing research has introduced attention mechanisms, cross-layer feature fusion, or boundary enhancement modules to optimize feature extraction, but the overall approach still focuses on foreground feature mining and lacks specific constraints against false saliency responses in the background.
[0007] Two-stage detection approaches generally employ a "coarse prediction + fine optimization" process. However, the effectiveness of the refinement process highly depends on the reliability of the initial coarse prediction results and the decoded features. If the coarse saliency map exhibits localization errors, missed target detections, or misjudgments of the background, subsequent refinement stages will perpetuate or even amplify these errors. Furthermore, if the refinement process relies solely on foreground features for optimization, it cannot effectively counteract false saliency regions using background information.
[0008] In summary, existing technologies urgently need a salient target detection method that can effectively manage the credibility of cross-domain semantic transfer and simultaneously correct prediction errors in both the foreground and background dimensions. Summary of the Invention
[0009] The technical problem to be solved by the present invention is to provide a salient target detection method for optical remote sensing images based on feature transfer reliability gating and foreground-background residual refinement, so as to solve the problems of the prior art, such as the difficulty in suppressing false background responses, poor restoration of the structure of weak and slender targets, and the difficulty in stably adapting and transferring general pre-trained semantic features to remote sensing salient detection tasks.
[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows.
[0011] A salient target detection method for optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement includes the following steps: S1. Hierarchical backbone feature extraction: The input optical remote sensing image is fed into a hierarchical feature encoder to obtain backbone features x1, x2, x3 and x4 at four scales. Each scale feature is mapped to a unified channel dimension through convolution and batch normalization to obtain low-level feature z1, mid-level features z2 and z3 and high-level feature z4. Among them, low-level feature z1 preserves the boundary, texture and elongated structure, mid-level features z2 and z3 take into account both spatial structure and semantic response, and high-level feature z4 is used for global semantic localization. S2. Frozen Dense Semantic Prior Feature Extraction and Alignment: A pre-trained semantic encoder is set up in parallel with the hierarchical feature encoder. The input image is inversely normalized and then fed into the frozen pre-trained semantic encoder, outputting dense semantic prior features T. For mid-to-high-level feature scales, T is bilinearly interpolated to the spatial size of the corresponding backbone features, and channel projection is completed using 1×1 convolution to obtain the feature transfer reliability cue R. i ; S3. Feature Transfer Reliability Gating: For each mid-to-high-level scale, the backbone feature z is gated through a gating mapping module. i With the corresponding feature transfer reliability clue R i The concatenation is performed along the channel dimension using a lightweight mapping function φ, which includes convolution, normalization, and ReLU activation.i And the Sigmoid activation function, to obtain the pixel-level reliability-gated map G. i The reliability enhancement feature z is obtained by modulating the original backbone features using residual multiplication. i r Furthermore, the lower-level feature z1 is not subjected to gating processing to maintain the original transmission path; S4. Pre-decoding Feature Construction: The highest-level reliability enhancement feature z4 is constructed using the pre-decoding feature construction module. r Input the spatial attention module to obtain high-level localization features H; input the low-level features z1 sequentially into the oriented convolutional unit and the spatial attention module to obtain low-level structural features L; input the adjacent middle-level reliability enhancement features z2... r z3 r Input the cross-layer knowledge transfer module to obtain the mid-layer semantic structure feature M; H, M and L constitute the complementary input feature group of the pre-decoder; S5. Pre-decoding and coarse saliency prediction: The pre-decoder first upsamples H to the scale of M and injects global localization information through element-wise multiplication; then it upsamples the fusion result to the scale of L and multiplies it element-wise with L to preserve the high-resolution structure; the fusion results at each scale are further progressively integrated through channel concatenation and convolution to generate the decoding feature D. r ; For D r Perform a 1×1 convolution to obtain a coarse significance log probability plot S0; S6. Foreground-Background Residual Refinement: The foreground-background residual refinement module transforms the coarse significance log probability map S0 into a coarse significance probability map S using the Sigmoid function. c ; Using S c and its complementarity Figure 1 -S c Decoding feature D r Element-wise weighting is performed separately to explicitly construct the foreground response feature D. f and background response features D b The foreground convolution branch and the background convolution branch process D separately. f and D b The two are concatenated along the channel dimension and input into the fusion convolution to obtain the residual log probability map R. The refined log probability map is S=S0+R. S7. Output: Perform a 4-fold bilinear upsampling on the thinned logarithmic probability map S to obtain the final logarithmic probability map restored to the spatial resolution of the input image. ,Will The significance probability map is obtained by Sigmoid transformation. And then Normalization and scaling are performed to obtain the final grayscale saliency map; S8. Loss Function Construction and Model Training: During training, the sum of the binary cross-entropy log probability loss and the cross-union ratio loss is used as the overall significance supervision target, and only for the final log probability map. Supervision is performed without introducing boundary loss or multi-scale auxiliary supervision; the pre-trained semantic encoder is kept frozen, and only the parameters of the hierarchical feature encoder, gating mapping module, pre-decoding feature construction module, pre-decoder and foreground-background residual refinement module are updated.
[0012] Preferably, in step S2, the pre-trained semantic encoder uses a frozen RADIO model; the specific formula for the inverse normalization process is: I R = clamp(I norm ×std+mean, 0, 1) Among them, I norm For the input image normalized according to the backbone encoder requirements, mean and std are the corresponding mean and standard deviation, respectively. R The parameters of the pre-trained semantic encoder are not included in gradient updates during training and are used as input to the semantic encoder of the RADIO model.
[0013] Preferably, in step S3, the formula for the residual multiplication modulation is: z i r = z i ⊙ (1 + αG i ), i∈{2,3,4} Where α is the scaling factor, and ⊙ represents element-wise multiplication; when the gating response is low, the residual path retains the original backbone features, and when the gating response is high, the semantic representation of the corresponding position is enhanced.
[0014] Preferably, in step S6, the foreground response feature D f Used to compensate for missed detection responses in the target region and repair the complete target structure; the background response feature D b Used to describe potential spurious regions and suppress various spurious responses; the residual log probability map R is added back to the coarse significance log probability map S0 as a correction amount, without replacing the overall structure of the coarse significance prediction.
[0015] Preferably, in step S6, the foreground response feature D f and background response features D b The formula for generating it is: S c = σ(S0) D f = D r ⊙ S c Db = D r ⊙ (1-S c ) Where σ is the Sigmoid activation function.
[0016] Preferably, in step S7, the formula for generating the final grayscale saliency map P is: Here, Normalize represents the min-max normalization operation.
[0017] Preferably, in step S8, the formula for the overall saliency supervision target is: in, The overall saliency monitoring objective; The log-probability loss is the binary cross-entropy. G This is the corresponding true saliency label.
[0018] Preferably, in step S8, the binary cross-entropy logarithmic probability loss is calculated using a numerically stable form to avoid numerical overflow caused by directly taking the logarithm when the Sigmoid probability is close to 0 or 1. The specific formula is as follows: in, N The total number of pixels contained in a training image; Let be the saliency score of the i-th pixel after upsampling; This is the true saliency label for the i-th pixel, with values {0,1}.
[0019] Preferably, in step S8, the expansion of the crossover ratio loss is: in, ε To prevent numerical stability constants with a denominator of zero; Let be the predicted significance probability of the i-th pixel.
[0020] The technological advancements achieved by this invention are as follows, thanks to the adoption of the above technical solutions.
[0021] This invention utilizes a feature transfer reliability gating mechanism to construct a position-related reliability gating map from pre-trained semantic features after scale alignment and channel projection. It selectively modulates high-level features in the backbone while maintaining the original transmission path for low-level features. This approach can enhance the feature expression of the target region with reliable general semantics and filter unreliable cross-domain feature responses, effectively reducing the domain differences between the pre-trained model of natural images and optical remote sensing images, while fully preserving the inherent boundary, texture, and fine structural information of remote sensing images.
[0022] This invention modulates the original backbone features using a residual multiplication method through a pixel-level reliability gating map. When the gating response is low, the original backbone features of the residual path remain basically unchanged. When the gating response is high, the semantic representation of the corresponding position is enhanced. This achieves a protective transfer of cross-domain semantic features by "enhancing reliable features and maintaining unreliable ones," reducing the interference of unreliable cross-domain responses on the decoding stage and improving the stability of the transfer of pre-trained semantic features to optical remote sensing saliency detection tasks.
[0023] This invention employs a foreground-background residual refinement mechanism, using a coarse saliency probability map and its complementary map to construct foreground and background responses, respectively. After extraction by independent branches, these responses are fused into a residual correction amount and added back to the coarse prediction result. This allows the foreground branch to compensate for missed responses in the target region and repair the complete target structure, while the background branch can suppress false saliency responses caused by complex backgrounds such as roads, water bodies, and building textures. Thus, without compromising the overall structure of the coarse prediction, the coherence of the target edge and the completeness of the foreground region are simultaneously optimized.
[0024] This invention sequentially connects feature transfer reliability gating and foreground / background residual refinement to form a progressive collaborative architecture. The feature end controls the reliable input of cross-domain semantics from the source to ensure the reliability of decoded features. On this basis, the prediction end performs differentiated correction for two types of errors: foreground missed detection and background false detection. The two modules form a benign collaboration through end-to-end training, jointly improving the model's detection stability and generalization ability for low-contrast targets, small-sized targets, thin linear features, and multi-target mixed scenes.
[0025] In this invention, the pre-trained semantic encoder remains frozen during training, serving only as an external semantic prior provider and not participating in gradient updates. Furthermore, the residual correction is added back to the coarse saliency log probability map in an additive manner, without altering the basic output structure of the pre-decoder. Therefore, this invention is easy to integrate into existing two-stage saliency target detection frameworks, exhibiting good framework compatibility and plug-and-play characteristics. Attached Figure Description
[0026] Figure 1 This is a flowchart of the present invention; Figure 2 This is a diagram of the overall network structure of the present invention; Figure 3 This is a schematic diagram of the gated mapping module structure of the present invention; Figure 4 This is a schematic diagram of the foreground-background residual refinement module structure of the present invention; Figure 5 This is a comparison chart of the PR curve and F-measure curve on three publicly available datasets of this invention; Figure 6This is a visual ablation comparison image of the gated mapping module and the foreground-background residual refinement module of the present invention; Figure 7 This is a qualitative comparison of the saliency maps of the present invention and various existing methods in complex backgrounds and scenes with slender targets; Figure 8 This is a qualitative comparison of the saliency maps of the present invention and various existing methods in small-object and multi-object scenarios. Detailed Implementation
[0027] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0028] A salient target detection method for optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement employs a sequentially connected coarse saliency prediction stage and a foreground / background residual refinement stage. The coarse saliency prediction stage includes hierarchical backbone feature extraction, frozen dense semantic prior feature extraction and alignment, feature transfer reliability gating, pre-decoding feature construction, and pre-decoding and coarse saliency prediction. The foreground / background residual refinement stage uses coarse saliency probability to decompose the decoded features into foreground and background responses and predicts the residual correction amount.
[0029] The input is an optical remote sensing image I. The first stage outputs a single-channel coarse significance log probability plot S0 and decoded features D. r The second stage outputs a logarithmic probability map R of the residuals, which is then refined into a logarithmic probability map S = S0 + R. S is upsampled to the input image resolution and subjected to a probability transformation to obtain the salient target detection result.
[0030] Combination Figures 1 to 2 As shown, it includes the following steps: S1. Hierarchical backbone feature extraction.
[0031] The input optical remote sensing image is fed into a hierarchical feature encoder to obtain backbone features x1, x2, x3 and x4 at four scales. Each scale feature is mapped to a unified channel dimension through convolution and batch normalization to obtain low-level feature z1, mid-level features z2 and z3 and high-level feature z4. Among them, low-level feature z1 preserves boundaries, texture and elongated structure, mid-level features z2 and z3 take into account both spatial structure and semantic response, and high-level feature z4 is used for global semantic localization.
[0032] S2. Freeze dense semantic prior feature extraction and alignment.
[0033] A pre-trained semantic encoder is set up in parallel with the hierarchical feature encoder. The input image is inversely normalized and then fed into the frozen pre-trained semantic encoder, which outputs dense semantic prior features T. For mid-to-high-level feature scales, T is bilinearly interpolated to the spatial size of the corresponding backbone features, and channel projection is completed using 1×1 convolution to obtain the feature transfer reliability cue R. i .
[0034] In step S2, the pre-trained semantic encoder uses a frozen RADIO model; the specific formula for the inverse normalization process is as follows: I R = clamp(I norm ×std+mean, 0, 1) Among them, I norm For the input image normalized according to the backbone encoder requirements, mean and std are the corresponding mean and standard deviation, respectively. R The parameters of the pre-trained semantic encoder are not included in gradient updates during training and are used as input to the semantic encoder of the RADIO model.
[0035] like Figure 3 As shown, in step S2, for i∈{2,3,4}, the dense semantic prior features T are bilinearly interpolated to the i-th layer backbone features z. i The spatial dimensions are determined, and the corresponding 1×1 convolution P is used. i Complete the channel projection to obtain the z-axis. i Same-size feature migration reliability clue R i : R i = P i (Resize(T, size(z i ))), i∈{2,3,4} Where Resize(·, size(z) i )) represents bilinear interpolation to z i Space dimensions.
[0036] S3. Feature migration reliability gating.
[0037] For each mid-to-high level scale, the backbone feature z is mapped using a gating mapping module. i With the corresponding feature transfer reliability clue R i The concatenation is performed along the channel dimension using a lightweight mapping function φ, which includes convolution, normalization, and ReLU activation. i And the Sigmoid activation function, to obtain the pixel-level reliability-gated map G. i The reliability enhancement feature z is obtained by modulating the original backbone features using residual multiplication. i rFurthermore, the low-level feature z1 is not subjected to gating processing to maintain the original transmission path, thereby preserving its boundary, texture, and fine structural information.
[0038] like Figure 3 As shown, in step S3, for i∈{2,3,4}, the backbone feature z i Corresponding feature transfer reliability clue R i Channel dimensions are concatenated via a lightweight mapping function φ i And the Sigmoid activation function is used to obtain the pixel-level reliability-gated map G. i : G i = σ(φ i ([z i , R i ])) Where [·,·] represents channel-dimensional concatenation, and σ is the Sigmoid activation function. Lightweight mapping function φ i This includes convolution, batch normalization, and ReLU activation.
[0039] In step S3, the formula for residual multiplication modulation is: z i r = z i ⊙ (1 + αG i ), i∈{2,3,4} Where α is the scaling factor, and ⊙ represents element-wise multiplication; when the gating response is low, the residual path retains the original backbone features, and when the gating response is high, the semantic representation of the corresponding position is enhanced.
[0040] S4. Pre-decoding feature construction.
[0041] By constructing a pre-decoding feature module, the highest-level reliability enhancement feature z4 is... r Input the spatial attention module to obtain high-level localization features H; input the low-level features z1 sequentially into the oriented convolutional unit and the spatial attention module to obtain low-level structural features L; input the adjacent middle-level reliability enhancement features z2... r z3 r Input the cross-layer knowledge transfer module to obtain the middle-layer semantic structure feature M; H, M and L constitute the complementary input feature group of the front decoder.
[0042] The directional convolutional unit contains multiple convolutional kernels in different directions, used to extract multi-directional textures and edge responses; the cross-layer knowledge transfer module is used for information exchange between adjacent mid-layer features.
[0043] S5. Pre-decoding and coarse significance prediction.
[0044] The pre-decoder first upsamples H to the scale of M and injects global localization information through element-wise multiplication; then it upsamples the fused result to the scale of L and multiplies it element-wise with L to preserve the high-resolution structure; the fused results at each scale are further progressively integrated through channel concatenation and convolution to generate the decoding feature D. r ; For D r Perform a 1×1 convolution to obtain the coarse significance log probability plot S0.
[0045] S6. Foreground and background residual refinement.
[0046] The coarse significance log probability map S0 is transformed into a coarse significance probability map S using the foreground-background residual refinement module via the Sigmoid function. c ; Using S c and its complementarity Figure 1 -S c Decoding feature D r Element-wise weighting is performed separately to explicitly construct the foreground response feature D. f and background response features D b The foreground convolution branch and the background convolution branch process D separately. f and D b The two are concatenated along the channel dimension and input into a fusion convolution to obtain the residual log probability map R. The refined log probability map is S=S0+R.
[0047] like Figure 4 As shown, in step S6, the foreground response feature D f and background response features D b The formula for generating it is: S c = σ(S0) D f = D r ⊙ S c D b = D r ⊙ (1-S c ) Where σ is the Sigmoid activation function.
[0048] In step S6, the residual logarithmic probability map R is generated in the following way: F f =Conv f (D f ), F b =Conv b (D b ) R=Conv fuse ([F f ,F b S=S0+R Among them, Conv f For the foreground convolution branch, Conv b For the background convolution branch, Conv fuse To fuse convolutions.
[0049] In step S6, the foreground response feature D f Used to compensate for missed responses in the target region and repair the complete target structure; Background response feature D b Used to describe potential spurious regions and suppress various spurious responses; the logarithmic probability map R of the residuals is added back to the coarse logarithmic probability map S0 as a correction, without replacing the overall structure of the coarse significance prediction.
[0050] S7. Output.
[0051] The refined logarithmic probability map S is upsampled by a factor of 4 using bilinear upsampling to obtain the final logarithmic probability map restored to the spatial resolution of the input image. : Will The significance probability map is obtained by Sigmoid transformation. : Again Normalization and scaling are performed to obtain the final grayscale saliency map. To simultaneously constrain pixel-level foreground-background classification accuracy and region-level overlap consistency, the sum of binary cross-entropy log probability loss and cross-union ratio loss is used during training.
[0052] In step S7, the significance probability map is... Perform min-max normalization and scale to a grayscale range of 0 to 255 to obtain the final grayscale saliency map P. The formula for generating the final grayscale saliency map P is: Here, Normalize represents the min-max normalization operation.
[0053] S8. Loss function construction and model training.
[0054] During training, the sum of the binary cross-entropy log probability loss and the cross-union ratio loss is used as the overall saliency supervision objective, and this is applied only to the final log probability map. Supervision is performed without introducing boundary loss or multi-scale auxiliary supervision; the pre-trained semantic encoder is kept frozen, and only the parameters of the hierarchical feature encoder, gating mapping module, pre-decoding feature construction module, pre-decoder and foreground-background residual refinement module are updated.
[0055] In step S8, assume a training image contains N pixels, Ŝ i G is the saliency score (logit) of the i-th pixel after upsampling. i ∈{0,1} is the corresponding true significance label, Q i =σ(Ŝ i () represents the predicted probability obtained after the Sigmoid transformation. ε To prevent numerical stability constants with a denominator of zero, the formula for the overall significance monitoring objective is: in, The overall saliency monitoring objective; G This is the corresponding true saliency label; The binary cross-entropy log probability loss is applied directly to the saliency score map Ŝ to constrain pixel-by-pixel foreground-background classification. Its expanded form is as follows: In actual calculations, the logarithmic probability loss of the binary cross-entropy is calculated using a numerically stable form to avoid numerical overflow caused by directly taking the logarithm when the Sigmoid probability is close to 0 or 1. The specific formula is as follows: in, N The total number of pixels contained in a training image; Let be the saliency score of the i-th pixel after upsampling; This is the true saliency label for the i-th pixel, with values {0,1}.
[0056] Cross-Union Loss Applied to Sigmoid Probability Map By constraining the overall consistency of the target region through the soft cross-union ratio between the predicted region and the real region, its expansion is as follows: in, ε To prevent numerical stability constants with a denominator of zero; Let be the predicted significance probability of the i-th pixel. Example
[0057] This embodiment provides experimental comparison results using the method of the present invention on three public datasets to illustrate the beneficial effects of the present invention.
[0058] Tables 1 to 3 present the quantitative comparison results of the present invention (RaFBNet) and existing methods on the ORSSD, EORSSD, and ORSI-4199 datasets, respectively, for eight metrics, including S. α , , , , , , And MAE. The specific meanings of each indicator are as follows: S α The S-measure comprehensively measures the similarity between the predicted map and the ground truth in terms of object and region structure. The higher the value, the better. It is used to evaluate structure preservation and region integrity, but it cannot be used to determine the number of false background detections on its own.
[0059] MAE calculates the mean absolute pixel error between the predicted probability map and the true annotation. The lower the better, and it is used to evaluate the overall pixel error control. However, it includes both false positives (FP) and false negatives (FN), and cannot distinguish between false positives and false negatives.
[0060] The best weighted harmonic value of Precision and Recall among all thresholds reflects the best operating point performance, but only reflects the best threshold and does not represent threshold stability.
[0061] The result is the average of the F-measures under multiple thresholds, reflecting the average overall performance within the threshold range, but it still cannot directly locate the error area.
[0062] The F-measure, calculated under adaptive thresholding, reflects availability under automatic thresholding conditions, and its results are influenced by the adaptive thresholding rules.
[0063] , It comprehensively considers local pixel matching and global region statistical consistency to evaluate overall region alignment and enhance alignment quality, but it cannot replace S. α Or demonstrate background false detection suppression separately.
[0064] The E-measure, calculated under an adaptive threshold, reflects the overall detection quality of the model under data-driven threshold conditions. Its result is affected by the adaptive threshold calculation rules.
[0065]
[0066]
[0067]
[0068] S in Tables 1 to 3 αThe MAE is used to evaluate the similarity between the predicted salient region and the ground truth in terms of object structure and region structure. The higher the value, the better the target outline, connectivity and region integrity are preserved. The MAE is used to evaluate the average pixel-by-pixel error between the predicted probability map and the ground truth. The lower the value, the smaller the overall pixel-level prediction error.
[0069] Combination Figure 5 As shown, Figure 5 The PR curves and F-measure curves of the present invention and the comparison method on three datasets are presented. For the PR curve, the horizontal axis represents Recall and the vertical axis represents Precision. A set of Precision and Recall values is obtained by changing the binarization threshold of the significance probability plot; connecting these points forms the PR curve. For the F-measure curve, the horizontal axis represents the binarization threshold, and the vertical axis represents the F-measure value at the corresponding threshold. β In salient object detection, β² is typically set to 0.3 to place greater emphasis on precision in the evaluation. The F-measure comprehensively measures both precision and recall, while the E-measure evaluates prediction quality from two aspects: local pixel matching and overall region statistical consistency.
[0070] The PR curve reflects the trade-off between precision and recall at different thresholds. A higher curve for the same recall indicates higher precision, meaning fewer false positives (FPs) of background being misclassified as targets while preserving the same target proportion. A curve further to the right for the same precision indicates higher recall, meaning fewer false positives of the true target region are missed with similar background false positive levels. A curve closer to the upper right indicates a better overall trade-off between foreground preservation and background suppression. A sudden drop in recall at high thresholds indicates that lowering the threshold introduces more background response in order to recover nearly all target pixels, thus decreasing precision. Figure 5 As can be seen from the top row, the red curve corresponding to the RaFBNet of this invention is generally in the high position or leading curve group on the three datasets, indicating that it can maintain high precision over a wide range of recall. This phenomenon can be used as curve evidence for background false detection suppression.
[0071] The F-measure threshold curve reflects the overall detection performance under different binarization thresholds. A generally high and stable curve indicates that the model has better overall performance and threshold stability. In the F-measure threshold curve, a generally higher curve indicates that better Precision-Recall performance can be achieved under a wider range of thresholds; a wider high-order interval indicates a wider range of usable thresholds and insensitivity to threshold selection; and a relatively stable curve indicates that threshold changes do not cause drastic performance fluctuations, and the output probability has good threshold stability. Only corresponds to the optimal threshold point. This curve better reflects the average performance across the entire threshold range. However, it cannot directly indicate which background regions experienced false positives; therefore, it still needs to be combined with the PR curve and qualitative results. Figure 5 As can be seen from the bottom row, the red curve of the RaFBNet of this invention remains at a high position in most of the threshold range of the three datasets, indicating that the model has good overall detection performance and threshold stability.
[0072] The verification evidence for this invention in different dimensions of technical effect is as follows: Regarding the similarity of the target structure, with S α The primary evidence was supported by the E-measure and qualitative significance plots. α A higher MAE indicates that the predicted target is closer to the true annotation in terms of object-level and region-level structure, but it cannot alone prove a reduction in background false positives. Regarding pixel-level prediction accuracy, MAE is the primary evidence, supplemented by F-measure and E-measure. A lower MAE indicates a smaller average pixel-wise error between the predicted probability map and the true annotation, but MAE includes both false positives and false negatives and cannot distinguish whether the error originates from the foreground or background. For background false positive suppression, Precision under the same Recall is the primary evidence, supplemented by MAE, F-measure, and qualitative maps. A higher Precision usually indicates fewer false positives (FPs), but it should be combined with qualitative results for background areas such as roads, water bodies, and rooftops, and cannot be based solely on a single S-measure. α Or the MAE value. The effect of background false detection suppression should be explained by the combined results of a high Precision PR curve, a low MAE, and qualitative results of complex background areas, rather than attributing it to a single indicator.
[0073] ablation experiment Table 4 presents the ablation experiment results of the feature transfer reliability gating module and the foreground / background residual refinement module, which are used to verify the respective roles and synergistic effects of the two modules.
[0074]
[0075] As shown in Table 4, when only reliability gating is used, MAE increases on some datasets (e.g., from 0.0079 to 0.0095 on ORSSD), indicating that simply introducing unreliable features without subsequent correction mechanisms will have a negative impact. When only foreground and background residual refinement is used, the performance is steadily improved but not significantly. The complete model achieves the best performance under the synergistic effect of the two modules, with Sα and MAE significantly better than either module alone, proving that the two modules have a clear synergistic effect. Simultaneously employing the complete model with feature transfer reliability gating and foreground-background residual refinement modules, compared to the baseline model without these two modules, the S-measure (Sα) is improved from 0.9469 to 0.9496 and the MAE is reduced from 0.0079 to 0.0063 on ORSSD; the S-measure is improved from 0.9376 to 0.9439 and the MAE is reduced from 0.0064 to 0.0045 on EORSSD; and the S-measure is improved from 0.8862 to 0.8888 on ORSI-4199. This demonstrates the synergistic effect of the two modules.
[0076] Combination Figure 6 As shown, Figure 6 A visual ablation comparison between the feature transfer reliability gating module and the foreground / background residual refinement module is presented. Figure 6 In the diagram, the first column is the original input image (Image), the second column is the ground truth (GT) annotation, the third column is the baseline model with both reliability gating and FGBG removed (w / o Gate and FGBG), the fourth column is the model with only reliability gating retained (w / o FGBG), the fifth column is the model with only foreground and background residual refinement retained (w / o Gate), and the sixth column is the complete model RaFBNet (this invention) containing both modules. Figure 6 It can be seen that the complete model can reduce background residue and structural breaks when dealing with small targets, slender structures, low-contrast targets and complex textured background scenes, making the prediction results closer to the real target area.
[0077] Combination Figure 7 and Figure 8 As shown, Figure 7 and Figure 8 The saliency map qualitative comparison of the present invention and various existing methods is presented in complex backgrounds and elongated target scenes, as well as small target and multi-target scenes. Figure 7 and Figure 8The same column order is used: the first column is the original input image Image, the second column is the ground truth (GT) label, the third column is Ours / RaFBNet of this invention, and the fourth to eleventh columns are comparison methods such as GeleNet, TSCNet, ADSTNet, MCCNet, ACCoNet, HFANet, CorrNet, and SeaNet. Figure 7 and Figure 8 The methods GeleNet, TSCNet, ADSTNet, MCCNet, ACCoNet, HFANet, CorrNet, SeaNet, and Ours / RaFBNet listed in the left-hand columns of Tables 1 to 3 are all methods from the table. However, Tables 1 to 3 also include other quantitative comparison models, therefore... Figure 7 and Figure 8 Only a portion of representative methods are shown; the two lists are not entirely identical in number. Figure 7 and Figure 8 As can be seen, when dealing with complex backgrounds, slender targets, small targets, and multi-target scenes, the present invention provides more complete prediction results, clearer boundaries, and less background noise, verifying the superiority of the present invention in complex scenes.
[0078] The pre-trained semantic encoder remains frozen during training, serving only as an external semantic prior provider; the residual correction is added back to the coarse saliency log probability map without altering the basic output structure of the pre-decoder. Therefore, this invention facilitates integration into existing two-stage salient object detection frameworks and avoids modifying the pre-trained semantic encoder parameters during the saliency training process.
Claims
1. A salient target detection method for optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement, characterized in that: Includes the following steps: S1. Hierarchical backbone feature extraction: The input optical remote sensing image is fed into a hierarchical feature encoder to obtain backbone features x1, x2, x3 and x4 at four scales. Each scale feature is mapped to a unified channel dimension through convolution and batch normalization to obtain low-level feature z1, mid-level features z2 and z3 and high-level feature z4. Among them, low-level feature z1 preserves the boundary, texture and elongated structure, mid-level features z2 and z3 take into account both spatial structure and semantic response, and high-level feature z4 is used for global semantic localization. S2. Frozen Dense Semantic Prior Feature Extraction and Alignment: A pre-trained semantic encoder is set up in parallel with the hierarchical feature encoder. The input image is inversely normalized and then fed into the frozen pre-trained semantic encoder, outputting dense semantic prior features T. For mid-to-high-level feature scales, T is bilinearly interpolated to the spatial size of the corresponding backbone features, and channel projection is completed using 1×1 convolution to obtain the feature transfer reliability cue R. i ; S3. Feature Transfer Reliability Gating: For each mid-to-high-level scale, the backbone feature z is gated through a gating mapping module. i With the corresponding feature transfer reliability clue R i The concatenation is performed along the channel dimension using a lightweight mapping function φ, which includes convolution, normalization, and ReLU activation. i And the Sigmoid activation function, to obtain the pixel-level reliability-gated map G. i The reliability enhancement feature z is obtained by modulating the original backbone features using residual multiplication. i r Furthermore, the lower-level feature z1 is not subjected to gating processing to maintain the original transmission path; S4. Pre-decoding Feature Construction: The highest-level reliability enhancement feature z4 is constructed using the pre-decoding feature construction module. r Input the spatial attention module to obtain high-level localization features H; input the low-level features z1 sequentially into the oriented convolutional unit and the spatial attention module to obtain low-level structural features L; input the adjacent middle-level reliability enhancement features z2... r z3 r Input the cross-layer knowledge transfer module to obtain the mid-layer semantic structure feature M; H, M and L constitute the complementary input feature group of the pre-decoder; S5. Pre-decoding and coarse saliency prediction: The pre-decoder first upsamples H to the scale of M and injects global localization information through element-wise multiplication; then it upsamples the fusion result to the scale of L and multiplies it element-wise with L to preserve the high-resolution structure; the fusion results at each scale are further progressively integrated through channel concatenation and convolution to generate the decoding feature D. r ; For D r Perform a 1×1 convolution to obtain a coarse significance log probability plot S0; S6. Foreground-Background Residual Refinement: The foreground-background residual refinement module transforms the coarse significance log probability map S0 into a coarse significance probability map S using the Sigmoid function. c ; Using S c And its complementary figure 1-S c Decoding feature D r Element-wise weighting is performed separately to explicitly construct the foreground response feature D. f and background response features D b The foreground convolution branch and the background convolution branch process D separately. f and D b The two are concatenated along the channel dimension and input into the fusion convolution to obtain the residual log probability map R. The refined log probability map is S=S0+R. S7. Output: Perform a 4-fold bilinear upsampling on the thinned logarithmic probability map S to obtain the final logarithmic probability map restored to the spatial resolution of the input image. ,Will The significance probability map is obtained by Sigmoid transformation. And then Normalization and scaling are performed to obtain the final grayscale saliency map; S8. Loss Function Construction and Model Training: During training, the sum of the binary cross-entropy log probability loss and the cross-union ratio loss is used as the overall significance supervision target, and only for the final log probability map. Supervision is performed without introducing boundary loss or multi-scale auxiliary supervision; the pre-trained semantic encoder is kept frozen, and only the parameters of the hierarchical feature encoder, gating mapping module, pre-decoding feature construction module, pre-decoder and foreground-background residual refinement module are updated.
2. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement as described in claim 1, characterized in that: In step S2, the pre-trained semantic encoder uses a frozen RADIO model; the specific formula for the inverse normalization process is: I R = clamp(I norm ×std+mean, 0, 1) Among them, I norm For the input image normalized according to the backbone encoder requirements, mean and std are the corresponding mean and standard deviation, respectively. R The parameters of the pre-trained semantic encoder are not included in gradient updates during training and are used as input to the semantic encoder of the RADIO model.
3. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement as described in claim 1, characterized in that: In step S3, the formula for residual multiplication modulation is: With i r = with i ⊙ (1 + αG i ), and∈{2,3,4} Where α is the scaling factor, and ⊙ represents element-wise multiplication; when the gating response is low, the residual path retains the original backbone features, and when the gating response is high, the semantic representation of the corresponding position is enhanced.
4. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement as described in claim 1, characterized in that: In step S6, the foreground response feature D f Used to compensate for missed detection responses in the target region and repair the complete target structure; the background response feature D b Used to describe potential spurious regions and suppress various spurious responses; the residual log probability map R is added back to the coarse significance log probability map S0 as a correction amount, without replacing the overall structure of the coarse significance prediction.
5. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement as described in claim 1, characterized in that: In step S6, the foreground response feature D f and background response features D b The formula for generating it is: S c = σ(S0) D f = D r ⊙ S c , D b = D r ⊙ (1-S c ) Where σ is the Sigmoid activation function.
6. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement according to claim 5, characterized in that: In step S7, the formula for generating the final grayscale saliency map P is: Here, Normalize represents the min-max normalization operation.
7. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement according to claim 6, characterized in that: In step S8, the formula for the overall saliency supervision target is: in, The overall saliency monitoring objective; The log-probability loss is the binary cross-entropy. G This is the corresponding true saliency label.
8. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement according to claim 7, characterized in that: In step S8, the binary cross-entropy logarithmic probability loss is calculated using a numerically stable form to avoid numerical overflow caused by directly taking the logarithm when the Sigmoid probability is close to 0 or 1. The specific formula is as follows: in, N The total number of pixels contained in a training image; Let be the saliency score of the i-th pixel after upsampling; This is the true saliency label for the i-th pixel, with values {0,1}.
9. The method for salient target detection in optical remote sensing images based on feature transfer reliability gating and foreground / background residual refinement as described in claim 8, characterized in that: In step S8, the expansion of the cross-union ratio loss is: in, ε To prevent numerical stability constants with a denominator of zero; Let be the predicted significance probability of the i-th pixel.
