A high-fidelity salient object detection method and system based on input-level scale refinement

Abstract

The application discloses a high-fidelity saliency target detection method and system based on input level scale refining, which generates a multi-scale multi-view input stream of normal view, overview view and gaze view through a learnable scale stem block constructed at the input end, so as to overcome the loss of detail information caused by traditional down-sampling from the source; high-level semantics and multi-scale detail features are respectively extracted by using semantic stream and detail stream heterogeneous coding, and effective information of different views is dynamically integrated by a cross-scale attention fusion module in a self-attention mechanism, so as to effectively bridge the semantic gap between scales; in the decoding stage, a context-structure collaborative decoder composed of contrastive channel attention and boundary space attention is designed, foreground and background are explicitly decoupled, and the blurred edges and transition areas of the optimization target are focused, a mixed loss function containing uncertainty perception loss is used for deep supervision, and the structural integrity and edge fidelity of the saliency mask are significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a high-fidelity saliency target detection method and system based on input-level scale refinement. Background Technology

[0002] Image salient object detection aims to simulate the human visual system, accurately extracting and segmenting the most visually attractive target regions from complex scenes. With the development of deep learning technology, the technological evolution in this field mainly revolves around the following three core directions: Firstly, multi-backbone network fusion: leveraging the advantages of heterogeneous architectures such as CNNs and Transformers to enhance feature representation. Examples include HRNet, which uses parallel branches to maintain high-resolution features, and DSRNet, which integrates global and local information using a cross-fusion mechanism.

[0003] Secondly, edge information enhancement: improving boundary prediction accuracy by introducing edge supervision or specific structural modules. Representative works include VST, which uses multi-task learning to jointly predict saliency masks and edge maps; PoolNet, which introduces auxiliary edge loss; and DSRNet, which uses morphological operations to refine edge textures.

[0004] Third, multi-scale feature fusion: dedicated to integrating semantic and detailed information at different levels. For example, M3Net uses high-level semantics to guide low-level features, ICON fuses adjacent scales through feature concatenation, and TCGNet introduces dilated convolution and cross-scale interaction mechanisms.

[0005] However, despite the significant progress made by the aforementioned methods in feature enhancement and fusion, a key technical bottleneck remains when dealing with salient targets that are rich in detail, have varying scales, or are structurally complex: the loss of downsampling information in the initial stage of the backbone network. Existing deep learning models typically use a single image with a fixed resolution as input, and the large step-downsampling operation at the beginning of their backbone network causes irreversible loss of texture information.

[0006] Specifically, the technical bottleneck mainly stems from the following three points: (1) Loss of source information: The tiny details and high-frequency textures of the image are lost in large quantities when entering the shallow layer of the network, which severely limits the upper limit of the quality of subsequent feature extraction; (2) Multi-scale semantic gap: There is a lack of effective mechanisms to dynamically bridge the feature differences between different input scales, which makes it difficult to deeply align the global context with local details; (3) Blurred boundaries in the decoding stage: In the process of restoring spatial resolution, there is a lack of explicit semantic decoupling and boundary guidance, which leads to noise interference or structural collapse of the target edge in the transition area.

[0007] Therefore, how to construct a high-fidelity information flow at the source of feature extraction and achieve end-to-end detail repair and collaborative optimization from the input level to the decoding level is the key challenge for improving the performance of salient object detection. Summary of the Invention

[0008] To address the problem of irreversible loss of high-frequency texture and detail information caused by initial downsampling of the backbone network in existing technologies, this invention provides a high-fidelity salient object detection method and system based on input-level scale refinement. Inspired by the "gaze-glance" collaboration mechanism in the human visual system, this method effectively overcomes the bottleneck of detail feature loss through end-to-end optimization of multi-view input generation, heterogeneous dual-stream encoding, cross-scale feature fusion, and collaborative decoding.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a high-fidelity saliency target detection method based on input-level scale refinement, comprising the following steps: Step 1: Input the input image into a learnable scale stem block and generate three multi-view input images containing different spatial frequency information through parallel processing paths. The multi-view input images include a normal view image that maintains the original resolution, an overview view image generated by downsampling, and a gaze view image generated by upsampling and residual repair mechanism. Step 2: Encode the multi-view input images using a semantic encoding path and a detail encoding path respectively. The semantic encoding path uses a pre-trained backbone network P2T to extract high-level semantic features and generates channel-uniform semantic features through a dimension reduction module. The detail encoding path uses a set of weight-shared convolutional neural network encoders to process the normal view image, overview view image, and gaze view image respectively, and uses the local inductive bias of the convolutional neural network to capture edge details and generate multi-scale detail feature maps. Step 3: Construct a cross-scale attention fusion module at the end of the encoding, align the multi-scale detail feature maps in the spatial dimension, use the self-attention mechanism to calculate spatial correlation to mine global spatial context dependency, dynamically weight and fuse information from different perspectives, and generate fused features and initial saliency mask. Step 4: Construct a decoding network consisting of multiple cascaded context-structure co-decoders; each context-structure co-decoder receives the current level encoder jump connection features from the semantic coding path, the output features of the previous level decoder, and the saliency mask predicted by the previous level. It uses a two-stream co-operation mechanism composed of contrast channel attention and boundary space attention to decouple the foreground and background features, and guides the network to focus on the blurred edges and transition regions of the target to output a high-fidelity saliency mask.

[0010] In one embodiment of the present invention, the learnable scale stem block in step 1 generates the multi-view input image in the following manner: The normal view image is the input image itself; The overview view image is obtained by downsampling the input image using a depthwise separable convolution with a stride of 2, followed by batch normalization and activation function processing. The gaze view image is obtained by first upsampling the input image using bilinear interpolation to obtain a coarse high-resolution image, then processing the coarse high-resolution image using a lightweight residual repair module to obtain residual information, adding the coarse high-resolution image and the residual information and processing them with an activation function, wherein the lightweight residual repair module includes a depthwise separable convolution with a stride of 1 and a batch normalization layer.

[0011] In one embodiment of the present invention, the processing procedure of the cross-scale attention fusion module in step 3 includes: The multi-scale detail feature maps are upsampled to the same resolution to obtain aligned feature map groups. The aligned feature map group is averaged to obtain an average feature map as the baseline feature. The baseline features are projected into queries, keys, and values ​​using parallel convolutional layers, and a spatial correlation matrix is ​​calculated by scaling dot product attention to generate weighted features. The weighted features are residually connected to the baseline features to generate the fused features, wherein the residual connection introduces a learnable scaling factor with an initial value of 0; The initial saliency mask is generated by mapping the fused features through convolutional layers.

[0012] In one embodiment of the present invention, the dual-stream cooperative mechanism in step 4 includes: The contrast channel attention branch is used to generate contrast vectors between the foreground and the background based on the saliency mask predicted by the previous level. The contrast relationship is learned through a multilayer perceptron to generate channel attention weights and recalibrate the input features to suppress the background channel response. The boundary spatial attention branch is used to generate a boundary guidance map based on the saliency mask predicted by the previous level, spatially weight the dimensionality-reduced query features, and generate a spatial attention map through depthwise separable convolution to obtain structurally refined features. The features processed by the contrast channel attention branch and the boundary space attention branch are then subjected to residual fusion.

[0013] In one embodiment of the present invention, the processing procedure of the contrast channel attention branch includes: The saliency mask of the previous level prediction is binarized based on a preset threshold to obtain the foreground mask and the background mask; The foreground and background masks are used to perform weighted average pooling on the input features to obtain the foreground vector and background vector, respectively. After concatenating the foreground and background vectors along the channel dimension, channel attention weights are generated using a multilayer perceptron and activation function. Multiply the channel attention weights by the input features to obtain the channel-recalibrated features.

[0014] In one embodiment of the present invention, the processing procedure of the boundary space attention branch includes: Based on the saliency mask M predicted at the previous level, using the formula... The boundary guidance map G is calculated; The boundary guidance graph is used to spatially weight the dimensionality-reduced query features to obtain guidance features; The guiding features are passed through a depthwise separable convolutional module to generate a spatial attention map; The spatial attention map is combined with the input features through residual connections to obtain the structurally refined features, wherein the residual connections introduce a learnable scaling parameter with an initial value of 0.

[0015] In one embodiment of the present invention, the method further includes step 5: supervising the training of the model using a hybrid loss function comprising weighted intersection-union loss, weighted binary cross-entropy loss, and uncertainty-aware loss; the uncertainty-aware loss is used to emphasize uncertain regions where the prediction probability is close to 0.5, and its loss value decreases as the prediction probability deviates from 0.5, and the balance coefficient of the uncertainty-aware loss is dynamically adjusted using a cosine strategy; the method further includes: using the hybrid loss function to calculate the loss of the initial saliency mask generated in step 3 for supervising training.

[0016] Secondly, the present invention provides a high-fidelity saliency target detection system based on input-level scale refinement, for implementing the method described above, the system comprising: A multi-view input generation module is used to process an input image into three multi-view input images containing different spatial frequency information. The multi-view input images include a normal view image, an overview view image, and a gaze view image. The heterogeneous dual-stream coding module is used to extract semantic features and multi-scale detail feature maps through semantic coding path and detail coding path, respectively; A cross-scale feature alignment and fusion module is used to align and fuse the multi-scale detail feature maps to generate fused features and an initial saliency mask; The context-structure co-decoding module is used to co-decode features to output a high-fidelity saliency mask.

[0017] Compared with the prior art, the present invention has at least the following beneficial effects: (1) By introducing learnable scale stem blocks, parallel multi-view paths are constructed in the input stage to generate normal view, overview view and gaze view images containing different spatial frequency information. In particular, the residual repair mechanism is used to adaptively restore high-frequency texture details under magnified view. This effectively solves the problem of irreversible detail loss caused by the initial downsampling of traditional backbone networks from the source, so that the network has high-fidelity information expression ability in the early stage of feature extraction, laying a high-quality input foundation for subsequent detection tasks.

[0018] (2) A cross-scale attention fusion module is adopted to align the detailed features of multiple perspectives and different scales in the spatial dimension. Then, the cross-perspective spatial correlation is dynamically calculated and weighted fusion is performed by using the benchmark construction based on average features and the self-attention mechanism. At the same time, a zero-initialization learnable scaling factor is introduced to achieve a smooth transition from common overview to detail enhancement, which effectively bridges the semantic gap between different input scales and significantly improves the deep fusion effect of global context and local details.

[0019] (3) Construct a context-structure collaborative decoder and use parallel contrast channel attention and boundary space attention to form a dual-stream collaborative mechanism. The contrast channel attention generates channel weights based on the contrast vectors of the foreground and background, explicitly decouples the foreground and background and suppresses background interference. The boundary space attention performs spatial weighting on the blurred transition region through the boundary guidance map, and accurately sharpens the target edge. The collaborative work of the two enables the decoding process to focus on the real structure and boundary region of the target, which greatly improves the structural integrity and boundary fidelity of the saliency mask and overcomes the problems of edge blurring and structural collapse caused by conventional upsampling.

[0020] (4) A hybrid loss function incorporating uncertainty-aware loss is introduced into the training supervision. This loss function adaptively emphasizes ambiguous regions with prediction probabilities close to 0.5. Combined with weighted intersection-union loss and weighted binary cross-entropy loss, as well as a deep supervision strategy targeting the intermediate initial saliency mask, the network continuously focuses on difficult samples and high-frequency details during training, further improving the high fidelity and generalization ability of saliency detection. Experimental results show that, based on the above comprehensive technical means, this invention outperforms existing state-of-the-art methods in key indicators such as structural similarity, weighted F-value, mean absolute error, and maximum E-value on multiple public benchmark datasets. In particular, it demonstrates significant advantages in large-scale scene integrity, fine structure preservation, and resistance to interference from complex backgrounds. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is the overall architecture diagram of the network of this invention.

[0023] Figure 2 This is a structural diagram of the learnable scale stem tuber (LSS) in this invention.

[0024] Figure 3 This is a diagram of the Cross-Scale Attention (CSA) module in this invention.

[0025] Figure 4 This is a structural diagram of the Context-Structure Collaborative Decoder (CSCD) in this invention.

[0026] Figure 5 This is a diagram of the Contrast Channel Attention (CCA) module in this invention.

[0027] Figure 6 This is a diagram of the Boundary Space Attention (BSA) module in this invention.

[0028] Figure 7 This is a visual comparison diagram of the present invention and the baseline method. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited to the following embodiments. For those skilled in the art, equivalent substitutions, combinations or conventional modifications made without departing from the concept of the present invention should all fall within the scope of protection of the present invention.

[0030] In this invention, the image data output by each module or decoder that represents the probability that each pixel in the input image belongs to a salient target is collectively referred to as a saliency mask; for example, the output of the cross-scale attention fusion module is called the initial saliency mask, and the final output of the context-structure co-decoder is called the high-fidelity saliency mask.

[0031] Example 1 This embodiment provides a high-fidelity saliency target detection method based on input-level scale refinement, including: Multi-view input generation: The input image is fed into a learnable scale stem block (LSS), abandoning the fixed single resolution input and constructing a parallel processing path; The processing path generates three high-quality multi-view input streams containing different spatial frequency information. The multi-view input streams include: a normal viewpoint as a reference. Preserving the overall context overview view And the gaze angle that adaptively restores texture details through residual repair mechanisms. ; Edge-guided modulation cooperative heterogeneous dual-stream coding: A heterogeneous dual-stream mechanism is constructed to encode the multi-view input stream, including semantic stream and detail stream; The semantic stream uses a pre-trained backbone network P2T to extract high-level semantic features at four levels. And through the dimensionality reduction module, channel-uniform semantic features are generated. Provide macro-level guidance; The detail flow utilizes a weight-sharing convolutional neural network (CNN) encoder group to process inputs from different viewpoints. By leveraging the local inductive bias of CNNs to capture edge details, multi-scale detail feature maps are generated. , , ; Cross-scale feature alignment and fusion: A cross-scale attention (CSA) fusion module is built at the end of the encoding to receive multi-scale feature maps from the detail stream; Align features from different perspectives in the spatial dimension and use a self-attention mechanism to calculate spatial correlations to deeply mine global spatial context dependencies; By dynamically weighting and fusing effective information from different perspectives, the semantic gap caused by scale differences is eliminated, ultimately generating detailed fused features. and the initial saliency mask ; Context-structure co-decoding: Construct a decoding network consisting of multiple cascaded context-structure cooperative decoders (CSCD); Each CSCD module receives the encoder skip connection features of the current level. Output features of the previous level decoder and the saliency mask of the previous level prediction Furthermore, a two-stream collaborative mechanism is introduced, consisting of contrast channel attention (CCA) and boundary space attention (BSA). The dual-stream collaborative mechanism is used to explicitly decouple foreground and background features, and guide the network to focus on and optimize the blurred edges and transition regions of the target. The refined output is a high-fidelity saliency mask with the same size as the original image. Model supervision: The model is trained under supervision using a hybrid loss function that includes weighted intersection-union loss, weighted binary cross-entropy loss, and uncertainty-aware loss (UAL), and the context-structure co-decoder of the final module is supervised.

[0032] Optionally, the specific generation mechanism of the learnable scale stem-stub (LSS) is as follows: Normal view generation Keeping the resolution of the original input image unchanged, it is used as a baseline reference. ; Overview view generation Downsampling is performed using grouped convolution, specifically utilizing stride. An overview graph is generated using a depthwise separable convolution (DWConv) of size 2, combined with batch normalization (BN) and ReLU activation function. While reducing resolution, the physical meaning and global context of the original channels are preserved:

[0033] in, It is the ReLU activation function; Gaze-based viewpoint generation: Employing an "upsampling-residual restoration" mechanism, the image is first physically enlarged using bilinear interpolation to obtain a coarse, high-resolution image. ; Subsequently, a lightweight residual repair convolution module is introduced to learn the high-frequency residual information lost during interpolation. This module consists of a stride... =1 It consists of depthwise separable convolutions and batch normalization (BN) layers. The final gaze-oriented input... Generated via residual joins and terminal activation:

[0034] Optionally, the cross-scale attention (CSA) fusion module is specifically implemented as follows: Feature Alignment and Benchmark Construction: Using Bilinear Interpolation and Upsampling to At the same resolution, the aligned feature sequences are obtained. Then, Input a cross-scale attention module; Attention-weighted: To construct a baseline feature that contains common information from all perspectives, the aligned features are concatenated along the channel dimension and the average feature is calculated. :

[0035] in The number of viewpoints. Next, to compute cross-viewpoint spatial attention, three parallel... The projection of the convolutional layer is: query Q, key K, value V.

[0036] Using a scaled dot product attention mechanism, the spatial correlation matrix is ​​calculated and weighted features are generated. :

[0037] in, The channel dimension of the key vector; Residual smooth transition: To ensure training stability and achieve a smooth transition of features, a learnable scaling factor initialized to 0 is introduced. The final fusion features This is generated through residual connections, and the module also outputs an initial saliency mask. Used to assist in supervision:

[0038] Optionally, the Context-Structure Co-decoder (CSCD) refines the encoder features through the following two parallel branches: Contrastive Channel Attention (CCA) branch: The masks are aligned and binarized into foreground and background masks using bilinear interpolation. The weighted global average pooling vectors of the two regions are calculated separately and concatenated along the channel dimension. The contrast relationship is learned through a multilayer perceptron to generate channel attention weights. These weights are used to recalibrate the input features and suppress the background channel response. Boundary Spatial Attention (BSA) branch: Based on the mask, calculate the boundary guidance map that has the largest response in the fuzzy transition region with a predicted value close to 0.5; use the guidance map to spatially weight the dimensionality-reduced query features; then feed it into the depthwise separable convolution module to expand the receptive field to generate a spatial attention map, and generate structurally refined features through residual connections with learnable scaling parameters; Fusion: Residual fusion is performed between the encoder features refined by CCA and BSA and the main decoding path features.

[0039] Example 2 This embodiment provides a high-fidelity salient target detection method, with a system architecture (ISRNet) as follows: Figure 1 As shown, the specific implementation process includes the following steps: Step 1: Generation of multi-view inputs based on learnable scale stems (LSS); Input image IThe input is fed into a Learnable Scale Stem Block (LSS), abandoning the traditional single-resolution input and generating three multi-view input streams containing different spatial frequency information through three parallel processing paths. (Reference) Figure 2 The three input streams are normal view images. Overview view image and gaze view images The specific generation method is as follows: Normal view generation ( Keep the original input resolution unchanged and use it as a baseline reference: .

[0040] Overview perspective generation ( ): Using a step size of 2 Depthwise separable convolution (DWConv), combined with batch normalized normalization (BN) and ReLU activation functions, generates an overview graph. Maintaining color channel independence and global context while reducing resolution:

[0041] in, Indicates the convolution stride. This represents the ReLU activation function. This design effectively reduces the number of parameters while preserving the physical meaning and global context of the original channels to the greatest extent possible.

[0042] Gaze generation ( The method employs an "upsampling-residual repair" mechanism. First, bilinear interpolation is used to physically enlarge the image, resulting in a coarse, high-resolution image. :

[0043] Subsequently, a lightweight residual repair convolution module is introduced to learn the high-frequency residual information lost during interpolation. This module consists of a stride... =1 It consists of depthwise separable convolutions and batch normalization (BN) layers. The final gaze-oriented input... Generated via residual joins and terminal activation:

[0044] Step 2: Heterogeneous dual-stream feature encoding; A heterogeneous dual-stream mechanism that combines semantic stream and detail stream is constructed to encode the multi-view input from step 1.

[0045] Semantic Stream Coding: A pre-trained P2T network is used as the backbone network to extract high-level semantic features at four levels. Channel-unified semantic features are generated through a dimensionality reduction module to provide macro-level guidance for saliency localization.

[0046] Detail flow encoding: The three view inputs generated in step 1 are fed into a CNN encoder group with shared weights for processing. The local inductive bias of the CNN is used to capture edge details and generate multi-scale detail feature maps.

[0047] The weight-sharing convolutional neural network encoder group consists of four sequentially connected convolutional stages. Each convolutional stage includes a 3×3 convolutional layer with a stride of 2 for downsampling, and two 3×3 convolutional layers with a stride of 1 for feature extraction. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The normal view image, overview view image, and gaze view image are independently input into this encoder group. For each input, the outputs of the four convolutional stages are taken as four detail feature maps at different scales. Thus, a total of twelve multi-scale detail feature maps are obtained.

[0048] Step 3: Feature alignment and fusion based on cross-scale attention (CSA); Set up a cross-scale attention fusion (CSA) module at the end of the encoding, see [link / reference]. Figure 3 Multi-scale features of the detail flow are fused to generate fused features rich in detail. and initial saliency mask The specific processing procedure is as follows: Spatial Alignment and Benchmark Feature Construction: Using Bilinear Interpolation and Upsampling to At the same resolution, the aligned feature sequences are obtained. Subsequently, Input a cross-scale attention module. To construct a baseline feature that contains common information from all viewpoints, the aligned features are concatenated along the channel dimension and the average feature is calculated. :

[0049] in The number of viewpoints. Next, to compute cross-viewpoint spatial attention, three parallel... The projection of the convolutional layer is: query Q, key K, value V.

[0050] Self-attention weighted: This method uses three parallel convolutional layers to project sequence features into query, key, and value tensors. A scaled dot product attention mechanism is then used to compute the spatial correlation matrix and generate weighted features. :

[0051] in, The channel dimension of the key vector; Smooth Transition and Fusion: To ensure training stability and achieve smooth feature transition, a learnable scaling factor initialized to 0 is introduced. The final fusion features This is generated through residual connections, and the module also outputs an initial saliency mask. Used to assist in supervision:

[0052] This zero-initialization residual design allows the network to directly utilize stable average features during the early stages of training. As training progresses, By gradually adjusting and adaptively injecting refined details captured by the attention mechanism, a progressive optimization from "common overview" to "detail enhancement" is achieved.

[0053] Generate fusion feature F in the cross-scale attention fusion module cnn and the initial saliency mask F pred Then, the semantic features output from the highest layer of the P2T backbone network in the semantic stream (after being processed by the dimensionality reduction module) are combined with the fused feature F. cnn The features are concatenated along the channel dimension and then fused through a 1×1 convolutional layer to generate the initial input features Fdec_init of the decoder with a uniform number of channels.

[0054] Step 4: Feature refinement based on context-structured co-decoder (CSCD); The decoding stage consists of four cascaded CSCD modules. For the first CSCD, its input "output feature of the previous level decoder" is the initial input feature Fdec_init of the decoder; for subsequent CSCDs, its input "output feature of the previous level decoder" is the decoding feature output by the previous level CSCD.

[0055] Encoder skip connection features of the current level of each CSCD module Output features of the previous level decoder and the saliency mask of the previous level prediction ,refer to Figure 4 Unlike traditional direct addition or splicing, CSCD utilizes the previous level's... As a spatial prior for the display, the encoder features are processed through parallel contrastive attention (CCA) and boundary spatial attention (BSA) branches. Targeted cleaning and refinement are performed, followed by comparison with the main decoding path features. Perform residual fusion. The dual-stream collaborative mechanism includes: Compare the Channel Attention (CCA) branch, refer to Figure 5 Given input features and mask First, bilinear interpolation is used to align the mask, and then it is binarized into a foreground mask. and background mask Then, the weighted global average pooling vectors for the foreground and background regions are calculated separately:

[0056] in To prevent division by zero by a small constant, the foreground vector is then... With background vector The channels are concatenated along the channel dimension, and the contrast relationship between them is learned through a shared multilayer perceptron to generate channel attention weights. :

[0057] Finally, the input features are recalibrated using these weights to obtain semantically enhanced features. This process effectively suppresses the response of the background channel, highlighting the foreground semantics.

[0058] Boundary Space Attention (BSA) branch, see reference Figure 6 Due to the effects of the upsampling process, the edges of salient targets often suffer from blurring. To accurately capture and recover these geometric details, the BSA module focuses on optimizing blurred edge regions in mask prediction. Firstly, based on the mask… Calculate the boundary guidance graph The graph shows the largest response in the fuzzy transition region where the predicted value is close to 0.5.

[0059] Next, the guided graph is used to refine the query features after convolutional dimensionality reduction. Spatial weighting forces the network to focus on high-frequency boundary regions:

[0060] To capture local edge details while maintaining contextual coherence, weighted features are fed into a DWConv module to expand the receptive field and generate a spatial attention map. The final structural refinement features Generated via residual join:

[0061] in These are learnable scaling parameters initialized to 0. This design allows the network to adaptively enhance the feature representation of blurred boundaries without destroying the internal texture information.

[0062] Residual fusion: The features cleaned and refined by CCA and BSA are residually fused with the features of the main decoding path. After four levels of refinement, the final high-fidelity saliency mask is output.

[0063] Step 5: Model supervision with hybrid loss functions; A hybrid loss function is used for end-to-end training supervision of the network: Uncertainty Perception Loss: This loss is introduced to emphasize areas of uncertainty and ambiguity in the prediction results, such as pixels with a prediction probability close to 0.5. A cosine strategy is used to set its balance coefficient, guiding the network to better identify high-frequency detailed information. The formula is as follows:

[0064] in, These are pixel values ​​in the image. The balance coefficients are set using a cosine strategy. :

[0065] in, t This is the current training round number. T This represents the total number of rounds.

[0066] Comprehensive Supervision: A total loss function is constructed by combining the weighted intersection-union (wIoU) loss and the weighted binary cross-entropy (wBCE) loss. This not only supervises the final prediction results but also applies the same supervision strategy to the coarse saliency mask of the CNN branch outputs, achieving deep supervision, accelerating model convergence, and improving the final detection fidelity. The formula is as follows: .

[0067] The effects of this invention can be further illustrated by the following experiments.

[0068] To verify the beneficial effects of the method proposed in this invention, extensive experiments were conducted. The invention was comprehensively compared and analyzed with current mainstream methods from multiple dimensions, including quantitative indicators, qualitative visual effects, model efficiency, and ablation experiments.

[0069] 1. Experimental setup The DUTS-TR public dataset was used as the training set. The model was then tested and evaluated on four widely used public benchmark datasets: DUTS-TE, DUT-OMRON, ECSSD, and HKU-IS. These datasets cover a variety of complex scenarios and can comprehensively evaluate the model's generalization ability.

[0070] Four widely accepted evaluation metrics in the field of salient object detection are used: S-measure (S) evaluates the structural similarity between the predicted results and the ground truth labels; Weighted F-measure (F) comprehensively considers accuracy and recall; Mean Absolute Error (M) measures the average absolute value of the prediction error; and Maximum E-measure (E) evaluates model performance from different perspectives. "↑" indicates higher performance, "↓" indicates lower performance, the bottom right corner represents the year of the model, bold indicates best performance, and underline indicates second-best performance.

[0071] This experiment was implemented using the PyTorch deep learning framework and trained and tested on a single NVIDIA RTX 4090 GPU. During the training phase, to adapt to the model input requirements and enhance data diversity, all input images were uniformly adjusted to a resolution of 352×352. The backbone network used was a P2T model pre-trained on ImageNet-1K to extract multi-scale feature representations.

[0072] The network was trained using a stochastic gradient descent optimizer with a momentum setting of 0.9 and a weight decay setting of 5 × 10⁻⁶. -4 The initial learning rate is set to 1e. -4 The learning rate is dynamically adjusted using a cosine annealing strategy, with a minimum learning rate set to 1e. -5 ,cycle The threshold is set to 20. To prevent gradient explosion, the gradient clipping threshold is set to 0.5. The training batch size is set to 2, and the total number of training epochs is 50. Furthermore, a weighted combination of structural loss and uncertainty-aware loss is used as the total loss function, where the weights of UALLoss are dynamically adjusted as training progresses. All comparative experiments are conducted under this uniform configuration to ensure fairness in the comparison.

[0073] 2. Quantitative analysis To objectively measure the performance of the proposed ISRNet model, a detailed quantitative comparison was conducted with 12 mainstream methods on four standard datasets. These models include: VST, DPNet, DSRNet, ICON, PGNet, SF, MENet, BBRF, SelfReformer, TCGNet, SwinSod, and SodDCNet. Evaluation metrics include S, F, M, and E. Tables 1 and 2 show the detailed scores of each model on different datasets, with the best results highlighted in bold.

[0074] Table 1 Comparison experiments on DUTS-TE and DUT-O datasets

[0075] To verify the effectiveness of ISRNet based on input-level scale refinement, this invention quantitatively compares it with 12 cutting-edge algorithms on the DUTS-TE and DUT-O datasets.

[0076] Specifically, ISRNet achieved S-measure and F-measure scores of 0.924 and 0.895 respectively on the DUTS-TE dataset, significantly outperforming the recent best-performing benchmark models. Notably, the MAE metric, reflecting pixel-level absolute error, reached 0.022 and 0.041 on both datasets, achieving the best performance across the entire dataset. This demonstrates that ISRNet's strategy of reconstructing multi-scale feature streams from the input effectively suppresses interference from complex backgrounds and preserves high-frequency detail information to the greatest extent possible.

[0077] Table 2 Comparison experiments on ECSSD and HKU-IS datasets

[0078] As shown in Table 2, the quantitative results demonstrate that ISRNet exhibits good generalization performance on the ECSSD and HKU-IS datasets. On the ECSSD dataset, which contains a large number of complex semantics and multi-objective scenes, ISRNet outperforms recent state-of-the-art models such as SwinSod and SODDCNet in all evaluation metrics, achieving the best performance. On the HKU-IS dataset, ISRNet also achieves the best or is tied for the best in the three key metrics of S-measure, MAE, and E-measure. This further validates the effectiveness of ISRNet in handling complex and variable scenarios.

[0079] 3. Qualitative Analysis To visually demonstrate the segmentation performance of each model, this invention... Figure 7 The document provides a comparison of ISRNet with baseline methods. It can be seen that ISRNet produces more accurate saliency masks under various challenging conditions, specifically: In terms of fine structure capture, such as the legs of the climber in the second row, the antennae of the mantis in the third row, and the head details of the ladybug in the fifth row, ISRNet generates sharp and clear edges, preserving the geometric features of the objects to the greatest extent. For large object scenes, such as the train in the first row, the model effectively suppresses complex background clutter and accurately isolates the main target. Regarding target integrity and multi-target separation, the dog on the sofa in the fourth row demonstrates the model's excellent anti-adhesion ability and filling fullness, effectively avoiding erroneous region merging or internal holes.

[0080] In summary, this invention provides a high-fidelity saliency target detection method based on input-level scale refinement. By constructing learnable-scale stem-blocks at the input end to generate multi-scale, multi-view input streams from normal, overview, and fixation perspectives, it overcomes the loss of detail information caused by traditional downsampling at the source. It utilizes heterogeneous encoding of semantic flow and detail flow to extract high-level semantics and multi-scale detail features respectively, and dynamically integrates effective information from different perspectives through a cross-scale attention fusion module with a self-attention mechanism, effectively bridging the semantic gap between scales. In the decoding stage, a context-structure co-decoder composed of contrastive channel attention and boundary space attention is designed to explicitly decouple the foreground and background and focus on optimizing the blurred edges and transition regions of the target. Combined with a hybrid loss function including uncertainty-aware loss for deep supervision, it significantly improves the structural integrity and edge fidelity of the saliency mask. Experimental results show that this invention meets or surpasses existing state-of-the-art methods in various evaluation metrics on multiple mainstream datasets, demonstrating significant advantages in large-size target integrity, fine structure preservation, and complex background interference suppression, exhibiting excellent generalization ability and practical application value.

[0081] Furthermore, in some embodiments, the present invention also provides a high-fidelity saliency target detection system based on input-level scale refinement, for implementing the aforementioned high-fidelity saliency target detection method based on input-level scale refinement, the system comprising: A multi-view input generation module is used to process an input image into three multi-view input images containing different spatial frequency information. The multi-view input images include a normal view image, an overview view image, and a gaze view image. The heterogeneous dual-stream coding module is used to extract semantic features and multi-scale detail feature maps through semantic coding path and detail coding path, respectively; A cross-scale feature alignment and fusion module is used to align and fuse the multi-scale detail feature maps to generate fused features and an initial saliency mask; The context-structure co-decoding module is used to co-decode features to output a high-fidelity saliency mask.

[0082] Although the invention has been described with reference to preferred embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, the technical features mentioned in the various embodiments can be combined in any manner as long as there is no structural conflict. The invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A high-fidelity saliency target detection method based on input-level scale refinement, characterized in that, Includes the following steps: Step 1: Input the input image into a learnable scale stem block and generate three multi-view input images containing different spatial frequency information through parallel processing paths. The multi-view input images include a normal view image that maintains the original resolution, an overview view image generated by downsampling, and a gaze view image generated by upsampling and residual repair mechanism. Step 2: Encode the multi-view input images using a semantic encoding path and a detail encoding path respectively. The semantic encoding path uses a pre-trained backbone network P2T to extract high-level semantic features and generates channel-uniform semantic features through a dimension reduction module. The detail encoding path uses a set of weight-shared convolutional neural network encoders to process the normal view image, overview view image, and gaze view image respectively, and uses the local inductive bias of the convolutional neural network to capture edge details and generate multi-scale detail feature maps. Step 3: Construct a cross-scale attention fusion module at the end of the encoding, align the multi-scale detail feature maps in the spatial dimension, use the self-attention mechanism to calculate spatial correlation to mine global spatial context dependency, dynamically weight and fuse information from different perspectives, and generate fused features and initial saliency mask. Step 4: Construct a decoding network consisting of multiple cascaded context-structure co-decoders; Each context-structured co-decoder receives the current level encoder skip connection features from the semantic coding path, the output features of the previous level decoder, and the saliency mask predicted by the previous level. It uses a two-stream co-mechanism consisting of contrast channel attention and boundary space attention to decouple the foreground and background features and guide the network to focus on the blurred edges and transition regions of the target to output a high-fidelity saliency mask.

2. The method according to claim 1, characterized in that, The learnable scale stem block mentioned in step 1 generates the multi-view input image in the following manner: The normal view image is the input image itself; The overview view image is obtained by downsampling the input image using a depthwise separable convolution with a stride of 2, followed by batch normalization and activation function processing. The gaze view image is obtained by first upsampling the input image using bilinear interpolation to obtain a coarse high-resolution image, then processing the coarse high-resolution image using a lightweight residual repair module to obtain residual information, adding the coarse high-resolution image and the residual information and processing them with an activation function, wherein the lightweight residual repair module includes a depthwise separable convolution with a stride of 1 and a batch normalization layer.

3. The method according to claim 1, characterized in that, The processing procedure of the cross-scale attention fusion module in step 3 includes: The multi-scale detail feature maps are upsampled to the same resolution to obtain aligned feature map groups. The aligned feature map group is averaged to obtain an average feature map as the baseline feature. The baseline features are projected into queries, keys, and values ​​using parallel convolutional layers, and a spatial correlation matrix is ​​calculated by scaling dot product attention to generate weighted features. The weighted features are residually connected to the baseline features to generate the fused features, wherein the residual connection introduces a learnable scaling factor with an initial value of 0; The initial saliency mask is generated by mapping the fused features through convolutional layers.

4. The method according to claim 1, characterized in that, The dual-stream collaborative mechanism described in step 4 includes: The contrast channel attention branch is used to generate contrast vectors between the foreground and the background based on the saliency mask predicted by the previous level. The contrast relationship is learned through a multilayer perceptron to generate channel attention weights and recalibrate the input features to suppress the background channel response. The boundary spatial attention branch is used to generate a boundary guidance map based on the saliency mask predicted by the previous level, spatially weight the dimensionality-reduced query features, and generate a spatial attention map through depthwise separable convolution to obtain structurally refined features. The features processed by the contrast channel attention branch and the boundary space attention branch are then subjected to residual fusion.

5. The method according to claim 4, characterized in that, The processing procedure for the contrast channel attention branch includes: The saliency mask of the previous level prediction is binarized based on a preset threshold to obtain the foreground mask and the background mask; The foreground and background masks are used to perform weighted average pooling on the input features to obtain the foreground vector and background vector, respectively. After concatenating the foreground and background vectors along the channel dimension, channel attention weights are generated using a multilayer perceptron and activation function. Multiply the channel attention weights by the input features to obtain the channel-recalibrated features.

6. The method according to claim 4, characterized in that, The processing procedure for the boundary space attention branch includes: Based on the saliency mask M predicted at the previous level, using the formula... The boundary guidance map G is calculated; The boundary guidance graph is used to spatially weight the dimensionality-reduced query features to obtain guidance features; The guiding features are passed through a depthwise separable convolutional module to generate a spatial attention map; The spatial attention map is combined with the input features through residual connections to obtain the structurally refined features, wherein the residual connections introduce a learnable scaling parameter with an initial value of 0.

7. The method according to claim 1, characterized in that, The method also includes: Step 5, supervising the training of the model using a hybrid loss function that includes weighted intersection-union loss, weighted binary cross-entropy loss, and uncertainty-aware loss; the uncertainty-aware loss is used to emphasize the uncertain region where the prediction probability is close to 0.5, and its loss value decreases as the prediction probability deviates from 0.5, and the balance coefficient of the uncertainty-aware loss is dynamically adjusted using a cosine strategy; the method also includes: using the hybrid loss function to calculate the loss of the initial saliency mask generated in Step 3 for supervised training.

8. A high-fidelity saliency target detection system based on input-level scale refinement, characterized in that, The system for implementing the method as described in any one of claims 1 to 7 comprises: A multi-view input generation module is used to process an input image into three multi-view input images containing different spatial frequency information. The multi-view input images include a normal view image, an overview view image, and a gaze view image. The heterogeneous dual-stream coding module is used to extract semantic features and multi-scale detail feature maps through semantic coding path and detail coding path, respectively; A cross-scale feature alignment and fusion module is used to align and fuse the multi-scale detail feature maps to generate fused features and an initial saliency mask; The context-structure co-decoding module is used to co-decode features to output a high-fidelity saliency mask.

Images