A saliency detection method for RGB-D images based on frequency-decoupled modal interactions

Abstract

This invention proposes a salient target detection method for RGB-D images based on frequency-decoupled modal interactions. First, multi-stage feature extraction is performed on the RGB-D modal data to obtain multi-stage feature representations of the RGB-D modalities. Frequency-domain-aware cross-modal interactions are then performed on these multi-stage feature representations to obtain frequency-aware cross-modal spatial interaction features. Discriminative enhancement and cross-modal fusion are then applied to these frequency-aware cross-modal spatial interaction features to obtain the final multi-modal fused features. Based on these final multi-modal fused features, a salient target detection prediction map is generated through multi-scale aggregation and global dependency modeling. This invention utilizes a frequency-crossing Mamba fusion module to explicitly model cross-modal correlations within the frequency domain during feature learning, significantly alleviating the problem that traditional methods may ignore the intrinsic relationships between modalities when directly fusing in the spatial domain, thus improving the performance and generalization ability of salient target detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and multimodal image processing technology, and in particular to a saliency detection method for RGB-D images based on frequency-decoupled modal interaction. Background Technology

[0002] In the fields of computer vision and multimodal image processing, saliency detection of RGB-D images has gradually become an important research direction in recent years. Because RGB-D images simultaneously contain rich semantic texture and spatial geometric information, they have been widely used in practical scenarios such as intelligent driving environment perception, industrial defect detection, and medical image analysis. Saliency detection aims to automatically locate the most visually attractive salient regions in an image by analyzing RGB-D bimodal data, providing core pre-processing support for subsequent high-level vision tasks such as target tracking, image segmentation, and visual navigation.

[0003] However, RGB-D images often suffer from modal information heterogeneity, depth map noise interference such as sensor artifacts, blurred boundaries, and semantic inconsistencies between modalities, such as overlap between RGB texture-rich regions and regions lacking depth structure. These issues pose significant challenges to the accuracy and robustness of RGB-D saliency detection. Traditional multimodal saliency detection methods rely on simple feature stitching or weighted fusion in the spatial domain, which struggles to effectively mine complementary features between RGB and depth modalities and has poor adaptability to complex scenes such as low light, occlusion, and low-quality depth data.

[0004] In recent years, the rapid development of deep learning technology has driven the advancement of RGB-D saliency detection techniques. In particular, convolutional neural networks (CNNs) and the Transformer architecture have demonstrated superior performance in feature extraction and global modeling, leading to significant achievements in deep learning-based RGB-D saliency detection methods. However, CNNs are limited by their local receptive fields, making it difficult to capture long-distance dependencies across modalities. While Transformers possess global modeling capabilities, their high computational complexity makes them unsuitable for the real-time processing requirements of high-resolution RGB-D images. Furthermore, salient targets typically exhibit complex scale variations and edge structures; relying solely on a single modal branch or single-scale feature fusion is insufficient to comprehensively depict the detailed textures and overall spatial layout of salient regions.

[0005] To address the aforementioned issues, existing research has introduced bi-branch modal decoupling, frequency domain feature interaction, and the Mamba linear state-space architecture to enhance the model's cross-modal complementary modeling and efficient global perception capabilities. In particular, the cross-modal frequency domain correlation and hierarchical feature fusion strategies effectively separate RGB high-frequency texture from low-frequency deep structural information, contributing to more accurate salient region localization and boundary refinement. However, existing methods still have limitations in learning cross-modal frequency domain interaction depth and hierarchical feature association. They struggle to fully exploit the complex structural features of salient targets and their fine-grained differences from the background region, and their detection performance in low-quality deep data or modal inconsistency scenarios still has room for improvement. Summary of the Invention

[0006] In view of the above situation, the main objective of this invention is to propose an RGB-D image saliency detection method based on frequency decoupled modal interaction to solve the above-mentioned technical problems.

[0007] This invention proposes a saliency detection method for RGB-D images based on frequency-decoupled modal interaction, the method comprising the following steps: Step 1: Perform multi-stage feature extraction on the RGB-D modal data to obtain the multi-stage feature representation of the RGB-D modality; Step 2: Perform frequency-domain-aware cross-modal interaction on the feature representation of the RGB-D modality multi-stage to obtain frequency-aware cross-modal spatial interaction features; Step 3: Perform discriminative enhancement and cross-modal fusion on the frequency-aware cross-modal spatial interaction features to obtain the final multimodal fusion features; Step 4: Based on the final multimodal fusion features, generate a salient target detection prediction map through multi-scale aggregation and global dependency modeling.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention introduces a frequency-decoupled cross-modal interaction module, which includes a cross-modal frequency interaction module and a multimodal feature fusion Mamba module. This effectively realizes the deep correlation and complementary fusion between RGB and deep bimodal high and low frequency features, fully explores the fine-grained texture features and global spatial structure information of salient regions, thereby improving the accuracy and completeness of RGB-D image saliency detection. 2. This invention utilizes a multi-level Mamba fusion module, which integrates the Visual State Space Module (VSSB) and channel attention to explicitly enhance the perception of salient target boundary regions during feature decoding. This significantly alleviates the problems of blurred boundaries and inaccurate positioning in traditional spatial fusion methods, and improves the precision and coherence of salient target contour extraction. 3. This invention is based on a multi-scale feature extraction strategy of a dual-branch Mamba encoder, combined with a joint interaction mechanism of frequency domain and spatial domain, to achieve efficient integration and optimization of features of different resolutions and modalities. This enables the system to adapt to the diverse changes in the shape, size and scene complexity of salient targets, and enhances the robustness and generalization ability of the saliency detection model in complex scenarios such as low-quality depth data and modal inconsistency. 4. This invention employs a joint optimization strategy that combines the redesigned structural quality-aware loss (SQA Loss) and pixel position-aware loss (wBCE+wIoU) to explicitly strengthen the constraint on structural distortion in salient regions during model training. This significantly alleviates the problem that traditional loss functions only focus on pixel-level errors and ignore the consistency of human visual perception, thereby improving the visual quality and structural integrity of the final saliency map.

[0009] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0010] Figure 1 This is a flowchart of the steps of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0011] Figure 2 This is a structural diagram of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0012] Figure 3 This is a schematic diagram of the cross-modal frequency interaction module of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0013] Figure 4 This is a schematic diagram of the Mamba module for multimodal feature fusion in the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0014] Figure 5 This is a schematic diagram of the multi-level Mamba fusion module of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0015] Figure 6 This is a schematic diagram of the reconstructed image of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0016] Figure 7 This is a comparison chart of the results of the frequency-decoupled modal interaction-based RGB-D image saliency detection method proposed in this invention with existing methods.

[0017] Figure 8 The results of quantitative comparison experiments between the frequency-decoupled modal interaction-based RGB-D image saliency detection method proposed in this invention and existing methods on the datasets NJU2K, NLPR, STERE, SIP, LFSD, and SSD are presented. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to provide some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0020] Please see Figure 1 This invention proposes a saliency detection method for RGB-D images based on frequency-decoupled modal interactions, which includes the following steps: Step 1: Perform multi-stage feature extraction on the RGB-D modal data to obtain the multi-stage feature representation of the RGB-D modality.

[0021] Please see Figure 2 In a preferred embodiment of the present invention, multi-stage feature extraction is performed on the RGB-D modal data to obtain multi-stage feature representations of the RGB-D modality. The corresponding relationship in the process is as follows: ; ; in, These represent the input RGB color image and the depth image, respectively. These represent multi-stage RGB and depth features, respectively. , ; The encoder backbone networks for RGB color images and Depth images are represented respectively. These encoder backbone networks use multiple sets of downsampling and Visual State Space Module (VSSB) for continuous processing to extract multi-scale RGB and depth features. These represent the batch size, channel size, height, and width of the feature map in the i-th stage, respectively. Indicates feature dimension as The space of the set of real numbers.

[0022] It should be noted that a multi-level feature extraction network is constructed based on the encoder-decoder structure. Figure 2 In this context, FCMF represents the frequency decoupling cross-modal interaction module, which includes the FCMI cross-modal frequency interaction module and the MFFM multimodal feature fusion Mamba module. MSMF represents the multi-level Mamba fusion module, DP represents the Drop Path operation, and P4, P3, P2, and P1 represent the prediction saliency maps of each layer of the decoder, with P1 being the final saliency prediction map.

[0023] Step 2: Perform frequency-domain-aware cross-modal interaction on the feature representation of the RGB-D modality multi-stage to obtain frequency-aware cross-modal spatial interaction features.

[0024] Please see Figure 3 As a preferred embodiment of the present invention, frequency-domain-aware cross-modal interaction is performed on the feature representation of RGB-D modality in multiple stages to obtain frequency-aware cross-modal spatial interaction features, specifically including the following steps: Frequency decoupling is performed on the feature representation of the RGB-D mode in multiple stages to obtain amplitude and phase features; The frequency domain features after the interaction are obtained by correlation modeling of the two types of features through cross-modal Mamba interaction; The frequency domain features after interaction are transformed into the spatial domain, and the residual connection is combined to obtain frequency-aware cross-modal spatial domain interaction features.

[0025] Frequency decoupling is performed on the feature representation of the RGB-D mode in multiple stages to obtain the amplitude and phase features. The corresponding relationship is as follows: ; ; ; ; in, These represent the RGB features and depth features in the frequency domain, respectively. Represents the Fast Fourier Transform. Indicates a linear projection layer. Indicates the activation function; and This represents a depthwise separable convolution operation with two 3×3 kernels. and This represents a convolution operation with two 1×1 kernels. These represent the amplitude components of the RGB features and the depth features, respectively. These represent the phase components of the RGB features and the depth features, respectively. This indicates that the magnitude is calculated element by element. This represents the operation of extracting the phase angle of each element in a complex tensor; In the step of obtaining the frequency domain features after cross-modal Mamba interaction by correlating the amplitude and phase features, since the amplitude and phase interaction processes are similar, the corresponding relationship for the amplitude process is as follows: ; ; ; ; ; ; ; ; ; ; in, Let represent the fundamental learnable parameters of the RGB amplitude features and the Depth amplitude features in the Mamba continuous state-space model, respectively. Indicates to The Mamba dynamic parameters obtained through two linear projection layers, Indicates to The dynamic parameters of Mamba obtained through two full connection operations. Indicates to and The Mamba state transition weights obtained by performing exponential operations. Indicates to and The Mamba state transition weights obtained by performing exponential operations. express and The Mamba state transition bias obtained by performing a product operation. express and The Mamba state transition bias obtained by performing a product operation. This represents the Mamba hidden state at step t of the RGB modality in stage i. This represents the Mamba hidden state at step t of the i-th stage deep mode. This represents the Mamba output feature of the RGB mode at step t in stage i. This represents the Mamba output feature of the i-th stage deep mode at step t. This represents the concatenation result of the Mamba output features of all steps in the RGB modality of the i-th stage; This represents the concatenation result of the Mamba output features of all steps in the i-th stage deep modality; The frequency domain features after interaction are transformed back to the spatial domain, and combined with residual connections to obtain frequency-aware cross-modal spatial domain interaction features. The corresponding relationship in the process is as follows: ; ; in, This represents the amplitude and phase of the RGB mode in the i-th stage after frequency domain interaction. This represents the amplitude and phase of the i-th stage deep mode after frequency domain interaction. This represents the inverse operation of the Fast Fourier Transform. This indicates the Drop Path operation. This represents the RGB characteristics after frequency domain interaction. This represents the deep features obtained through frequency domain interaction.

[0026] It should be noted that amplitude interaction and phase interaction were constructed based on the feature frequency domain interaction mechanism, which fully facilitated the interaction of multimodal features.

[0027] Step 3 performs discriminative enhancement and cross-modal fusion on the frequency-aware cross-modal spatial interaction features to obtain the final multimodal fusion features.

[0028] Please see Figure 4 In a preferred embodiment of the present invention, discriminative enhancement and cross-modal fusion are performed on frequency-aware cross-modal spatial interaction features to obtain the final multimodal fusion features, specifically including the following steps: The frequency-aware cross-modal spatial interaction features are refined using single-modal features to obtain features with enhanced discriminative power. The corresponding relationship in this process is as follows: ; ; in, RGB features that enhance discriminative power Deep features that enhance discriminative power and This represents two 3×3 depthwise separable convolutional layers. Indicates the SiLU activation function; Multimodal correlation features are obtained by fusing cross-modal state spaces using a structured state-space model to enhance discriminative features. The corresponding relationship in this process is as follows: ; ; ; in, Indicates a channel-level connection. Represents the state space module. Indicated by depth features RGB interaction features after state-space model interaction processing Indicates using RGB features Deep features after interactive processing using the state-space model These are the correlated RGB features and depth features, respectively. Indicates feature separation operation; Adaptive enhancement of key feature channels is performed on multimodal correlation features to obtain primary multimodal fusion features. The corresponding relationship in this process is as follows: ; ; ; in, This indicates that all draws are pooled. This represents RGB features that have undergone adaptive enhancement of key feature channels. This represents deep features that have undergone adaptive enhancement of key feature channels. This represents the multimodal fusion feature of the i-th stage. This represents a stacked operation consisting of a linear layer, a SiLU activation function, another linear layer, and a Sigmoid activation function. This represents element-wise multiplication; The final fused feature is obtained by performing residual operations on the primary multimodal fusion features. The corresponding relationship in this process is as follows: ; in, This represents the hierarchical features of the i-th stage. This represents the element-wise summation operation.

[0029] It should be noted that the Mamba module for multimodal feature fusion performs bimodal feature fusion and refinement on the multi-stage frequency-aware cross-modal spatial interaction features of RGB-D images. Specifically, this branch first mines discriminative features of RGB and deep modalities through single-modal feature refinement, then uses a structured state-space model to achieve cross-modal state-space fusion to construct multimodal correlation features. Subsequently, it enhances key feature channels through adaptive channel enhancement, ultimately outputting hierarchical fusion features that combine modal complementarity and structural integrity, providing high-quality feature support for subsequent saliency prediction.

[0030] Step 4: Based on the final multimodal fusion features, generate a salient target detection prediction map through multi-scale aggregation and global dependency modeling.

[0031] Please see Figure 5 In a preferred embodiment of the present invention, based on the final multimodal fusion features, a salient target detection prediction map is generated through multi-scale aggregation and global dependency modeling, specifically including the following steps: The final multimodal fusion feature of the previous layer is upsampled and then added to the final multimodal fusion feature of the current layer to obtain the multi-scale aggregated feature. The corresponding relationship is as follows: ; in, This indicates a 2x upsampling operation. Indicates multi-scale aggregation features; By modeling long-range dependencies using a structured state-space model and then combining residual enhancement to preserve the original information, we obtain features with a coherent global structure and preserved local details. The corresponding relationship in this process is as follows: ; in, Indicates batch normalization, This indicates that the learnable scaling factor parameter is used to balance the residual contributions. Features that indicate a coherent global structure while preserving local details; Using a channel attention mechanism on features that are globally coherent while preserving local details, semantically focused features are obtained. The corresponding relationship in this process is as follows: ; in, Indicates a channel attention block. This indicates that the learnable scaling factor parameter is used to balance the residual contributions. Features that indicate semantic focus.

[0032] In a preferred embodiment of the present invention, the method is based on a saliency detection model. The training method for the saliency detection model in performing steps 1 to 4 includes the following steps: Given a training set, which includes RGB images, depth images, and ground truth labels for salient object detection; The RGB image and depth image are reconstructed using salient object detection, ground truth labels, and predicted maps, resulting in RGB-distorted reconstructed images and depth-distorted reconstructed images, respectively. The corresponding relationship in the process is as follows: ; ; ; in, Represents the actual label, and 1 represents a tensor where all elements are equal to 1. This represents the RGB foreground image extracted from an RGB image. This represents the depth foreground image extracted from the depth image. Represents an RGB background image. Represents a depth background image. This represents a significance prediction plot. This indicates an image reconstructed from RGB distortion. Represents a depth-distorted reconstructed image; The relationship between reconstructing an image using RGB distortion and constructing an RGB image quality perception loss from an RGB image is as follows: ; in, L1 paradigm loss, This represents a constant used to balance the importance of the two types of losses. This indicates the perceived quality loss of RGB images. This represents the loss as a structural similarity index measure; The relationship between reconstructing an image using depth distortion and constructing a pixel-position-aware loss from a depth image is as follows: ; in, This represents the perceived loss of depth image quality. The total SQA loss is obtained by adding the RGB image quality-perceived loss to the depth image quality-perceived loss, and the corresponding process has the following relationship: ; in, This represents the total SQA loss; A pixel position-aware loss is constructed using weighted binary cross-entropy loss and weighted IoU loss, and the total SQA loss is fused with the pixel position-aware loss as the final optimized loss. The corresponding relationship in this process is as follows: ; in, This represents the final optimization loss. This represents the pixel position awareness loss, which consists of a weighted binary cross-entropy loss and a weighted IoU loss. The RGB-D image saliency detection model is trained by minimizing the final optimization loss through updating the weights and learning parameters.

[0033] Please see Figure 7 , Figure 7 This diagram shows a comparison of prediction results between the method proposed in this invention and existing methods. RGB represents the input RGB image, Depth represents the depth image, GT represents the ground truth label for salient object detection, Ours represents the result of the proposed method, and SPSN, HAINet, HiDANet, RD3D, SPNet, DIGRNet, BBSNet, D3Net, BiANet, DSNet, CAVER, and TPCL represent the results of other existing salient object detection methods. Figure 6 As can be seen, the method proposed in this embodiment of the invention can achieve better segmentation results. (Comparison) Figure 7 Lines 1 and 7 show that the method proposed in this embodiment can more accurately segment fine-grained structures such as fence meshes and eye regions obscured by masks. These regions are characterized by narrow boundaries and dense details; traditional models are prone to missed detections or blurred boundaries, while the method proposed in this embodiment has a stronger ability to preserve details. Figure 7 In line 3, when faced with a cluttered background, the method proposed in this embodiment of the invention can effectively distinguish between foreground targets and background interference, resulting in a clearer saliency map; in comparison... Figure 7 In line 8, when the depth map has problems such as noise and missing data, the method proposed in this embodiment can still output more complete prediction results, indicating that the model is robust to depth input and will not fail due to the deterioration of depth quality.

[0034] Please see Figure 8 , Figure 8 This paper presents quantitative comparative experimental results between the proposed method of this invention and existing methods on the datasets NJU2K, NLPR, STERE, SIP, LFSD, and SSD. Figure 8 As can be seen, on the NJU2K dataset, the MAE of the method proposed in this embodiment is 0.024, which is lower than CAVER's 0.029 and DIGRNet's 0.026. The MAE of the proposed method is 0.948, which is higher than CAVER's 0.928 and DIGRNet's 0.939, achieving more accurate depiction of object edges and subtle textures. On the NLPR dataset, the MAE of the proposed method is 0.015, significantly lower than CATNet's 0.018 and RD3D's 0.022, effectively alleviating the detail loss problem of the above models in complex edge scenes. On the STERE dataset, the MAE of the proposed method is 0.029, which is better than DIGRNet's 0.037, further verifying the model's superiority in detail feature mining and achieving more refined salient region segmentation. It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0035] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0036] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A saliency detection method for RGB-D images based on frequency-decoupled modal interaction, characterized in that, The method includes the following steps: Step 1: Perform multi-stage feature extraction on the RGB-D modal data to obtain the multi-stage feature representation of the RGB-D modality; Step 2: Perform frequency-domain-aware cross-modal interaction on the feature representation of the RGB-D modality multi-stage to obtain frequency-aware cross-modal spatial interaction features, specifically including the following steps: Frequency decoupling is performed on the feature representation of the RGB-D mode in multiple stages to obtain amplitude and phase features. The corresponding relationship is as follows: ； ； ； ； in, These represent the RGB features and depth features in the frequency domain, respectively. Represents the Fast Fourier Transform. Indicates a linear projection layer. Indicates the activation function; and This represents a depthwise separable convolution operation with two 3×3 kernels. and This represents a convolution operation with two 1×1 kernels. These represent the amplitude components of the RGB features and the depth features, respectively. These represent the phase components of the RGB features and the depth features, respectively. This indicates that the magnitude is calculated element by element. This represents the operation of extracting the phase angle of each element in a complex tensor; The amplitude and phase features are correlated and modeled through cross-modal Mamba interaction to obtain the frequency domain features after the interaction. The corresponding relationship of the amplitude feature cross-modal Mamba interaction is as follows: ； ； ； ； ； ； ； ； ； ； in, Let represent the fundamental learnable parameters of the RGB amplitude features and the Depth amplitude features in the Mamba continuous state-space model, respectively. Indicates to The Mamba dynamic parameters obtained through two linear projection layers. Indicates to The dynamic parameters of Mamba obtained through two full connection operations. Indicates to and The Mamba state transition weights obtained by performing exponential operations Indicates to and The Mamba state transition weights obtained by performing exponential operations express and The Mamba state transition bias obtained by performing a product operation. express and The Mamba state transition bias obtained by performing a product operation. This represents the Mamba hidden state at step t of the RGB modality in stage i. This represents the Mamba hidden state at step t of the i-th stage deep mode. This represents the Mamba output feature of the RGB mode at step t in stage i. This represents the Mamba output feature of the i-th stage deep mode at step t. This represents the concatenation result of the Mamba output features of all steps in the RGB modality of the i-th stage; This represents the concatenation result of the Mamba output features of all steps in the i-th stage deep modality; The frequency domain features after interaction are transformed into spatial domain features, and combined with residual connections to obtain frequency-aware cross-modal spatial domain interactive features. The corresponding relationship in the process is as follows: ； ； in, This represents the amplitude and phase of the RGB mode in the i-th stage after frequency domain interaction. This represents the amplitude and phase of the i-th stage deep mode after frequency domain interaction. This represents the inverse operation of the Fast Fourier Transform. This represents the DropPath operation. This represents the RGB characteristics after frequency domain interaction. This represents the depth features after frequency domain interaction; Step 3: Perform discriminative enhancement and cross-modal fusion on the frequency-aware cross-modal spatial interaction features to obtain the final multimodal fusion features; Step 4: Based on the final multimodal fusion features, generate a salient target detection prediction map through multi-scale aggregation and global dependency modeling.

2. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 1, characterized in that, In step 1, multi-stage feature extraction is performed on the RGB-D modal data to obtain multi-stage feature representations of the RGB-D modality. The corresponding relationship in the process is as follows: ； ； in, These represent the input RGB color image and the depth image, respectively. These represent multi-stage RGB and depth features, respectively. , ; These represent the encoder backbone networks for RGB color images and Depth images, respectively. They represent the first i Batch size, channel size, height, and width of the stage feature map; Indicates feature dimension as The space of the set of real numbers.

3. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 2, characterized in that, In step 3, the frequency-aware cross-modal spatial interaction features are discriminatively enhanced and fused across modally to obtain the final multimodal fusion features. This process specifically includes the following steps: The frequency-aware cross-modal spatial interaction features are refined into single-modal features to obtain features with enhanced discriminative power. Multimodal correlation features are obtained by fusing cross-modal state spaces using a structured state-space model to enhance discriminative features; Adaptive enhancement of key feature channels is performed on multimodal correlation features to obtain primary multimodal fusion features; The final fused features are obtained by performing residual operations on the primary multimodal fusion features.

4. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 3, characterized in that, The frequency-aware cross-modal spatial interaction features are refined using single-modal features to obtain features with enhanced discriminative power. The corresponding relationship in this process is as follows: ； ； in, RGB features that enhance discriminative power Deep features that enhance discriminative power and This represents two 3×3 depthwise separable convolutional layers. Indicates the SiLU activation function; In the step of fusing cross-modal state spaces of the features that enhance discriminativeness using a structured state-space model to obtain multimodal associated features, the corresponding relationship in the process is as follows: ； ； ； in, Indicates a channel-level connection. Represents the state space module. Indicated by depth features RGB interaction features after state-space model interaction processing Indicates using RGB features Deep features after interactive processing using the state-space model These are the correlated RGB features and depth features, respectively. Indicates feature separation operation; In the steps of adaptively enhancing key feature channels on multimodal correlation features to obtain primary multimodal fusion features, the corresponding relationships in the process are as follows: ； ； ； in, This indicates that all draws are pooled. This represents RGB features that have undergone adaptive enhancement of key feature channels. This represents deep features that have undergone adaptive enhancement of key feature channels. This represents the multimodal fusion feature of the i-th stage. This represents a stacked operation consisting of a linear layer, a SiLU activation function, another linear layer, and a Sigmoid activation function. This represents element-wise multiplication; In the step of performing residual operations on the primary multimodal fusion features to obtain the final fusion features, the corresponding relationship in the process is as follows: ； in, This represents the hierarchical features of the i-th stage. This indicates the element-wise summation operation.

5. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 4, characterized in that, In step 4, the final multimodal fusion features are upsampled sequentially and then added to the features of the next layer to obtain multi-scale aggregated features. A structured state-space model is used to model long-range dependencies, and residual enhancement is combined to preserve the original information, resulting in features with a coherent global structure and preserved local details. Then, a channel attention mechanism is used to obtain semantically focused features. Finally, the output of the last layer is processed to obtain the final salient object detection prediction map. Specifically, the steps include: The final multimodal fusion feature of the previous layer is upsampled and then added to the final multimodal fusion feature of the current layer to obtain the multiscale aggregated feature; The structured state-space model is used to model long-range dependencies of multi-scale aggregated features, and then residual enhancement is combined to preserve the original information to obtain features with global structural coherence and local details. For features that are coherent in global structure and preserve local details, a channel attention mechanism is used to obtain semantically focused features. The final output of the last layer is then processed to obtain the final salient object detection prediction map.

6. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 5, characterized in that, The final multimodal fusion feature of the previous layer is upsampled and then added to the final multimodal fusion feature of the current layer to obtain the multi-scale aggregated feature. The corresponding relationship is as follows: ； in, This indicates a 2x upsampling operation. Indicates multi-scale aggregation features; By modeling long-range dependencies using a structured state-space model on multi-scale aggregated features and then combining residual enhancement to preserve the original information, we obtain features with a coherent global structure and preserved local details. The corresponding relationship in this process is as follows: ； in, Indicates batch normalization, This indicates that the learnable scaling factor parameter is used to balance the residual contributions. Features that indicate a coherent global structure while preserving local details; In the step of obtaining semantically focused features by using a channel attention mechanism on features that are coherent in the global structure while preserving local details, the corresponding relationship in the process is as follows: ； in, Indicates a channel attention block. This indicates that the learnable scaling factor parameter is used to balance the residual contributions. Features that indicate semantic focus.

7. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 6, characterized in that, The method is based on a saliency detection model. The training method for the saliency detection model in steps 1 to 4 includes the following steps: Given a training set, which includes RGB images, depth images, and ground truth labels for salient object detection; RGB and depth images are reconstructed using real labels and predicted maps from salient object detection, resulting in RGB-distorted reconstructed images and depth-distorted reconstructed images, respectively. Construct RGB image quality perceptual loss by reconstructing an image using RGB distortion and comparing it with an RGB image; Pixel position-aware loss is constructed by reconstructing images using depth distortion and using depth images; The total SQA loss is obtained by adding the RGB image quality perception loss to the depth image quality perception loss. We construct a pixel position-aware loss using weighted binary cross-entropy loss and weighted IoU loss, and then fuse the total SQA loss with the pixel position-aware loss to obtain the final optimized loss. The final optimization loss is minimized by updating the weights and learning parameters to train the RGB-D image saliency detection model.

8. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 7, characterized in that, The calculation processes for RGB distortion reconstructed images and depth distortion reconstructed images are represented by the following formulas: ； ； ； in, Represents the actual label, and 1 represents a tensor where all elements are equal to 1. This represents the RGB foreground image extracted from an RGB image. This represents the depth foreground image extracted from the depth image. Represents an RGB background image. Represents a depth background image. This represents a significance prediction plot. This indicates an image reconstructed from RGB distortion. Represents a depth-distorted reconstructed image; The calculation process for the perceived quality loss of an RGB image is as follows: ； in, L1 paradigm loss, This represents a constant used to balance the importance of the two types of losses. This indicates the perceived quality loss of RGB images. This represents the loss as a structural similarity index measure; The pixel position-aware loss corresponds to the following relationship in the process: ； in, This represents the perceived loss of depth image quality. The formula for calculating total SQA loss is as follows: ； in, This represents the total SQA loss; The final optimization loss corresponds to the following relationship in the process: ； in, This represents the final optimization loss. This represents the pixel position awareness loss, which consists of weighted binary cross-entropy loss and weighted IoU loss.

Images