RGB-D image saliency detection method based on frequency decoupling mode interaction

Abstract

The invention provides an RGB-D image salient target detection method based on frequency decoupling mode interaction, and the method comprises the steps: firstly carrying out the multi-stage feature extraction of RGB-D mode data, and obtaining the multi-stage feature representation of an RGB-D mode; performing frequency domain sensing cross-modal interaction on the RGB-D modal multi-stage feature representation to obtain frequency sensing cross-modal spatial domain interaction features; discriminative enhancement and cross-modal fusion are carried out on the cross-modal spatial domain interaction features of frequency sensing, and final multi-modal fusion features are obtained; and based on the final multi-modal fusion features, through multi-scale aggregation and global dependence modeling, generating a saliency target detection prediction map. According to the method, cross-modal association in a frequency domain is explicitly modeled in a feature learning process by using a frequency cross Mama fusion module, so that the problem that internal relationships among modals may be ignored when fusion is directly performed in a spatial domain in a traditional method is remarkably relieved, and the performance and generalization ability of salient target detection are improved.

Description

A saliency detection method for RGB-D images based on frequency-decoupled modal interactions Technical Field

[0001] This invention relates to the fields of computer vision and multimodal image processing technology, and in particular to a saliency detection method for RGB-D images based on frequency-decoupled modal interaction. Background Technology

[0002] In the fields of computer vision and multimodal image processing, saliency detection of RGB-D images has gradually become an important research direction in recent years. Because RGB-D images simultaneously contain rich semantic texture and spatial geometric information, they have been widely used in practical scenarios such as intelligent driving environment perception, industrial defect detection, and medical image analysis. Saliency detection aims to automatically locate the most visually attractive salient regions in an image by analyzing RGB-D bimodal data, providing core pre-processing support for subsequent high-level vision tasks such as target tracking, image segmentation, and visual navigation.

[0003] However, RGB-D images often suffer from modal information heterogeneity, depth map noise interference such as sensor artifacts, blurred boundaries, and semantic inconsistencies between modalities, such as overlap between RGB texture-rich regions and regions lacking depth structure. These issues pose significant challenges to the accuracy and robustness of RGB-D saliency detection. Traditional multimodal saliency detection methods rely on simple feature stitching or weighted fusion in the spatial domain, which struggles to effectively mine complementary features between RGB and depth modalities and has poor adaptability to complex scenes such as low light, occlusion, and low-quality depth data.

[0004] In recent years, the rapid development of deep learning technology has driven the advancement of RGB-D saliency detection techniques. In particular, convolutional neural networks (CNNs) and the Transformer architecture have demonstrated superior performance in feature extraction and global modeling, leading to significant achievements in deep learning-based RGB-D saliency detection methods. However, CNNs are limited by their local receptive fields, making it difficult to capture long-distance dependencies across modalities. While Transformers possess global modeling capabilities, their high computational complexity makes them unsuitable for the real-time processing requirements of high-resolution RGB-D images. Furthermore, salient targets typically exhibit complex scale variations and edge structures; relying solely on a single modal branch or single-scale feature fusion is insufficient to comprehensively depict the detailed textures and overall spatial layout of salient regions.

[0005] To address the aforementioned issues, existing research has introduced bi-branch modal decoupling, frequency domain feature interaction, and the Mamba linear state-space architecture to enhance the model's cross-modal complementary modeling and efficient global perception capabilities. In particular, the cross-modal frequency domain correlation and hierarchical feature fusion strategies effectively separate RGB high-frequency texture from low-frequency deep structural information, contributing to more accurate salient region localization and boundary refinement. However, existing methods still have limitations in learning cross-modal frequency domain interaction depth and hierarchical feature association. They struggle to fully exploit the complex structural features of salient targets and their fine-grained differences from the background region, and their detection performance in low-quality deep data or modal inconsistency scenarios still has room for improvement. Summary of the Invention

[0006] In view of the above situation, the main objective of this invention is to propose an RGB-D image saliency detection method based on frequency decoupled modal interaction to solve the above-mentioned technical problems.

[0007] This invention proposes a saliency detection method for RGB-D images based on frequency-decoupled modal interaction. The method includes the following steps: Step 1, performing multi-stage feature extraction on RGB-D modal data to obtain multi-stage feature representations of RGB-D modality; Step 2, performing frequency-domain-aware cross-modal interaction on the multi-stage feature representations of RGB-D modality to obtain frequency-aware cross-modal spatial interaction features; Step 3, performing discriminative enhancement and cross-modal fusion on the frequency-aware cross-modal spatial interaction features to obtain final multi-modal fusion features; Step 4, generating a saliency target detection prediction map based on the final multi-modal fusion features through multi-scale aggregation and global dependency modeling.

[0008] Compared with existing technologies, the beneficial effects of this invention are as follows: 1. This invention introduces a frequency-decoupled cross-modal interaction module, which includes a cross-modal frequency interaction module and a multi-modal feature fusion Mamba module. This effectively realizes the deep correlation and complementary fusion between RGB and depth bimodal high and low frequency features, fully mining the fine-grained texture features and global spatial structure information of salient regions, thereby improving the accuracy and completeness of RGB-D image saliency detection; 2. This invention utilizes a multi-level Mamba fusion module, integrating the Visual State Space Module (VSSB) and channel attention, to explicitly enhance the perception ability of salient target boundary regions during feature decoding. This significantly alleviates the problems of boundary blurring and inaccurate positioning in traditional spatial fusion methods, improving the precision and coherence of salient target contour extraction; 3. This invention is based on dual-branch Mamba. The encoder's multi-scale feature extraction strategy, combined with the joint interaction mechanism of the frequency domain and spatial domain, achieves efficient integration and optimization of features of different resolutions and modalities. This enables the system to adapt to the diverse changes in the shape, size, and scene complexity of salient targets, enhancing the robustness and generalization ability of the saliency detection model in complex scenarios such as low-quality depth data and modal inconsistency. 4. This invention designs a joint optimization strategy of reconstructing image redesign structure quality perception loss (SQA Loss) and pixel position perception loss (wBCE+wIoU). In model training, it explicitly strengthens the constraint on structural distortion of salient regions, significantly alleviating the problem that traditional loss functions only focus on pixel-level errors and ignore the consistency of human visual perception, thus improving the visual quality and structural integrity of the final saliency map.

[0009] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by means of embodiments of the invention. Attached Figure Description

[0010] Figure 1 is a flowchart of the steps of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0011] Figure 2 is a structural diagram of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0012] Figure 3 is a schematic diagram of the cross-modal frequency interaction module of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0013] Figure 4 is a schematic diagram of the Mamba module for multimodal feature fusion in the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0014] Figure 5 is a schematic diagram of the multi-level Mamba fusion module of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0015] Figure 6 is a schematic diagram of the reconstructed image of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention.

[0016] Figure 7 is a comparison of the results of the RGB-D image saliency detection method based on frequency decoupling modal interaction proposed in this invention with existing methods.

[0017] Figure 8 shows the quantitative comparison results of the proposed frequency-decoupled modal interaction-based RGB-D image saliency detection method with existing methods on the datasets NJU2K, NLPR, STERE, SIP, LFSD, and SSD. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] These and other aspects of the embodiments of the present invention will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of the present invention are specifically disclosed to provide some ways of implementing the principles of the embodiments of the present invention; however, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0020] Please refer to Figure 1. This embodiment of the invention proposes an RGB-D image saliency detection method based on frequency decoupling modal interaction. The method includes the following steps: Step 1: Perform multi-stage feature extraction on RGB-D modal data to obtain multi-stage feature representation of RGB-D modality.

[0021] Please refer to Figure 2. As a preferred embodiment of the present invention, multi-stage feature extraction is performed on the RGB-D modal data to obtain multi-stage feature representations of the RGB-D modality. The corresponding relationship in the process is as follows: ; ;in, These represent the input RGB color image and the depth image, respectively. These represent multi-stage RGB and depth features, respectively. , ; The encoder backbone networks for RGB color images and Depth images are represented respectively. These encoder backbone networks use multiple sets of downsampling and Visual State Space Module (VSSB) for continuous processing to extract multi-scale RGB and depth features. These represent the batch size, channel size, height, and width of the feature map in the i-th stage, respectively. Indicates feature dimension as The space of the set of real numbers.

[0022] It should be noted that a multi-level feature extraction network is constructed based on the encoder-decoder structure. In Figure 2, FCMF represents the frequency decoupling cross-modal interaction module. The FCMF module includes the FCMI cross-modal frequency interaction module and the MFFM multi-modal feature fusion Mamba module. MSMF represents the multi-level Mamba fusion module. DP represents the Drop Path operation. P4, P3, P2, and P1 represent the prediction saliency maps of each layer of the decoder, where P1 is the final saliency prediction map.

[0023] Step 2: Perform frequency-domain-aware cross-modal interaction on the feature representation of the RGB-D modality multi-stage to obtain frequency-aware cross-modal spatial interaction features.

[0024] Please refer to Figure 3. As a preferred embodiment of the present invention, frequency-aware cross-modal interaction is performed on the feature representation of the RGB-D modal multi-stage to obtain frequency-aware cross-modal spatial interaction features. Specifically, the steps include: performing frequency decoupling processing on the feature representation of the RGB-D modal multi-stage to obtain amplitude and phase features; performing correlation modeling on the two types of features through cross-modal Mamba interaction to obtain the frequency domain features after interaction; converting the frequency domain features after interaction to the spatial domain and combining residual connections to obtain frequency-aware cross-modal spatial interaction features.

[0025] Frequency decoupling is performed on the feature representation of the RGB-D mode in multiple stages to obtain the amplitude and phase features. The corresponding relationship is as follows: ; ; ; ;in, These represent the RGB features and depth features in the frequency domain, respectively. Represents the Fast Fourier Transform. Indicates a linear projection layer. Indicates the activation function; and This represents a depthwise separable convolution operation with two 3×3 kernels. and This represents a convolution operation with two 1×1 kernels. These represent the amplitude components of the RGB features and the depth features, respectively. These represent the phase components of the RGB features and the depth features, respectively. This indicates that the magnitude is calculated element by element. This represents the operation of extracting the phase angle of each element in a complex tensor; in the step of modeling the correlation between amplitude and phase features through cross-modal Mamba interaction to obtain the frequency domain features after interaction, since the interaction processes of amplitude and phase are similar, the corresponding relationship in the process is as follows, taking amplitude as an example. ; ; ; ; ; ; ; ; ; ;in, Let represent the fundamental learnable parameters of the RGB amplitude features and the Depth amplitude features in the Mamba continuous state-space model, respectively. Indicates to The Mamba dynamic parameters obtained through two linear projection layers. Indicates to The dynamic Mamba parameters obtained through two full connection operations Indicates to and The Mamba state transition weights obtained by performing exponential operations. Indicates to and The Mamba state transition weights obtained by performing exponential operations. express and The Mamba state transition bias obtained by performing a product operation. express and The Mamba state transition bias obtained by performing a product operation. This represents the Mamba hidden state at step t of the RGB modality in stage i. This represents the Mamba hidden state at step t of the i-th stage deep mode. This represents the Mamba output feature of the RGB mode at step t in stage i. This represents the Mamba output feature of the i-th stage deep mode at step t. This represents the concatenation result of the Mamba output features of all steps in the RGB modality of the i-th stage; This represents the concatenation result of the Mamba output features of all steps in the deep modality at stage i; the frequency domain features after interaction are transformed back to the spatial domain, and combined with residual connections to obtain frequency-aware cross-modal spatial domain interactive features. The corresponding relationship in the process is as follows: ; ;in, This represents the amplitude and phase of the RGB mode in the i-th stage after frequency domain interaction. This represents the amplitude and phase of the i-th stage deep mode after frequency domain interaction. This represents the inverse operation of the Fast Fourier Transform. This indicates the Drop Path operation. This represents the RGB characteristics after frequency domain interaction. This represents the deep features obtained through frequency domain interaction.

[0026] It should be noted that amplitude interaction and phase interaction were constructed based on the feature frequency domain interaction mechanism, which fully facilitated the interaction of multimodal features.

[0027] Step 3 performs discriminative enhancement and cross-modal fusion on the frequency-aware cross-modal spatial interaction features to obtain the final multimodal fusion features.

[0028] Please refer to Figure 4. As a preferred embodiment of the present invention, the frequency-aware cross-modal spatial interaction features are subjected to discriminative enhancement and cross-modal fusion to obtain the final multimodal fusion features. Specifically, the following steps are included: the frequency-aware cross-modal spatial interaction features are refined into single-modal features to obtain features with enhanced discriminative power. The corresponding relationship in the process is as follows: ; ;in, RGB features that enhance discriminative power Deep features that enhance discriminative power and This represents two 3×3 depthwise separable convolutional layers. Let SiLU be the activation function. Multimodal correlation features are obtained by fusing cross-modal state spaces using a structured state-space model to enhance discriminative power. The corresponding relationship in this process is as follows: ; ; ;in, Indicates a channel-level connection. Represents the state space module. Indicated by depth features RGB interaction features after state-space model interaction processing Indicates using RGB features Deep features after interactive processing using the state-space model These are the correlated RGB features and depth features, respectively. The feature separation operation is represented; adaptive enhancement of key feature channels is performed on the multimodal associated features to obtain the primary multimodal fusion features. The corresponding relationship in the process is as follows: ; ; ;in, This indicates that all draws are pooled. This represents RGB features that have undergone adaptive enhancement of key feature channels. This represents deep features that have undergone adaptive enhancement of key feature channels. This represents the multimodal fusion feature of the i-th stage. This represents a stacked operation consisting of a linear layer, a SiLU activation function, another linear layer, and a Sigmoid activation function. This represents element-wise multiplication; the final fused feature is obtained by performing residual operations on the primary multimodal fusion features, and the corresponding relationship in the process is as follows: ;in, This represents the hierarchical features of the i-th stage. This represents the element-wise summation operation.

[0029] It should be noted that the Mamba module for multimodal feature fusion performs bimodal feature fusion and refinement on the multi-stage frequency-aware cross-modal spatial interaction features of RGB-D images. Specifically, this branch first mines discriminative features of RGB and deep modalities through single-modal feature refinement, then uses a structured state-space model to achieve cross-modal state-space fusion to construct multimodal correlation features. Subsequently, it enhances key feature channels through adaptive channel enhancement, ultimately outputting hierarchical fusion features that combine modal complementarity and structural integrity, providing high-quality feature support for subsequent saliency prediction.

[0030] Step 4: Based on the final multimodal fusion features, generate a salient target detection prediction map through multi-scale aggregation and global dependency modeling.

[0031] Please refer to Figure 5. As a preferred embodiment of the present invention, based on the final multimodal fusion features, a salient target detection prediction map is generated through multi-scale aggregation and global dependency modeling. Specifically, the following steps are included: upsampling the final multimodal fusion features of the previous layer and then adding them to the final multimodal fusion features of the current layer to obtain multi-scale aggregated features. The corresponding relationship is as follows: ;in, This indicates a 2x upsampling operation. This represents multi-scale aggregated features. A structured state-space model is used to model long-range dependencies in these features, and residual enhancement is then applied to preserve the original information, resulting in features with a coherent global structure and retained local details. The corresponding relationship in this process is as follows: ;in, Indicates batch normalization, This indicates that the learnable scaling factor parameter is used to balance the residual contributions. Features that represent a coherent global structure while preserving local details; using a channel attention mechanism on these features to obtain semantically focused features, the corresponding relationship is as follows: ;in, Indicates a channel attention block. This indicates that the learnable scaling factor parameter is used to balance the residual contributions. Features that indicate semantic focus.

[0032] In a preferred embodiment of the present invention, the method is based on a saliency detection model. In performing steps 1 to 4 above, the training method for the saliency detection model includes the following steps: Given a training set, which includes RGB images, depth images, and ground truth labels for salient object detection; reconstructing the RGB images and depth images using the ground truth labels and prediction maps for salient object detection, respectively obtaining RGB-distorted reconstructed images and depth-distorted reconstructed images. The corresponding relationship in this process is as follows: ; ; ;in, Represents the actual label, and 1 represents a tensor where all elements are equal to 1. This represents the RGB foreground image extracted from an RGB image. This represents the depth foreground image extracted from the depth image. Represents an RGB background image. Represents a depth background image. This represents a significance prediction plot. This indicates an image reconstructed from RGB distortion. The image represents a depth-distorted reconstructed image. The RGB image quality-perceived loss is constructed by comparing the RGB-distorted reconstructed image with the RGB image. The corresponding relationship in this process is as follows: ;in, L1 paradigm loss, This represents a constant used to balance the importance of the two types of losses. This indicates the perceived quality loss of RGB images. The structural similarity index loss is used to represent the loss; pixel location-aware loss is constructed by reconstructing the image using depth distortion and the depth image, and the corresponding relationship in the process is as follows: ;in, Let represent the depth image quality perception loss; add the RGB image quality perception loss to the depth image quality perception loss to obtain the total SQA loss, and the corresponding process has the following relationship: ;in, Let SQA represent the total SQA loss. A pixel position-aware loss is constructed using weighted binary cross-entropy loss and weighted IoU loss. The total SQA loss is then fused with the pixel position-aware loss to form the final optimized loss. The corresponding relationship in this process is as follows: ;in, This represents the final optimization loss. The pixel position-aware loss is represented by a weighted binary cross-entropy loss and a weighted IoU loss. The final optimized loss is minimized by updating the weights and learning parameters to train the RGB-D image saliency detection model.

[0033] Please refer to Figure 7, which shows a comparison of the prediction results of the method proposed in this embodiment and existing methods. RGB represents the input RGB image, Depth represents the depth image, GT represents the ground truth label for salient object detection, Ours represents the result of the method proposed in this invention, and SPSN, HAINet, HiDANet, RD3D, SPNet, DIGRNet, BBSNet, D3Net, BiANet, DSNet, CAVER, and TPCL represent the results of other existing salient object detection methods. As can be seen from Figure 6, the method proposed in this embodiment can achieve better segmentation results. Comparing rows 1 and 7 of Figure 7, it can be seen that the method proposed in this embodiment can more accurately segment fine-grained structures such as fence grids and eye regions obscured by masks. These regions are characterized by narrow boundaries and dense details, making them prone to missed detections or blurred boundaries in traditional models. However, the method proposed in this embodiment of the invention has a stronger ability to preserve details. Comparing with row 3 of Figure 7, when faced with cluttered backgrounds, the method proposed in this embodiment of the invention can effectively distinguish between foreground targets and background interference, generating a clearer saliency map. Comparing with row 8 of Figure 7, even when the depth map has noise, missing data, or other issues, the method proposed in this embodiment of the invention can still output more complete prediction results, indicating that the model is robust to depth input and will not fail due to a decrease in depth quality.

[0034] Please refer to Figure 8, which shows the quantitative comparative experimental results of the proposed method of this invention and existing methods on the datasets NJU2K, NLPR, STERE, SIP, LFSD, and SSD. As can be seen from Figure 8, on the NJU2K dataset, the MAE of the proposed method of this invention is 0.024, which is lower than CAVER's 0.029 and DIGRNet's 0.026. The MAE of the proposed method is 0.948, which is higher than CAVER's 0.928 and DIGRNet's 0.939, achieving more accurate depiction of object edges and subtle textures. On the NLPR dataset, the MAE of the proposed method is 0.015, significantly lower than CATNet's 0.018 and RD3D's 0.022, effectively alleviating the detail loss problem of the above models in complex edge scenes. On the STERE dataset, the MAE of the proposed method is 0.029, which is better than DIGRNet's 0.037, further verifying the model's superiority in detail feature mining and achieving more refined salient region segmentation. It should be understood that the various parts of this invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0035] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0036] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A saliency detection method for RGB-D images based on frequency-decoupled modal interaction, characterized in that, The method includes the following steps: Step 1, performing multi-stage feature extraction on RGB-D modal data to obtain multi-stage feature representations of RGB-D modality; Step 2, performing frequency-domain-aware cross-modal interaction on the multi-stage feature representations of RGB-D modality to obtain frequency-aware cross-modal spatial interaction features; Step 3, performing discriminative enhancement and cross-modal fusion on the frequency-aware cross-modal spatial interaction features to obtain the final multi-modal fusion features; Step 4: Based on the final multimodal fusion features, generate a salient target detection prediction map through multi-scale aggregation and global dependency modeling.

2. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 1, characterized in that, In step 1, multi-stage feature extraction is performed on the RGB-D modal data to obtain multi-stage feature representations of the RGB-D modality. The corresponding relationship in the process is as follows: ； ;in, These represent the input RGB color image and the depth image, respectively. These represent multi-stage RGB and depth features, respectively. ， ； These represent the encoder backbone networks for RGB color images and Depth images, respectively. These represent the batch size, channel size, height, and width of the feature map in the i-th stage, respectively. Indicates feature dimension as The space of the set of real numbers.

3. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 2, characterized in that, In step 2, frequency-aware cross-modal interaction is performed on the feature representation of the RGB-D modal multi-stage to obtain frequency-aware cross-modal spatial interaction features. Specifically, this includes the following steps: frequency decoupling processing is performed on the feature representation of the RGB-D modal multi-stage to obtain amplitude features and phase features. The amplitude and phase features are correlated and modeled through cross-modal Mamba interaction to obtain the frequency domain features after the interaction; The frequency domain features after interaction are transformed into spatial domain features, and combined with residual connections to obtain frequency-aware cross-modal spatial domain interaction features.

4. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 3, characterized in that, Frequency decoupling is performed on the feature representation of the RGB-D mode in multiple stages to obtain the amplitude and phase component features. The corresponding relationship in this process is as follows: ； ； ； ;in, These represent the RGB features and depth features in the frequency domain, respectively. Represents the Fast Fourier Transform. Indicates a linear projection layer. Indicates the activation function; and This represents a depthwise separable convolution operation with two 3×3 kernels. and This represents a convolution operation with two 1×1 kernels. These represent the amplitude components of the RGB features and the depth features, respectively. These represent the phase components of the RGB features and the depth features, respectively. This indicates that the magnitude is calculated element by element. This represents the operation of extracting the phase angle of each element in a complex tensor; in the step of modeling the correlation between amplitude and phase features through cross-modal Mamba interaction to obtain the frequency domain features after interaction, since the interaction processes of amplitude and phase are similar, the corresponding relationship for amplitude is as follows: ； ； ； ； ； ； ； ； ； ;in, Let represent the fundamental learnable parameters of the RGB amplitude features and the Depth amplitude features in the Mamba continuous state-space model, respectively. Indicates to The Mamba dynamic parameters obtained through two linear projection layers. Indicates to The dynamic parameters of Mamba obtained through two full connection operations. Indicates to and The Mamba state transition weights obtained by performing exponential operations Indicates to and The Mamba state transition weights obtained by performing exponential operations express and The Mamba state transition bias obtained by performing a product operation. express and The Mamba state transition bias obtained by performing a product operation. This represents the Mamba hidden state at step t of the RGB modality in stage i. This represents the Mamba hidden state at step t of the i-th stage deep mode. This represents the Mamba output feature of the RGB mode at step t in stage i. This represents the Mamba output feature of the i-th stage deep mode at step t. This represents the concatenation result of the Mamba output features of all steps in the RGB modality of the i-th stage; This represents the concatenation result of the Mamba output features of all steps in the i-th stage of the deep modality. In the step of converting the frequency domain features after interaction to spatial domain features and combining residual connections to obtain frequency-aware cross-modal spatial interaction features, the corresponding relationship in the process is as follows: ； ;in, This represents the amplitude and phase of the RGB mode in the i-th stage after frequency domain interaction. This represents the amplitude and phase of the i-th stage deep mode after frequency domain interaction. This represents the inverse operation of the Fast Fourier Transform. This indicates the Drop Path operation. This represents the RGB characteristics after frequency domain interaction. This represents the deep features obtained through frequency domain interaction.

5. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 4, characterized in that, In step 3, the frequency-aware cross-modal spatial interaction features are subjected to discriminative enhancement and cross-modal fusion to obtain the final multimodal fusion features. Specifically, the steps include: refining the frequency-aware cross-modal spatial interaction features into single-modal features to obtain features with enhanced discriminativeness. Multimodal correlation features are obtained by fusing cross-modal state spaces using a structured state-space model to enhance discriminative features; Adaptive enhancement of key feature channels is performed on multimodal correlation features to obtain primary multimodal fusion features; The final fused features are obtained by performing residual operations on the primary multimodal fusion features.

6. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 5, characterized in that, The frequency-aware cross-modal spatial interaction features are refined using single-modal features to obtain features with enhanced discriminative power. The corresponding relationship in this process is as follows: ； ;in, RGB features that enhance discriminative power Deep features that enhance discriminative power and This represents two 3×3 depthwise separable convolutional layers. Let SiLU represent the activation function. In the step of fusing cross-modal state spaces of features to enhance discriminative power using a structured state-space model to obtain multimodal correlated features, the corresponding relationship in the process is as follows: ； ； ;in, Indicates a channel-level connection. Represents the state space module. Indicated by depth features RGB interaction features after state-space model interaction processing Indicates using RGB features Deep features after interactive processing using the state-space model These are the correlated RGB features and depth features, respectively. The steps involving feature separation and adaptive enhancement of key feature channels on multimodal associated features to obtain primary multimodal fusion features are described below. ； ； ;in, This indicates that all draws are pooled. This represents RGB features that have undergone adaptive enhancement of key feature channels. This represents deep features that have undergone adaptive enhancement of key feature channels. This represents the multimodal fusion feature of the i-th stage. This represents a stacked operation consisting of a linear layer, a SiLU activation function, another linear layer, and a Sigmoid activation function. This represents element-wise multiplication. In the step of performing residual operations on the initial multimodal fusion features to obtain the final fusion features, the corresponding relationship in the process is as follows: ;in, This represents the hierarchical features of the i-th stage. This indicates the element-wise summation operation.

7. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 6, characterized in that, In step 4, the final multimodal fusion features are upsampled sequentially and then added to the features of the next layer to obtain multi-scale aggregated features. A structured state-space model is used to model long-range dependencies, and residual enhancement is combined to preserve the original information, resulting in features with global structural coherence and local detail retention. Then, a channel attention mechanism is used to obtain semantically focused features. The final output of the last layer is processed to obtain the final salient target detection prediction map. Specifically, the steps include: upsampling the final multimodal fusion features of the previous layer and adding them to the final multimodal fusion features of the current layer to obtain multi-scale aggregated features; modeling long-range dependencies using a structured state-space model for the multi-scale aggregated features, and then combining residual enhancement to preserve the original information, resulting in features with global structural coherence and local detail retention; using a channel attention mechanism to obtain semantically focused features for the features with global structural coherence and local detail retention; and processing the final output of the last layer to obtain the final salient target detection prediction map.

8. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 7, characterized in that, The final multimodal fusion feature of the previous layer is upsampled and then added to the final multimodal fusion feature of the current layer to obtain the multi-scale aggregated feature. The corresponding relationship is as follows: ;in, This indicates a 2x upsampling operation. This represents multi-scale aggregated features. By modeling long-range dependencies using a structured state-space model and then combining residual enhancement to preserve the original information, we obtain features with a coherent global structure and retained local details. The corresponding relationship in this process is as follows: ;in, Indicates batch normalization, This indicates that the learnable scaling factor parameter is used to balance the residual contributions. This represents features that are coherent in global structure while preserving local details. In the step of obtaining semantically focused features using a channel attention mechanism on these coherent features, the corresponding relationship is as follows: ;in, Indicates a channel attention block. This indicates that the learnable scaling factor parameter is used to balance the residual contributions. Features that indicate semantic focus.

9. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 8, characterized in that, The method is based on a saliency detection model. In steps 1 to 4, the training method for the saliency detection model includes the following steps: Given a training set, which includes RGB images, depth images, and ground truth labels for salient object detection; reconstructing the RGB images and depth images using the ground truth labels and prediction maps to obtain RGB-distorted reconstructed images and depth-distorted reconstructed images, respectively; constructing an RGB image quality-aware loss using the RGB-distorted reconstructed images and the RGB images; constructing a pixel position-aware loss using the depth-distorted reconstructed images and the depth images; adding the RGB image quality-aware loss and the depth image quality-aware loss to obtain the total SQA loss; constructing a pixel position-aware loss using weighted binary cross-entropy loss and weighted IoU loss, and fusing the total SQA loss with the pixel position-aware loss to obtain the final optimized loss; The final optimization loss is minimized by updating the weights and learning parameters to train the RGB-D image saliency detection model.

10. The saliency detection method for RGB-D images based on frequency decoupling modal interaction according to claim 9, characterized in that, The calculation processes for RGB distortion reconstructed images and depth distortion reconstructed images are represented by the following formulas: ； ； ;in, Represents the actual label, and 1 represents a tensor where all elements are equal to 1. This represents the RGB foreground image extracted from an RGB image. This represents the depth foreground image extracted from the depth image. Represents an RGB background image. Represents a depth background image. This represents a significance prediction plot. This indicates an image reconstructed from RGB distortion. The image represents a depth-distortion reconstructed image; the calculation process for the quality-perceived loss of an RGB image is as follows: ;in, L1 paradigm loss, This represents a constant used to balance the importance of the two types of losses. This indicates the perceived quality loss of RGB images. The structural similarity index loss and pixel location-aware loss are represented by the following relationship: ;in, The depth image quality perceived loss is represented by the following formula; the calculation process for the total SQA loss is as follows: ;in, Let represent the total SQA loss; the final optimization loss corresponds to the following relationship in the process: ;in, This represents the final optimization loss. This represents the pixel position awareness loss, which consists of weighted binary cross-entropy loss and weighted IoU loss.

Images