A denoising method for multi-modal recommendation

Abstract

The application discloses a denoising method for multi-modal recommendation, comprising the following steps: collecting interaction data of users and items and corresponding visual and text multi-modal data; extracting original multi-modal feature vectors through a pre-training model; performing denoising processing on the original features by using a spectrum purification unit to obtain denoised multi-modal feature vectors; constructing a double-channel heterogeneous graph according to the denoised multi-modal feature vectors; processing the double-channel heterogeneous graph and the denoised multi-modal feature vectors by using a modal perception preference unit to obtain user preferences and item feature vectors; and calculating interaction scores according to the user preferences and the item feature vectors to generate a recommendation result. The application removes noise in modal features through spectrum purification, fully excavates multi-modal preferences of users in combination with a double-channel heterogeneous graph and a modal perception preference unit, and significantly improves the accuracy and robustness of multi-modal recommendation.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a denoising method for multimodal recommendation. Background Technology

[0002] The explosive growth of online content has made recommender systems a key tool for alleviating information overload and improving user engagement. Among them, multimodal recommender systems further enhance this function: they integrate multiple data sources—including implicit user-item interactions, textual feedback, and visual content—to capture cross-modal insights that cannot be obtained by unimodal methods. With the help of these complementary signals, multimodal recommender systems significantly outperform traditional methods in both recommendation accuracy and user satisfaction, and have therefore become the core focus of current research.

[0003] Implicit feedback has become a primary training source for recommendation systems due to its low acquisition cost, but its inherent "false positive" interactions (accidental clicks, accidental clicks, etc.) can significantly interfere with model learning. Therefore, "denoising" has become a key direction for reconstructing users' true preferences. Early work relied on robust architectures to indirectly suppress noise, such as reducing noise propagation by simplifying graph convolutions and using self-attention to reduce irrelevant interaction weights. Subsequent research shifted to explicit denoising, proposing adaptive loss, auxiliary tasks, or reinforcement learning mechanisms to dynamically prune or filter high-noise samples during the training phase, and designing dedicated modules such as trainable binary masks, temporal consistency constraints, or sparse gating networks to clean up features. In recent years, to cope with data distribution drift, difficult negative sample mining has been introduced, and memory networks have been used to self-guide noise identification. The rise of diffusion models has also spurred generative denoising approaches. After 2025, research has further expanded to the semantic and spectral levels, opening a new paradigm of signal processing-based denoising. Existing methods mainly focus on complex fusion mechanisms, often neglecting this dynamic heterogeneity and feature contamination, leading to models learning misleading associations rather than true user intent, ultimately impairing prediction accuracy and model generalization ability. Summary of the Invention

[0004] This invention provides a denoising method for multimodal recommendation to overcome the problem that dynamic heterogeneity and feature contamination cause the model to learn misleading associations instead of true user intent, ultimately impairing prediction accuracy and model generalization ability.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A denoising method for multimodal recommendation includes: S1. Define the user set and the item set, and collect the interaction data between users and items; based on the interaction data, obtain the multimodal data of the items that interact with the users; The multimodal data includes: visual modal data and text modal data; S2. Extract features from multimodal data using pre-trained VGG16 and pre-trained Transformer models to obtain the original multimodal feature vectors. S3. Based on the original multimodal feature vector, a denoised multimodal feature vector is obtained through a pre-constructed spectral cleanup unit; S4. Construct a dual-channel heterogeneous graph based on the denoised multimodal feature vectors; The dual-channel heterogeneous graph includes a visual similarity graph and a semantic similarity graph; S5. Based on the dual-channel heterogeneous graph and the denoised multimodal feature vector, user preferences and item feature vectors are obtained through the pre-constructed modality-aware preference unit. S6. Based on user preferences and item feature vectors, obtain the final interaction score to generate the final recommendation result after removing noise interference.

[0006] Furthermore, the original multimodal feature vector includes the original visual feature vector and the original text feature vector; The original visual feature vector is obtained through a pre-trained VGG16 model, and its expression is:

[0007] In the formula, For items The original visual feature vector; For visual modality; For pre-trained VGG16 models; For items Visual modal data; The original text feature vector is obtained through a pre-trained Transformer model, and its expression is:

[0008] In the formula, For items The original text feature vector; For text modality; For pre-trained Transformer models; For items Text modal data.

[0009] Furthermore, the specific execution steps of the pre-built spectrum cleanup unit are as follows: S31. Based on the original multimodal feature vector, a hidden representation is obtained by projection through a shared multilayer perceptron; the expression for projection through the shared multilayer perceptron is:

[0010] In the formula, To hide the representation; The weight matrix is ​​shared across all modalities; This is the original multimodal feature vector; A bias term shared across all modalities; Modal; S32. The hidden representation is transformed to the frequency domain using a Fast Fourier Transform to obtain a frequency domain signal; the expression for transforming the hidden representation to the frequency domain is:

[0011] In the formula, It is a frequency domain signal; For Fast Fourier Transform; S33. By using a fixed Gaussian low-pass filter, high-frequency noise in the frequency domain signal is suppressed to obtain a purified frequency domain signal; the expression for suppressing high-frequency noise in the frequency domain signal is:

[0012] In the formula, For purified frequency domain signals; It is a fixed Gaussian low-pass filter; The weights of the filter on the frequency components; It is the frequency index; The standard deviation of the Gaussian function; S34. The purified frequency domain signal is transformed using inverse fast Fourier transform to obtain a denoised multimodal feature vector; the expression for transforming the purified frequency domain signal is as follows:

[0013] In the formula, This is a denoised multimodal feature vector; This is the inverse fast Fourier transform.

[0014] Furthermore, the specific steps for constructing a dual-channel heterogeneous graph include: S41. Based on the denoised multimodal feature vectors, calculate the similarity score between any two items using exponentially enhanced cosine similarity, as expressed in the following expression:

[0015] In the formula, For any two items and Similarity score between them; For temperature parameters; For items Denoising multimodal feature vectors; For items Denoising multimodal feature vectors; S42. Calculate the graph sparsity of the modes using a dynamic sparsity strategy. The expression is:

[0016] In the formula, For modality The sparsity of the graph structure; Based on sparsity; For the current mode Signal-to-noise ratio; This represents the maximum signal-to-noise ratio among all modes. For the signal-to-noise ratio of all modes; S43. Based on the similarity score between any two items and the sparsity of the modality's graph structure, a Top-K filtering operation is performed to obtain the filtered similarity score; the expression for the Top-K filtering operation is:

[0017] In the formula, Similarity score for filtering; For items In modality The set of similarity scores with all items below; From Select the highest scorer Each similarity score; S44. Based on the filtered similarity scores, construct a sparse adjacency matrix, expressed as:

[0018] In the formula, It is a sparse adjacency matrix; A collection of items; S45. Based on the sparse adjacency matrix, through graph regularization, a two-channel heterogeneous graph is obtained, expressed as:

[0019] In the formula, This is a dual-channel heterogeneous diagram; For modality The degree matrix; For hyperparameters; It is the identity matrix; Specifically, visual and textual modalities are substituted into the two-channel heterogeneous graph. Obtain a visual similarity map With semantic similarity graph .

[0020] Furthermore, the specific execution steps of the pre-built modality-aware preference unit include: S51. Based on the denoised multimodal feature vector, the initial embedding features of the graph network are obtained through the Sigmoid gated activation function, and the expression is:

[0021] In the formula, Initialize the embedding features for the graph network; For items The collaborative signal generated by the basic ID embedding; It is a Sigmoid-gated activation function; and These are learnable parameters in the gating mechanism; S52. Based on the initial embedding features of the graph network and the dual-channel heterogeneous graph, the embedding features of each layer of the graph convolutional network are obtained through an L-layer graph convolutional network. The expression is as follows:

[0022] In the formula, For graph convolutional networks Layer embedding features; For graph convolutional networks Layer embedding features; For the index of the graph convolutional network layer, and ; S53. Calculate the mean of the embedded features of each layer of the graph convolutional network to obtain the graph-enhanced features, expressed as:

[0023] In the formula, For the enhanced features of the image; S54. The enhanced features of each modality are fused to obtain the item feature vector, expressed as:

[0024] In the formula, For item feature vectors; For modality Adaptive weights; S55. Based on the item feature vector, user preferences are obtained through an attention mechanism, expressed as follows:

[0025] In the formula, For user preferences; In order to connect with users A collection of interacted items; For users The collaborative signal generated by the basic ID embedding; For the attention mechanism query vector; and All are related to users Among the interacted items, The current target item; Items used as a global reference; For the current target item Feature vector; Items for global reference Eigenvectors.

[0026] Furthermore, the specific steps to obtain the final interaction score include: S61. Based on user preferences and item feature vectors, respectively, compare them with user... The basic ID embedding generates collaborative signals and items The collaborative signals generated by the basic ID embedding are fused to obtain user splicing features and item splicing features, expressed as:

[0027]

[0028] In the formula, For users, splice features; Features of item assembly; S62. Based on the user's splicing features and the item's splicing features, the final interaction score is obtained through a multilayer perceptron, expressed as:

[0029] In the formula, The final interaction score; It is a multilayer perceptron.

[0030] Beneficial Effects: This invention provides a denoising method for multimodal recommendation. By introducing a spectrum purification unit, it can efficiently filter high-frequency noise in the frequency domain, addressing issues such as user misoperation, cluttered backgrounds in product images, and lengthy text descriptions. It retains key features such as the product's style, material, and core functions. Even with complex backgrounds in the input image or titles containing irrelevant words, the extracted features accurately focus on the product itself, significantly improving the robustness of the recommendation system in environments with varying raw data quality. This results in a cleaner and more representative denoised multimodal feature vector, providing high-quality input for subsequent recommendation tasks. By constructing a dual-channel heterogeneous graph that includes visual similarity and semantic similarity, items can be connected through visual similarity ("looks like") and semantic similarity ("describes like"), thereby building accurate neighbor relationships, effectively alleviating the data sparsity problem, and improving the coverage and novelty of recommendations; and enhancing the robustness of the graph structure to noise, effectively preventing the propagation of noise and over-smoothing in the modal graph. By using modal-aware preference units, the distribution of user interests across both visual and semantic channels can be comprehensively considered, thereby more accurately and comprehensively characterizing users' personalized preferences. This achieves fine-grained characterization of user interests and precise alignment with cross-modal semantics, significantly improving recommendation accuracy. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic flowchart of the noise reduction method of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] This embodiment provides a denoising method for multimodal recommendation, such as... Figure 1 As shown, it includes: S1. Define the user set and the item set, and collect the interaction data between users and items; based on the interaction data, obtain the multimodal data of the items that interact with the users; The multimodal data includes: visual modal data and text modal data; S2. Extract features from multimodal data using pre-trained VGG16 and pre-trained Transformer models to obtain the original multimodal feature vectors. S3. Based on the original multimodal feature vector, a denoised multimodal feature vector is obtained through a pre-constructed spectral cleanup unit; S4. Construct a dual-channel heterogeneous graph based on the denoised multimodal feature vectors; The dual-channel heterogeneous graph includes a visual similarity graph and a semantic similarity graph; S5. Based on the dual-channel heterogeneous graph and the denoised multimodal feature vector, user preferences and item feature vectors are obtained through the pre-constructed modality-aware preference unit. S6. Based on user preferences and item feature vectors, obtain the final interaction score to generate the final recommendation result after removing noise interference.

[0035] Specifically, the user set is The collection of items is as follows: ; The user-item interaction data is recorded by an implicit feedback matrix, which is: When users With items When an interactive action (click, purchase, favorite) occurs, the elements in the implicit feedback matrix... ;otherwise, ; The visual modal features are the visual content (images) of the item; the text modal features are the text description (title, comments) of the item.

[0036] Preferably, the original multimodal feature vector includes the original visual feature vector and the original text feature vector; The original visual feature vector is obtained through a pre-trained VGG16 model, and its expression is:

[0037] In the formula, For items The original visual feature vector; For visual modality; For pre-trained VGG16 models; For items Visual modal data; The original text feature vector is obtained through a pre-trained Transformer model, and its expression is:

[0038] In the formula, For items The original text feature vector; For text modality; For pre-trained Transformer models; For items Text modal data.

[0039] Preferably, the specific execution steps of the pre-constructed spectrum purification unit are as follows: S31. Based on the original multimodal feature vector, a hidden representation is obtained by projection through a shared multilayer perceptron; the expression for projection through the shared multilayer perceptron is:

[0040] In the formula, To hide the representation; The weight matrix is ​​shared across all modalities; This is the original multimodal feature vector; A bias term shared across all modalities; Modal; S32. The hidden representation is transformed to the frequency domain using a Fast Fourier Transform to obtain a frequency domain signal; the expression for transforming the hidden representation to the frequency domain is:

[0041] In the formula, It is a frequency domain signal; For Fast Fourier Transform; S33. By using a fixed Gaussian low-pass filter, high-frequency noise in the frequency domain signal is suppressed to obtain a purified frequency domain signal; the expression for suppressing high-frequency noise in the frequency domain signal is:

[0042] In the formula, For purified frequency domain signals; It is a fixed Gaussian low-pass filter; The weights of the filter on the frequency components; It is the frequency index; The standard deviation of the Gaussian function; For element-wise multiplication; S34. The purified frequency domain signal is transformed using inverse fast Fourier transform to obtain a denoised multimodal feature vector; the expression for transforming the purified frequency domain signal is as follows:

[0043] In the formula, This is a denoised multimodal feature vector; This is the inverse fast Fourier transform.

[0044] Preferably, the specific steps for constructing a dual-channel heterogeneous graph include: S41. Based on the denoised multimodal feature vectors, calculate the similarity score between any two items using exponentially enhanced cosine similarity, as expressed in the following expression:

[0045] In the formula, For any two items and Similarity score between them; For temperature parameters; For items Denoising multimodal feature vectors; For items Denoising multimodal feature vectors; S42. Calculate the graph sparsity of the modes using a dynamic sparsity strategy. The expression is:

[0046] In the formula, For modality The sparsity of the graph structure; Based on sparsity; For the current mode Signal-to-noise ratio; This represents the maximum signal-to-noise ratio among all modes. For the signal-to-noise ratio of all modes; S43. Based on the similarity score between any two items and the sparsity of the modality's graph structure, a Top-K filtering operation is performed to obtain the filtered similarity score; the expression for the Top-K filtering operation is:

[0047] In the formula, Similarity score for filtering; For items In modality The set of similarity scores with all items below; From Select the highest scorer Each similarity score; S44. Based on the filtered similarity scores, construct a sparse adjacency matrix, expressed as:

[0048] In the formula, It is a sparse adjacency matrix; A collection of items; S45. Based on the sparse adjacency matrix, through graph regularization, a two-channel heterogeneous graph is obtained, expressed as:

[0049] In the formula, This is a dual-channel heterogeneous diagram; For modality The degree matrix; For hyperparameters; It is the identity matrix; Specifically, visual and textual modalities are substituted into the two-channel heterogeneous graph. Obtain a visual similarity map With semantic similarity graph .

[0050] Preferably, the specific execution steps of the pre-constructed modality-aware preference unit include: S51. Based on the denoised multimodal feature vector, the initial embedding features of the graph network are obtained through the Sigmoid gated activation function, and the expression is:

[0051] In the formula, Initialize the embedding features for the graph network; For items The collaborative signal generated by the basic ID embedding; It is a Sigmoid-gated activation function; and These are learnable parameters in the gating mechanism; S52. Based on the initial embedding features of the graph network and the dual-channel heterogeneous graph, the embedding features of each layer of the graph convolutional network are obtained through an L-layer graph convolutional network. The expression is as follows:

[0052] In the formula, For graph convolutional networks Layer embedding features; For graph convolutional networks Layer embedding features; For the index of the graph convolutional network layer, and , among which, when When it is 1, This refers to the initial embedding features of the graph network; S53. Calculate the mean of the embedded features of each layer of the graph convolutional network to obtain the graph-enhanced features, expressed as:

[0053] In the formula, For the enhanced features of the image; S54. The enhanced features of each modality are fused to obtain the item feature vector, expressed as:

[0054] In the formula, For item feature vectors; For modality Adaptive weights; S55. Based on the item feature vector, user preferences are obtained through an attention mechanism, expressed as follows:

[0055] In the formula, For user preferences; In order to connect with users A collection of interacted items; For users The collaborative signal generated by the basic ID embedding; For the attention mechanism query vector; and All are related to users Among the interacted items, The current target item; Items used as a global reference; For the current target item Feature vector; Items for global reference Eigenvectors.

[0056] In this embodiment, to ensure that the preference representations across different modalities remain consistent and aligned, an InfoNCE contrastive loss is introduced to ensure that the user's behavioral preference representations are consistent. Modal feature representation compared to other users A representation that more closely resembles its own modal features; The expression for the InfoNCE contrast loss is:

[0057] In the formula, For InfoNCE contrast loss; For users' index; For items , where the item's feature vector For a collection of items Index of items in the text; This is a temperature hyperparameter used to control the sharpness of the distribution.

[0058] Preferably, the specific steps to obtain the final interaction score include: S61. Based on user preferences and item feature vectors, respectively, compare them with user... The basic ID embedding generates collaborative signals and items The collaborative signals generated by the basic ID embedding are fused to obtain user splicing features and item splicing features, expressed as:

[0059]

[0060] In the formula, For users, splice features; Features of item assembly; S62. Based on the user's splicing features and the item's splicing features, the final interaction score is obtained through a multilayer perceptron, expressed as:

[0061] In the formula, The final interaction score; It is a multilayer perceptron.

[0062] In this embodiment, a composite loss function is introduced for joint optimization; the expression of the composite loss function is:

[0063] In the formula, It is a composite loss function; The Bayesian personalized ranking loss is used to rank the final interaction scores. The weights of the learning and regularization tasks relative to the main recommendation objective; The InfoNCE contrast loss is used to normalize the representation space; L2 regularization is used to prevent overfitting; The Bayesian personalized ranking loss and L2 regularization mentioned above are existing technologies and will not be described in detail here.

[0064] In this embodiment, the proposed denoising method is applicable to multimedia content recommendation scenarios, such as product recommendations on e-commerce platforms (e.g., Taobao, Amazon) or content recommendations on short video platforms (e.g., Douyin, TikTok). In these scenarios, items typically contain rich multimodal data (e.g., product main images, detailed description text), but the original data often contains a large amount of noise unrelated to user interests (e.g., image background interference, text marketing terms). The denoising method in this embodiment can remove relevant noise, and the generated recommendation results are more in line with the user's actual needs and aesthetic preferences.

[0065] In this embodiment, an experimental method comparing the denoising method with other algorithms is used to evaluate the effectiveness of the denoising method in this embodiment. The data for the comparative experiment were selected from the Amazon Reviews collection, using widely used benchmark datasets for evaluation. A 5-core filter was applied to each dataset, retaining only users and products with at least five interactions. The experimental data is shown in Table 1. Table 1 Experimental Dataset

[0066] For visual modal features, a 4096-dimensional visual modal feature vector is extracted from the product image using a pre-trained VGG16 model. For text modal features, a 384-dimensional text modal feature vector is extracted from the product text data (name, title, description, category) using a pre-trained Transformer model; The experimental environment for the comparative experiment is as follows: Central Processing Unit (CPU): Intel Core i7-12700K; Graphics Processing Unit (GPU): NVIDIA RTX A5000 24 GB; Memory: 32 GB DDR4; Operating system: Ubuntu 20.04; Development language: Python 3.8; Machine learning framework: PyTorch 1.13; Parallel computing platform: CUDA 11.7; The comparative algorithms used in the comparative experiments include: Bayesian Personalized Ranking (BPR), Lightweight Graph Convolutional Network (LightGCN), Visual Enhancement Bayesian Personalized Ranking (VBPR), Multimodal Graph Convolutional Network (MMGCN), Graph Denoising Convolutional Network (GRCN), Dual-channel Graph Neural Network (DualGNN), Self-Supervised Lightweight Multimodal Recommendation Algorithm (SLMRec), Third-order Bayesian Multimodal Matching Algorithm (BM3), Multi-Relationship Graph Convolutional Network (MGCN), and Feature Re-enhanced Denoising Multimodal Recommendation Algorithm (FREEDOM). The experimental results of the comparative experiment are shown in Tables 2 and 3: Table 2 Experimental results of the clothing dataset

[0067] Table 3 Experimental results of the electrical appliance dataset

[0068] Based on the experimental results, the denoising method in this embodiment is stable and significantly outperforms other comparative algorithms on all datasets and all metrics; among them, the improvement in Recall@20 and NDCG@20 is the most outstanding. In the experimental results, MMGCN, which uses direct feature fusion, performed poorly, confirming the impact of noise in the original multimodal data on the model. Even though more advanced methods such as GRCN, MGCN, and FREEDOM achieved better results through indirect fusion or explicit denoising, they still did not fully explore the complementary information between different modalities. In contrast, the denoising method in this embodiment effectively suppressed noise through domain fusion and dynamic filtering techniques.

[0069] The present invention has the following beneficial effects: This invention presents a denoising method for multimodal recommendation. By introducing a spectrum purification unit, it can efficiently filter out high-frequency noise in the frequency domain, addressing issues such as user errors, cluttered backgrounds in product images, and lengthy text descriptions. It retains key features such as the product's style, material, and core functions. Even with complex backgrounds in the input image or titles containing irrelevant words, the extracted features accurately focus on the product itself, significantly improving the robustness of the recommendation system in environments with varying raw data quality. This results in a cleaner and more representative denoised multimodal feature vector, providing high-quality input for subsequent recommendation tasks. By constructing a dual-channel heterogeneous graph that includes visual similarity and semantic similarity, items can be connected through visual similarity ("looks like") and semantic similarity ("describes like"), thereby building accurate neighbor relationships, effectively alleviating the data sparsity problem, and improving the coverage and novelty of recommendations; and enhancing the robustness of the graph structure to noise, effectively preventing the propagation of noise and over-smoothing in the modal graph. By using modal-aware preference units, the distribution of user interests across both visual and semantic channels can be comprehensively considered, thereby more accurately and comprehensively characterizing users' personalized preferences. This achieves fine-grained characterization of user interests and precise alignment with cross-modal semantics, significantly improving recommendation accuracy.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A denoising method for multimodal recommendation, characterized in that, include: S1. Define the user set and the item set, and collect user interaction data with items; Based on the interaction data, obtain multimodal data of the items that the user interacts with; The multimodal data includes: visual modal data and text modal data; S2. Extract features from multimodal data using pre-trained VGG16 and pre-trained Transformer models to obtain the original multimodal feature vectors. S3. Based on the original multimodal feature vector, a denoised multimodal feature vector is obtained through a pre-constructed spectral cleanup unit; S4. Construct a dual-channel heterogeneous graph based on the denoised multimodal feature vectors; The dual-channel heterogeneous graph includes a visual similarity graph and a semantic similarity graph; S5. Based on the dual-channel heterogeneous graph and the denoised multimodal feature vector, user preferences and item feature vectors are obtained through the pre-constructed modality-aware preference unit. S6. Based on user preferences and item feature vectors, obtain the final interaction score to generate the final recommendation result after removing noise interference.

2. The denoising method for multimodal recommendation according to claim 1, characterized in that, The original multimodal feature vector includes the original visual feature vector and the original text feature vector; The original visual feature vector is obtained through a pre-trained VGG16 model, and its expression is: In the formula, For items The original visual feature vector; For visual modality; For pre-trained VGG16 models; For items Visual modal data; The original text feature vector is obtained through a pre-trained Transformer model, and its expression is: In the formula, For items The original text feature vector; For text modality; For pre-trained Transformer models; For items Text modal data.

3. The denoising method for multimodal recommendation according to claim 2, characterized in that, The specific execution steps of the pre-constructed spectrum purification unit are as follows: S31. Based on the original multimodal feature vector, a hidden representation is obtained by projection through a shared multilayer perceptron; the expression for projection through the shared multilayer perceptron is: In the formula, To hide the representation; The weight matrix is ​​shared across all modalities; This is the original multimodal feature vector; A bias term shared across all modalities; Modal; S32. The hidden representation is transformed to the frequency domain using a Fast Fourier Transform to obtain a frequency domain signal; the expression for transforming the hidden representation to the frequency domain is: In the formula, It is a frequency domain signal; For Fast Fourier Transform; S33. By using a fixed Gaussian low-pass filter, high-frequency noise in the frequency domain signal is suppressed to obtain a purified frequency domain signal; the expression for suppressing high-frequency noise in the frequency domain signal is: In the formula, For purified frequency domain signals; It is a fixed Gaussian low-pass filter; The weights of the filter on the frequency components; It is the frequency index; The standard deviation of the Gaussian function; S34. The purified frequency domain signal is transformed using inverse fast Fourier transform to obtain a denoised multimodal feature vector; the expression for transforming the purified frequency domain signal is as follows: In the formula, This is a denoised multimodal feature vector; This is the inverse fast Fourier transform.

4. The denoising method for multimodal recommendation according to claim 3, characterized in that, The specific steps for constructing a dual-channel heterogeneous graph include: S41. Based on the denoised multimodal feature vectors, calculate the similarity score between any two items using exponentially enhanced cosine similarity, as expressed in the following expression: In the formula, For any two items and Similarity score between them; For temperature parameters; For items Denoising multimodal feature vectors; For items Denoising multimodal feature vectors; S42. Calculate the graph sparsity of the modes using a dynamic sparsity strategy. The expression is: In the formula, For modality The sparsity of the graph structure; Based on sparsity; For the current mode Signal-to-noise ratio; This represents the maximum signal-to-noise ratio among all modes. For the signal-to-noise ratio of all modes; S43. Based on the similarity score between any two items and the sparsity of the modality's graph structure, a Top-K filtering operation is performed to obtain the filtered similarity score; the expression for the Top-K filtering operation is: In the formula, Similarity score for filtering; For items In modality The set of similarity scores with all items below; From Select the highest scorer Each similarity score; S44. Based on the filtered similarity scores, construct a sparse adjacency matrix, expressed as: In the formula, It is a sparse adjacency matrix; A collection of items; S45. Based on the sparse adjacency matrix, through graph regularization, a two-channel heterogeneous graph is obtained, expressed as: In the formula, This is a dual-channel heterogeneous diagram; For modality The degree matrix; For hyperparameters; It is the identity matrix; Specifically, visual and textual modalities are substituted into the two-channel heterogeneous graph. Obtain a visual similarity map With semantic similarity graph .

5. A denoising method for multimodal recommendation according to claim 4, characterized in that, The specific execution steps of the pre-built modality-aware preference unit include: S51. Based on the denoised multimodal feature vector, the initial embedding features of the graph network are obtained through the Sigmoid gated activation function, and the expression is: In the formula, Initialize the embedding features for the graph network; For items The collaborative signal generated by the basic ID embedding; It is a Sigmoid-gated activation function; and These are learnable parameters in the gating mechanism; S52. Based on the initial embedding features of the graph network and the dual-channel heterogeneous graph, the embedding features of each layer of the graph convolutional network are obtained through an L-layer graph convolutional network. The expression is as follows: In the formula, For graph convolutional networks Layer embedding features; For graph convolutional networks Layer embedding features; For the index of the graph convolutional network layer, and ; S53. Calculate the mean of the embedded features of each layer of the graph convolutional network to obtain the graph-enhanced features, expressed as: In the formula, For the enhanced features of the image; S54. The enhanced features of each modality are fused to obtain the item feature vector, expressed as: In the formula, For item feature vectors; For modality Adaptive weights; S55. Based on the item feature vector, user preferences are obtained through an attention mechanism, expressed as follows: In the formula, For user preferences; In order to connect with users A collection of interacted items; For users The collaborative signal generated by the basic ID embedding; For the attention mechanism query vector; and All are related to users Among the interacted items, The current target item; Items used as a global reference; For the current target item Feature vector; Items for global reference Eigenvectors.

6. A denoising method for multimodal recommendation according to claim 5, characterized in that, The specific steps to obtain the final interaction score include: S61. Based on user preferences and item feature vectors, respectively, compare them with user... The basic ID embedding generates collaborative signals and items The collaborative signals generated by the basic ID embedding are fused to obtain user splicing features and item splicing features, expressed as: In the formula, For users, splice features; Features of item assembly; S62. Based on the user's splicing features and the item's splicing features, the final interaction score is obtained through a multilayer perceptron, expressed as: In the formula, The final interaction score; It is a multilayer perceptron.

Images