Multi-modal recommendation method for anti-noise article and interactive behavior modeling

Abstract

The invention discloses an anti-noise article and interactive behavior modeling multi-modal recommendation method, which belongs to the technical field of personalized recommendation, and comprises the following steps: (1) homomorphic composition learning: respectively constructing an article-article homomorphic composition and a user-user homomorphic composition which are synergistically enhanced, processing original modal features of an article and modal perception preference representation of a user, and learning the article-article homomorphic composition and the user-user homomorphic composition; preliminary object embedding and user embedding are obtained; (2) modal attention optimization: introducing an intra-modal attention module and a cross-modal attention module after homomorphic composition learning, and processing the preliminary article embedding to obtain refined article representation; (3) heterogeneous graph relation learning; (4) performing dynamic multi-view contrast learning; and (5) model training. According to the method, more accurate recommended article modeling is realized through explicit modeling of cooperative signals among articles, optimization of multi-modal feature denoising and complementation and fine-grained distinguishing of user-article interaction importance, and the personalized recommendation effect is improved.

Description

Technical Field

[0001] This invention relates to the field of personalized recommendation technology, specifically a multimodal recommendation method that models noise-resistant items and interactive behaviors. Background Technology

[0002] With the widespread adoption of mobile internet and smart devices, the amount of goods, information, and services users encounter in the online environment has increased exponentially. While this rapid expansion of information sources brings users more choices, it also makes it difficult for them to quickly sift through the massive amount of content to find what truly meets their needs within a limited timeframe. To alleviate users' "choice paralysis" and information overload, personalized recommendation technology has emerged. This technology intelligently filters and sorts massive amounts of candidate content by mining historical interaction records, contextual information, and potential interests between users and items. This allows for the automatic delivery of results that better match the user's interests without requiring any additional user interaction, effectively improving the efficiency and accuracy of information retrieval.

[0003] In practical applications, personalized recommendations have been widely deployed in e-commerce platforms, social media, and content streaming media. By analyzing multi-source data such as users' browsing and purchasing behavior, social relationships and interaction records, and viewing or listening history, the system dynamically generates personalized product lists, content feeds, and multimedia resources for users. This not only significantly improves user experience and stickiness but also helps businesses optimize resource allocation and advertising, enhancing market competitiveness and business revenue.

[0004] In recent years, many studies have utilized graph neural networks to characterize higher-order relationships between users and items, and further constructed item-item semantic graphs and user-user isomorphic graphs to aid representation learning on heterogeneous graphs. However, existing techniques still have the following shortcomings: 1. Existing item-item semantic graphs typically connect semantically similar items based solely on content similarity, lacking explicit modeling of collaborative signals between items, which results in an inability to fully capture the potential associations between items arising from shared user interactions; 2. The original multimodal information of items often contains a lot of noise and redundant content that is irrelevant to the recommendation task. If the modal noise cannot be effectively removed and the complementarity between multimodal features cannot be fully utilized, it will be difficult to characterize the user's true modal preferences in a timely and accurate manner. 3. Most existing work tends to assume that every user-item interaction is a genuine feedback of preference, while ignoring the diverse prototypes of interaction behavior in real-world scenarios (such as "accidental clicks", "comparative browsing", "exploratory clicks" etc.). This coarse-grained approach makes it difficult to distinguish the importance of different interactions, resulting in the model being unable to accurately measure the contribution of a single interaction to the user's long-term preferences, and thus the recommendation results still deviate from the user's actual needs.

[0005] Therefore, those skilled in the art provide a multimodal recommendation method for modeling noise-resistant items and interactive behaviors to solve the problems mentioned in the background art. Summary of the Invention

[0006] The purpose of this invention is to provide a multimodal recommendation method for modeling noise-resistant items and interactive behaviors. By explicitly modeling collaborative signals between items, optimizing multimodal feature denoising and complementarity, and finely distinguishing the importance of user-item interactions, this method achieves more accurate item modeling for recommendations and improves personalized recommendation results, thereby solving the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A multimodal recommendation method based on noise-resistant item and interaction behavior modeling includes the following steps: (1) Isomorphic graph learning: construct collaboratively enhanced item-item isomorphic graphs and user-user isomorphic graphs respectively, process the original modal features of items and the user modal perceived preference representations to obtain preliminary item embeddings and user embeddings; (2) Modal attention optimization: After isomorphic graph learning, intramodal attention module and cross-modal attention module are introduced to process the initial item embedding and obtain refined item representation; (3) Heterogeneous graph relationship learning: Based on the user-item interaction graph, a symmetric matrix is ​​constructed, and the final user and item representations are obtained by multi-layer graph convolution operation and averaging of the embeddings of each layer; (4) Dynamic multi-view comparison learning: Multiple sets of complementary edge weights are generated through a learnable view generator, and the final edge weights are obtained through an adaptive multi-view fusion mechanism. A comparison graph is constructed and the embedded representation under the comparison view is obtained through graph convolution. (5) Model training: The overall objective function is constructed by combining the Bayesian Personalized Ranking (BPR) loss, the multimodal alignment loss, the dynamic multi-view contrast loss and L2 regularization, and the model parameters are trained to complete the joint modeling of recommended items and user preferences.

[0008] As a further aspect of the present invention: the process of constructing the collaboratively enhanced item-item isomorphism graph in step (1) includes: (101): For each modality, calculate the cosine similarity between the original feature vectors of any two items in that modality to obtain the modality-specific similarity; (102): For each item, select the Top-K most similar items based on the similarity vector, use the corresponding similarity values ​​as edge weights, construct a modality-specific item-item adjacency matrix, and normalize the adjacency matrix. (103): Weighted summation and aggregation of normalized adjacency matrices of different modalities are performed to construct an item-item semantic graph, wherein the importance weight of visual modalities in semantic graph construction is adjusted by hyperparameters; (104): Construct an item co-occurrence matrix based on the number of users who interact with two items simultaneously; (105): The co-occurrence matrix of items and the semantic graph of items are weighted and fused according to preset weights to obtain a co-enhanced isomorphic graph of items.

[0009] As a further solution of the present invention: the feature processing process of item isomorphism graph learning in step (1) is as follows: first, the original modal features of the items are linearly transformed and mapped to the same vector space as the ID embedding; then, the ID embedding, the linearly transformed visual features, and the linearly transformed text features are concatenated, and the concatenated vector is then used for graph propagation learning on the collaboratively enhanced item-item isomorphism graph to obtain the preliminary item embedding representation.

[0010] As a further solution of the present invention: the processing of the intramodal attention module in step (2) is as follows: the item modal representation after isomorphic graph learning is transformed by three learnable parameter matrices, the weight of each feature dimension is calculated by using the softmax normalization function, and the item modal features are reweighted; then residual connection and layer normalization operation are introduced to obtain the item representation after intramodal attention optimization.

[0011] As a further solution of the present invention: the processing procedure of the cross-modal attention module in step (2) is as follows: cross-modal attention is applied to the item representations of the visual modality and the text modality respectively; residual connection and layer normalization operations are applied to the attention calculation results to obtain the item representations that fuse cross-modal complementary information.

[0012] As a further aspect of the present invention: the learnable view generator in step (4) includes three complementary view generation modules, specifically: Graph structure view generation module: Using the user representation and item representation obtained from heterogeneous graph learning, the edge weights of the graph structure view are calculated by inputting them into the sigmoid function after element-wise multiplication. Prototype-aware view generation module: Represent each user-item interaction as a weighted combination of multiple potential behavior prototypes. Through learnable prototype latent variables, combined with softmax normalization, the weight distribution of the interaction is calculated, and then the edge weights of the prototype-aware view are obtained through element-wise multiplication and the sigmoid function. Graph Attention View Generation Module: Employs a lightweight attention mechanism, processing the user-item interaction matrix through learnable attention vectors, assigning personalized weights to each interaction, and obtaining the edge weights of the graph attention view.

[0013] As a further aspect of the present invention: the processing procedure of the adaptive multi-view fusion mechanism in step (4) includes: (401): The edge weight matrix of each view is transformed by a shared learnable function, which is achieved by a linear transformation and a hyperbolic tangent activation function; (402): Softmax normalize the edge weights of each transformed view to obtain the attention weights of each view. Based on these weights, the edge weights of all views are weighted and summed to obtain the shared edge weights of the views. (403): The unique edge weights of each view are obtained by subtracting the shared edge weights from the original edge weights of each view; (404): The shared edge weights of the views are summed and combined with the unique edge weights of all views to obtain the final merged edge weights.

[0014] As a further aspect of the present invention: the specific process of learning heterogeneous graph relationships in step (3) is as follows: The item representation optimized by modal attention is concatenated with the user representation learned from the user isomorphic graph, and used as the initial embedding of the heteromorphic graph; A symmetric matrix is ​​constructed based on the user-item interaction matrix. A convolution update rule is constructed using the diagonal form of the degree matrix of this symmetric matrix and the symmetric matrix itself. Multi-layer graph convolution operation is then performed on the initial embedding. The embeddings of each layer from layer 0 to layer L are averaged to obtain the final user and item representations after heterogeneous graph learning.

[0015] As a further aspect of the present invention: the process of constructing the comparison diagram in step (4) is as follows: Based on the edge index set of user-item interaction and the final edge weight obtained by adaptive multi-view fusion, a corresponding sparse adjacency matrix is ​​constructed according to the predefined tensor shape. This sparse adjacency matrix is ​​the completed comparison graph. Perform graph convolution on the comparison graph to output embedded representations of users and items in the comparison view.

[0016] As a further aspect of the present invention: the overall objective function in step (5) is constructed as follows: Based on Bayesian Personalized Ranking (BPR) loss, the BPR loss is obtained by applying a sigmoid function to the difference in predicted ratings between observed user-item interaction pairs and unobserved interaction pairs in the training set, taking the negative logarithm, and then summing the results. The multimodal alignment loss and dynamic multi-view contrast loss are added to the BPR loss according to preset weights, and then an L2 regularization term (where the regularization strength is controlled by a hyperparameter) is added to obtain the overall objective function used for model parameter training.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention constructs a collaboratively enhanced item-item isomorphism graph, which integrates a semantic graph based on content similarity with a collaborative matrix based on user co-occurrence, and explicitly models the collaborative signals between items. This makes up for the shortcomings of existing technologies that rely solely on semantic similarity, and can capture more comprehensive relationships between items.

[0018] 2. This invention introduces intra-modal attention and cross-modal attention modules. Through feature reweighting, residual connections and layer normalization, it effectively suppresses noise components in multimodal features, while strengthening complementary information between different modalities, thereby improving the discriminativeness and reliability of item representation.

[0019] 3. This invention designs a learnable multi-view generator and an adaptive fusion mechanism, constructing comparative views from three complementary perspectives: global structure, behavioral prototype, and interaction importance. This enables fine-grained modeling of user-item interaction behavior, distinguishes the true preference contributions of different interactions, and reduces interference from noisy interactions.

[0020] 4. This invention combines BPR loss, multimodal alignment loss, and multi-view contrast loss for model optimization, ensuring the rationality of recommendation ranking, the consistency of modal features, and the robustness of contrast representation, thereby comprehensively improving the accuracy and generalization ability of personalized recommendations. Attached Figure Description

[0021] Figure 1 A flowchart for a multimodal recommendation method that models noise-resistant items and interactive behaviors. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] As mentioned in the background section of this application, research has found that existing item-item semantic graphs typically connect semantically similar items based solely on content similarity, lacking explicit modeling of collaborative signals between items. This results in an inability to fully capture the potential associations between items arising from shared user interactions, thus presenting certain shortcomings.

[0024] To address the aforementioned shortcomings, this application discloses a multimodal recommendation method for modeling noise-resistant items and interactive behaviors. By explicitly modeling collaborative signals between items, optimizing multimodal feature denoising and complementarity, and finely distinguishing the importance of user-item interactions, this method achieves more accurate item modeling for recommendations and improves personalized recommendation performance.

[0025] The following will describe in detail, with reference to the accompanying drawings, how the solution of this application solves the above-mentioned technical problems.

[0026] Please see Figure 1 In this embodiment of the invention, a multimodal recommendation method for modeling noise-resistant items and interactive behaviors includes the following steps: (1) Isomorphic graph learning: construct collaboratively enhanced item-item isomorphic graphs and user-user isomorphic graphs respectively, process the original modal features of items and the user modal perceived preference representations to obtain preliminary item embeddings and user embeddings; (2) Modal attention optimization: After isomorphic graph learning, intramodal attention module and cross-modal attention module are introduced to process the initial item embedding and obtain refined item representation; (3) Heterogeneous graph relationship learning: Based on the user-item interaction graph, a symmetric matrix is ​​constructed, and the final user and item representations are obtained by multi-layer graph convolution operation and averaging of the embeddings of each layer; (4) Dynamic multi-view comparison learning: Multiple sets of complementary edge weights are generated through a learnable view generator, and the final edge weights are obtained through an adaptive multi-view fusion mechanism. A comparison graph is constructed and the embedded representation under the comparison view is obtained through graph convolution. (5) Model training: The overall objective function is constructed by combining the Bayesian Personalized Ranking (BPR) loss, the multimodal alignment loss, the dynamic multi-view contrast loss and L2 regularization, and the model parameters are trained to complete the joint modeling of recommended items and user preferences.

[0027] In this embodiment, the process of constructing the collaboratively enhanced item-item isomorphism graph in step (1) includes: (101): For each modality, calculate the cosine similarity between the original feature vectors of any two items in that modality to obtain the modality-specific similarity; specifically: construct a modality-specific item-item similarity map A based on the original features of each modality. m For each mode m, the original feature vectors of item i and item j in that mode are used. and The cosine similarity between them is used to calculate their modality-specific similarity: .

[0028] (102): For each item i, based on the similarity vector Select the top-K most similar items and use their corresponding similarity scores as edge weights to construct a modality-specific item-item adjacency matrix. ,in, Then, the adjacency matrix is ​​normalized: ;in, This represents the corresponding angle matrix.

[0029] (103): Weighted summation and aggregation of normalized adjacency matrices of different modalities are performed to construct item-item semantic graphs. , where hyperparameters Adjusting the importance weights of visual modalities in semantic graph construction; (104): Construct an item co-occurrence matrix C based on the number of users who interact with two items simultaneously; the motivation for this design is that even if item i and item j are not similar at the modal level, as long as they are frequently interacted with by the same group of users, there is a strong collaborative relationship between them. Therefore, the matrix elements C are... ij Defined as the number of users who have interacted with both item i and item j simultaneously: .

[0030] (105): Combine the item co-occurrence matrix C with the item-item semantic graph Weighted summation and fusion are performed according to preset weights; Items with enhanced synergy - Item isomorphism ,in, The weights of the co-occurrence matrix are adjustable hyperparameters.

[0031] In this embodiment, the feature processing process for item isomorphism learning in step (1) is as follows: first, the original modal features of the items are linearly transformed and mapped to the same vector space as the ID embedding, which is formally represented as follows: ,in, d represents the dimension of ID embedding; because the goal of multimodal recommendation is to provide users with more accurate recommendation results, rather than to perform comprehensive semantic analysis of multimodal content, the ID embedding is directly concatenated with the linearly transformed features of each modality: The concatenated vector is then used for graph propagation learning on the collaboratively enhanced item-item isomorphism graph, i.e. This yields a preliminary item embedding representation, where, Let S represent the set of neighbors of item i in the item isomorphism graph S. Similar to the item-side representation, this is based on the user's modality-aware preference representation. Constructing a user-user preference similarity graph This is then fused with the user-user co-occurrence graph to obtain the user-user isomorphism graph. Subsequently, representation learning is performed on this user isomorphism graph to obtain user embeddings that can characterize potential user relationships. .

[0032] In this embodiment, the processing of the intramodal attention module in step (2) is as follows: the item modal representation after isomorphic graph learning is transformed by three learnable parameter matrices, the weights of each feature dimension are calculated using the softmax normalization function, and the item modal features are reweighted; then residual connections and layer normalization operations are introduced to obtain the item representation optimized by intramodal attention. More specifically, since the modal representation of items often contains noise, and the graph propagation process will further amplify this noise, an intramodal attention module and a cross-modal attention module are introduced after isomorphic graph learning to refine the modeling and denoising of item features. Specifically, the intramodal attention module mainly focuses on the reweighting of features within the same modality, and emphasizes features highly relevant to the recommendation task by learning the importance of different dimensions, while suppressing irrelevant or noisy components. In form, this application has: ; in, , , Let be the learnable parameter matrix, and softmax(·) denote the normalization function. Based on this, residual connections and layer normalization are further introduced to improve training stability and generalization ability. The layer normalization operation is denoted as LN(·). The multimodal item representation after updating by the intramodal attention module is as follows: .

[0033] In this embodiment, the processing procedure of the cross-modal attention module in step (2) is as follows: cross-modal attention is applied to the item representations of the visual modality and the text modality respectively; residual connection and layer normalization operations are applied to the attention calculation results to obtain the item representation that integrates cross-modal complementary information. More specifically, the cross-modal attention module is used to model the interaction relationship between different modalities, so that the generated multimodal features can describe the same item from a complementary perspective. More specifically, this application performs bidirectional cross-modal attention on visual and text representations: when updating the visual representation, the text representation is treated as a query and the visual representation is treated as a key / value pair. The attention weight is calculated by the similarity between the query and the key, and then the visual value vector is weighted and converged using this weight to obtain a "text-guided visual enhancement representation"; when updating the text representation, the opposite is true, the visual representation is treated as a query and the text is treated as a key / value pair, and a "visual-guided text enhancement representation" is generated in the same way. This mechanism essentially uses another modality to ask where the current modality should focus, thereby highlighting consistent and useful cross-modal cues and reducing noise and redundant information during aggregation. Finally, residual fusion and layer normalization are performed with the representations within each modality to obtain an item representation that combines self-modal semantics with complementary cross-modal information. This module preserves shared information between modalities while emphasizing the unique features of each modality. Specifically, this application applies cross-modal attention to the visual and textual modalities respectively to capture their cross-modal dependencies: .

[0034] After completing the cross-modal attention calculation, this application maintains consistency with the intra-modal attention part, applying residual connections and layer normalization to its output to further improve training stability and enhance the model's generalization ability. ; By employing intra-modal and cross-modal attention mechanisms, the model effectively suppresses noise in item modal representations and highlights complementary information between different modalities, thereby learning more discriminative and task-relevant item representations. Finally, this application concatenates the refined modal representations with ID embeddings to obtain item representations for subsequent recommendation tasks. .

[0035] In this embodiment, the learnable view generator in step (4) includes three complementary view generation modules, specifically: The graph structure view generation module utilizes user and item representations learned from heterogeneous graphs. After element-wise multiplication, the edge weights of the graph structure view are calculated using the sigmoid function. The motivation for the graph structure view lies in the fact that the structural information contained in the original interaction graph is crucial for suppressing noise and constructing a reliable view. Therefore, this application utilizes representations learned from heterogeneous graphs. Involves edge weight modeling to mitigate the impact of noisy interactions on the generated view. Here, This represents the Hadamard product (element-by-element multiplication). The Sigmoid function is represented; the edge weights of the graph structure view are calculated as follows: .

[0036] The prototype-aware view generation module represents each user-item interaction as a weighted combination of multiple potential behavioral prototypes. It calculates the interaction weight distribution using learnable prototype latent variables and softmax normalization, then obtains the edge weights of the prototype-aware view through element-wise multiplication and the sigmoid function. By further introducing a prototype-aware view to characterize diverse behavioral patterns in user-item interactions, unlike simply assigning a scalar weight to each edge, this module represents each interaction as a weighted combination of multiple potential behavioral prototypes, thus achieving finer-grained view construction. Here, Representing the prototype latent variables, which are learnable parameters; the prototype-aware view is calculated as follows: .

[0037] The graph attention view generation module employs a lightweight attention mechanism. It processes the user-item interaction matrix using learnable attention vectors, assigning personalized weights to each interaction to obtain the edge weights of the graph attention view. Unlike GCN views that rely on a global structure and prototype views that aggregate behavioral patterns, graph attention views focus on modeling the importance of each specific interaction. This application uses a lightweight attention mechanism to assign personalized weights to all interactions. This mechanism highlights key connections highly relevant to current user preferences and suppresses spurious edges caused solely by co-occurring or high-frequency behaviors, resulting in a more focused view. ; Where g is a learnable attention vector and R represents the user-item interaction matrix.

[0038] In this embodiment, based on the aforementioned learnable view generator, this application generates multiple sets of complementary edge weights for each user-item interaction. These edge weights characterize the relationship between the user and the item from different perspectives, such as global topology, behavioral prototype, and instance-level interaction importance. To fuse these heterogeneous but complementary edge weights into a more compact comparative view, this application designs an adaptive multi-view fusion mechanism, enabling the model to flexibly fuse based on the contribution of different views. This mechanism first extracts the shared edge weights among the views, and then combines them with the edge weights specific to each view, thereby achieving joint modeling of multi-source edge information. The processing steps of the adaptive multi-view fusion mechanism in step (4) include: (401): The edge weight matrix of each view is transformed by a shared learnable function, which is implemented by a linear transformation and a hyperbolic tangent activation function; wherein, a simple attention module is designed for the multi-view edge weights, and the view set is given. , Let represent the edge weight matrix of the k-th view, where For each view k, this application first transforms its edge weights using a learnable function: ; Where W and b are learnable parameters shared across all views. This indicates element-wise multiplication.

[0039] (402): Softmax normalize the transformed edge weights of each view to obtain the attention weights of each view. Based on these weights, perform a weighted sum of the edge weights of all views to obtain the shared edge weights of the views; whereby the shared edge weights of the views are defined as: ;in, This represents the shared edge weights obtained after fusing multiple views.

[0040] (403): The unique edge weights of each view are obtained by subtracting the shared edge weights from the original edge weights of each view, i.e. .

[0041] (404): Summing and combining the shared edge weights of the views with the unique edge weights of all views yields the final merged edge weights, i.e. .

[0042] In this embodiment, the specific process of learning heterogeneous graph relationships in step (3) is as follows: The item representation optimized by modal attention is concatenated with the user representation learned from the user isomorphic graph, serving as the initial embedding of the heterogeneous graph; based on the user-item interaction graph, the... The update rule for a convolutional layer is defined as follows: ,in, , This represents the user representation learned from the user isomorphism graph. This represents the item representation after being refined by isomorphic graph convolution and intra-modal and cross-modal attention modules.

[0043] A symmetric matrix is ​​constructed based on the user-item interaction matrix. A convolution update rule is then built using the diagonal form of the degree matrix of this symmetric matrix and the symmetric matrix itself. Multi-layer graph convolution operations are then performed on the initial embedding. Let G be the diagonal form of the degree matrix of graph G. G is a symmetric matrix constructed from the interaction matrix R. .

[0044] Averaging the embeddings from layer 0 to layer L yields the final user and item representations after heterogeneous graph learning: .

[0045] In this embodiment, the process of constructing the comparison chart in step (4) is as follows: Based on the edge index set of user-item interactions and the final edge weights obtained through adaptive multi-view fusion, a corresponding sparse adjacency matrix is ​​constructed according to a predefined tensor shape. This sparse adjacency matrix is ​​the completed comparison graph. Specifically, after obtaining the edge index set... and final border rights Next, this application constructs a comparison chart: ,in, This represents the comparison chart obtained after construction. (Function) It will construct the corresponding sparse adjacency matrix based on the edge index, the learned edge weight, and the predefined tensor shape.

[0046] Perform graph convolution on the comparison graph to output embedded representations of users and items in the comparison view. .

[0047] In this embodiment, the overall objective function in step (5) is constructed as follows: Based on Bayesian Personalized Ranking (BPR) loss, the BPR loss is obtained by applying a sigmoid function to the difference in predicted ratings between observed user-item interaction pairs and unobserved interaction pairs in the training set, taking the negative logarithm, and then summing the results. The multimodal alignment loss and dynamic multi-view contrast loss are added to the BPR loss according to preset weights, and then an L2 regularization term (where the regularization strength is controlled by a hyperparameter) is added to obtain the overall objective function used for model parameter training.

[0048] The multimodal alignment loss specifically addresses this issue: To reduce semantic differences between different modalities, this application performs cross-modal alignment based on multimodal features. Therefore, this application employs InfoNCE loss to constrain the consistency of different modal representations in the semantic space. The multimodal alignment loss for the item side is described below. In this application, the text modality is aligned to the image modality as shown in the following formula: image representations of the same item are considered positive samples, and image representations of other items are considered negative samples: ; Where B is the batch size. Let sim(·) represent the set of items in the current batch, and let cosine similarity function be used. This represents the temperature coefficient in the alignment loss. Similarly, this application can obtain the user-side multimodal alignment loss. Finally, the total multimodal alignment loss is defined as follows: .

[0049] The multi-view contrastive learning loss is specifically defined as follows: Since the contrast graph is constructed based on prototype-aware multi-view edge weights, this application further encourages consistency between the representations learned on the original heterogeneous graph and the representations learned on the contrastive views. This application treats these two embeddings as different perspectives of the same entity and adopts an InfoNCE-style contrastive objective function. For the user side, the multi-view contrastive learning loss is defined as: ; Where B is the batch size. Let sim(·) represent the set of users in the current batch, and let cosine similarity function be used. This refers to the temperature hyperparameter of the multi-view loss. Similarly, this application uses a comparative article representation on the article side. and Constructing the multi-view loss: ; in, For the set of items in the current batch. By bringing the embeddings of the same user (or item) closer together in both views and pushing the embeddings of different users (or items) further apart, this loss helps the prototype-aware contrast map provide more informative supervision. Ultimately, the total multi-view contrast loss is... .

[0050] The recommendation loss is specifically as follows: To optimize recommendation performance, this application adopts Bayesian PersonalizedRanking (BPR) loss as the primary training objective. Its core idea is that for the same user, the model should assign a higher predicted score to observed user-item interactions than to unobserved interactions. The form of the BPR loss is: ; in, Represents the training set, For the Sigmoid function, Let (u, j) be the observed user-item interaction pairs, and (u, j) be the unobserved interaction pairs. Finally, this application jointly optimizes the BPR loss, multimodal alignment loss, and multi-view contrast loss. The overall objective is: ; in, To control The hyperparameter of regularization strength, This represents the set of all learnable model parameters. The weights represent the multimodal alignment loss. The weights represent the dynamic multi-view contrastive learning loss.

[0051] This invention constructs a collaboratively enhanced item-item isomorphism graph, fusing a semantic graph based on content similarity with a collaborative matrix based on user co-occurrence. This explicitly models collaborative signals between items, overcoming the limitations of existing technologies that rely solely on semantic similarity, and capturing more comprehensive relationships between items. Furthermore, this invention introduces intra-modal and cross-modal attention modules, effectively suppressing noise components in multimodal features through feature reweighting, residual connections, and layer normalization. It also strengthens and mines complementary information between different modalities, improving the discriminativeness and reliability of item representations. Simultaneously, this invention designs a learnable multi-view generator and an adaptive fusion mechanism, constructing comparative views from three complementary perspectives: global structure, behavioral prototypes, and interaction importance. This achieves fine-grained modeling of user-item interactions, distinguishing the true preference contributions of different interactions and reducing interference from noisy interactions. Finally, this invention combines BPR loss, multimodal alignment loss, and multi-view contrast loss for model optimization, ensuring the rationality of recommendation ranking, the consistency of modal features, and the robustness of contrast representations, comprehensively improving the accuracy and generalization ability of personalized recommendations.

[0052] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0053] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multimodal recommendation method for modeling noise-resistant items and interactive behaviors, characterized in that, Includes the following steps: (1) Isomorphic graph learning: construct collaboratively enhanced item-item isomorphic graphs and user-user isomorphic graphs respectively, process the original modal features of items and the user modal perceived preference representations to obtain preliminary item embeddings and user embeddings; (2) Modal attention optimization: After isomorphic graph learning, intramodal attention module and cross-modal attention module are introduced to process the initial item embedding and obtain refined item representation; (3) Heterogeneous graph relationship learning: Based on the user-item interaction graph, a symmetric matrix is ​​constructed, and the final user and item representations are obtained by multi-layer graph convolution operation and averaging of the embeddings of each layer; (4) Dynamic multi-view comparison learning: Multiple sets of complementary edge weights are generated through a learnable view generator, and the final edge weights are obtained through an adaptive multi-view fusion mechanism. A comparison graph is constructed and the embedded representation under the comparison view is obtained through graph convolution. (5) Model training: The overall objective function is constructed by combining the Bayesian personalized ranking loss, the multimodal alignment loss, the dynamic multi-view contrast loss and L2 regularization, and the model parameters are trained to complete the joint modeling of recommended items and user preferences.

2. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 1, characterized in that, The process of constructing the collaboratively enhanced item-item isomorphism graph in step (1) includes: (101): For each modality, calculate the cosine similarity between the original feature vectors of any two items in that modality to obtain the modality-specific similarity; (102): For each item, select the Top-K most similar items based on the similarity vector, use the corresponding similarity values ​​as edge weights, construct a modality-specific item-item adjacency matrix, and normalize the adjacency matrix. (103): Weighted summation and aggregation of normalized adjacency matrices of different modalities are performed to construct an item-item semantic graph, wherein the importance weight of visual modalities in semantic graph construction is adjusted by hyperparameters; (104): Construct an item co-occurrence matrix based on the number of users who interact with two items simultaneously; (105): The co-occurrence matrix of items and the semantic graph of items are weighted and fused according to preset weights to obtain a co-enhanced isomorphic graph of items.

3. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 2, characterized in that, The feature processing process of item isomorphism graph learning in step (1) is as follows: First, the original modal features of the items are linearly transformed and mapped to the same vector space as the ID embedding; then, the ID embedding, the linearly transformed visual features, and the linearly transformed text features are concatenated, and the concatenated vector is then used for graph propagation learning on the collaboratively enhanced item-item isomorphism graph to obtain the preliminary item embedding representation.

4. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 3, characterized in that, The processing procedure of the intramodal attention module in step (2) is as follows: the item modal representation after isomorphic graph learning is transformed by three learnable parameter matrices, the weight of each feature dimension is calculated by using the softmax normalization function, and the item modal features are reweighted. Then, residual connections and layer normalization operations are introduced to obtain the item representation optimized by intramodal attention.

5. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 4, characterized in that, The processing procedure of the cross-modal attention module in step (2) is as follows: cross-modal attention is applied to the item representations of the visual modality and the text modality respectively; residual connection and layer normalization operations are applied to the attention calculation results to obtain the item representations that fuse cross-modal complementary information.

6. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 5, characterized in that, The learnable view generator in step (4) includes three complementary view generation modules, specifically: Graph structure view generation module: Using the user representation and item representation obtained from heterogeneous graph learning, the edge weights of the graph structure view are calculated by inputting them into the sigmoid function after element-wise multiplication. Prototype-aware view generation module: Represent each user-item interaction as a weighted combination of multiple potential behavior prototypes. Through learnable prototype latent variables, combined with softmax normalization, the weight distribution of the interaction is calculated, and then the edge weights of the prototype-aware view are obtained through element-wise multiplication and the sigmoid function. Graph Attention View Generation Module: Employs a lightweight attention mechanism, processing the user-item interaction matrix through learnable attention vectors, assigning personalized weights to each interaction, and obtaining the edge weights of the graph attention view.

7. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 6, characterized in that, The processing steps of the adaptive multi-view fusion mechanism in step (4) include: (401): The edge weight matrix of each view is transformed by a shared learnable function, which is achieved by a linear transformation and a hyperbolic tangent activation function; (402): Softmax normalize the edge weights of each transformed view to obtain the attention weights of each view. Based on these weights, the edge weights of all views are weighted and summed to obtain the shared edge weights of the views. (403): The unique edge weights of each view are obtained by subtracting the shared edge weights from the original edge weights of each view; (404): The shared edge weights of the views are summed and combined with the unique edge weights of all views to obtain the final merged edge weights.

8. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 7, characterized in that, The specific process of learning heterogeneous graph relationships in step (3) is as follows: The item representation optimized by modal attention is concatenated with the user representation learned from the user isomorphic graph, and used as the initial embedding of the heteromorphic graph; A symmetric matrix is ​​constructed based on the user-item interaction matrix. A convolution update rule is constructed using the diagonal form of the degree matrix of this symmetric matrix and the symmetric matrix itself. Multi-layer graph convolution operation is then performed on the initial embedding. The embeddings of each layer from layer 0 to layer L are averaged to obtain the final user and item representations after heterogeneous graph learning.

9. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 8, characterized in that, The process of constructing the comparison chart in step (4) is as follows: Based on the edge index set of user-item interaction and the final edge weight obtained by adaptive multi-view fusion, a corresponding sparse adjacency matrix is ​​constructed according to the predefined tensor shape. This sparse adjacency matrix is ​​the completed comparison graph. Perform graph convolution on the comparison graph to output embedded representations of users and items in the comparison view.

10. The multimodal recommendation method for modeling noise-resistant items and interactive behaviors according to claim 9, characterized in that, The overall objective function in step (5) is constructed as follows: Based on the Bayesian personalized ranking loss, the BPR loss is obtained by applying a sigmoid function to the difference in predicted ratings between observed user-item interaction pairs and unobserved interaction pairs in the training set, taking the negative logarithm, and then summing the results. The multimodal alignment loss and dynamic multi-view contrast loss are added to the BPR loss according to preset weights, and then an L2 regularization term is added to obtain the overall objective function used for model parameter training.

Images