A training method for a knowledge distillation recommendation model that integrates comments and sentiment.

Abstract

This invention belongs to the field of model training technology and relates to a training method for a knowledge distillation recommendation model that integrates reviews and sentiment. The method includes: generating rating embedding vectors, review embedding vectors, and sentiment embedding vectors based on user ratings of items; the knowledge distillation recommendation model includes a review graph convolution module, a sentiment graph convolution module, a rating prediction module, and a distillation fusion gating module; using the above modules, generating corresponding review preference embedding features and sentiment preference embedding features, adaptive fusion weights, review semantic probability distributions, and sentiment probability distributions based on the rating embedding vectors, review embedding vectors, and sentiment embedding vectors; calculating the user's predicted rating based on the above results and generating task loss values ​​and distillation constraint loss values; and training and optimizing the knowledge distillation recommendation model based on the above loss values. This invention constructs a dual-channel parallel graph learning framework for reviews and sentiment, improving the accuracy of user preference learning and item feature representation, and enhancing the recommendation performance of the recommendation model.

Description

Technical Field

[0001] This invention belongs to the field of model training technology, specifically relating to a method for training a knowledge distillation recommendation model that integrates comments and sentiment. Background Technology

[0002] Recommender systems are a key technology in the information overload era, widely used in e-commerce, online video, social media, news feeds, and other fields. Their core task is to predict a user's preference for unknown items based on their historical behavior, thereby providing personalized content recommendations.

[0003] Existing item recommendation methods typically rely on historical interaction information between users and items for modeling. However, when interaction data is sparse, relying solely on structured information such as ratings or clicks often fails to fully capture users' true preferences for items. In contrast, review texts often contain richer semantic information about item attributes, user experience, and sentiment. Therefore, incorporating review information into recommendation tasks has become an important research direction for improving rating prediction performance.

[0004] However, existing review-based recommendation methods still have the following shortcomings: First, most methods mainly focus on the semantic representation of the review text, using the entire review as a unified feature input to the model. While this can enhance user and item representations to some extent, it does not fully utilize the explicit or implicit sentiment reflected in the reviews, resulting in incomplete preference modeling. Second, although some methods further introduce sentiment information, most process the semantic features of the reviews and the sentiment embedding vectors separately, lacking an effective collaborative mechanism and making it difficult to fully leverage the complementary effects between the two types of information. Third, in scenarios with sparse data, unstable review quality, or high text noise, single-channel modeling or weakly coupled modeling methods are prone to unstable representation learning, thus affecting the accuracy and robustness of the recommendation results.

[0005] In summary, how to simultaneously utilize semantic and sentiment information from reviews and establish effective knowledge transfer relationships between these two information channels to improve the accuracy of user preference learning and item feature representation, and enhance the recommendation performance of recommendation models in complex review scenarios, has become a crucial technical problem that urgently needs to be solved in the field of model recommendation. Summary of the Invention

[0006] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a training method for a knowledge distillation recommendation model that integrates comments and sentiment. This method is used to construct a dual-channel parallel graph learning framework for comment channels and sentiment channels, realizing the collaborative modeling of comment semantic information and sentiment information. This allows user and item representations to integrate feature information from different sources, thereby improving the accuracy of recommendation results.

[0007] This invention provides a method for training a knowledge distillation recommendation model that integrates comments and sentiment, comprising: Obtain a user set and an item set, and generate a rating embedding vector, a comment embedding vector, and a sentiment embedding vector based on the user set, the item set, and the user ratings of the items in the item set; The pre-defined knowledge distillation recommendation model utilizes the comment graph convolutional module and the sentiment graph convolutional module in the pre-defined knowledge distillation recommendation model to generate corresponding comment preference embedding features and sentiment preference embedding features based on the rating embedding vector, the comment embedding vector, and the sentiment embedding vector, respectively; wherein, the knowledge distillation recommendation model also includes a rating prediction module and a distillation fusion gating module; Based on the comment preference embedding features, calculate the comment semantic interaction features and comment preference scores respectively; and based on the sentiment preference embedding features, calculate the sentiment interaction features and sentiment preference scores respectively. The distillation fusion gating module is used to perform gating fusion calculations based on the semantic interaction features of the comments and the emotional interaction features to obtain adaptive fusion weights. The rating prediction module calculates the rating probability based on the semantic interaction features and the sentiment interaction features of the comments, thereby obtaining the corresponding semantic probability distribution and sentiment probability distribution of the comments. The user's predicted rating is calculated based on the adaptive fusion weights, the comment preference score, and the sentiment preference score; A task loss value is generated based on the rating, the predicted rating, the comment preference score, and the sentiment preference score; a distillation constraint loss value is generated based on the comment semantic probability distribution and the sentiment probability distribution. The knowledge distillation recommendation model is optimized based on the task loss value and the distillation constraint loss value, and the steps of generating corresponding comment preference embedding features and sentiment preference embedding features using the knowledge distillation recommendation model are returned until the target training conditions are met, thus obtaining the target knowledge distillation recommendation model.

[0008] In one embodiment of the present invention, the step of generating a rating embedding vector, a comment embedding vector, and a sentiment embedding vector based on the user set, the item set, and the ratings given by users in the user set to the items in the item set, respectively, includes: Generate a rating embedding vector based on the ratings of users in the user set for items in the project set; Based on the target comments from users in the user set to the items in the project set, comment embedding vectors and sentiment embedding vectors are generated respectively.

[0009] In one embodiment of the present invention, it further includes: A first triplet is constructed based on the rating embedding vector and the comment embedding vector to generate a comment-rating awareness map; wherein, the first triplet also includes initialized user-comment embedding features and item-comment embedding features; A second triplet is constructed based on the rating embedding vector and the sentiment embedding vector to generate a sentiment-rating perception map; wherein, the second triplet also includes initialized user-sentiment embedding features and item-sentiment embedding features.

[0010] In one embodiment of the present invention, the step of generating corresponding comment preference embedding features and sentiment preference embedding features based on the rating embedding vector, the comment embedding vector, and the sentiment embedding vector using the comment graph convolutional module and the sentiment graph convolutional module in the preset knowledge distillation recommendation model, respectively, includes: The first user neighbor set and the first item neighbor set are determined based on the comment-rating awareness graph corresponding to the comment embedding vector; Obtain the historical user-comment preference embedding features and historical item-comment preference embedding features generated in the previous training round for the first user neighbor set and the first item neighbor set; The comment graph convolutional module generates user-comment preference embedding features based on the first user neighbor set and the corresponding historical item-comment preference embedding features; The comment graph convolutional module generates item-comment preference embedding features based on the first item neighbor set and the corresponding historical user-comment preference embedding features; The user-comment preference embedding feature and the project-comment preference embedding feature are used as comment preference embedding features; The second user neighbor set and the second item neighbor set are determined based on the sentiment-rating perception map corresponding to the sentiment embedding vector. Obtain the historical user-sentiment preference embedding features and historical item-sentiment preference embedding features generated in the previous training round for the second user neighbor set and the second item neighbor set; The sentiment graph convolutional module generates user-sentiment preference embedding features based on the second user neighbor set and the corresponding historical item-sentiment preference embedding features; The sentiment graph convolutional module generates item-sentiment preference embedding features based on the second item neighbor set and the corresponding historical user-sentiment preference embedding features; The user-emotional preference embedding feature and the item-emotional preference embedding feature are used as emotional preference embedding features.

[0011] In one embodiment of the present invention, the step of calculating the comment semantic interaction features and the comment preference score based on the comment preference embedding features includes: Element-wise multiplication of the user-comment preference embedding feature and the item-comment preference embedding feature yields the comment semantic interaction feature; The user-comment preference embedding feature and the item-comment preference embedding feature are transposed and multiplied to obtain the comment preference score.

[0012] In one embodiment of the present invention, the step of using the distillation fusion gating module to perform gating fusion calculation based on the semantic interaction features of the comments and the emotional interaction features to obtain adaptive fusion weights includes: The distillation fusion gating module calculates the distillation gating coefficient based on the semantic interaction features and the emotional interaction features of the comment. The distillation gating coefficient is gated and fused with the comment semantic interaction feature and the sentiment interaction feature to obtain the fused feature; Multi-head attention is performed on the fused features to obtain adaptive fusion weights.

[0013] In one embodiment of the present invention, the distillation gate coefficient is calculated using the following distillation-fusion gate formula:

[0014] in, This is expressed as the distillation gate coefficient. Represented as the first The user on the first The semantic interaction features of comments for each project Represented as the first The user on the first The emotional interaction characteristics of each project Represented as a preset gated transformation weight matrix, Represented as a preset bias vector; The fused features are calculated using the following gated fusion formula:

[0015] in, Represented as fusion features, This is expressed as the distillation gate coefficient. Represented as the first The user on the first The semantic interaction features of comments for each project Represented as the first The user on the first Emotional interaction characteristics of each project; The adaptive fusion weights are calculated using the following multi-head attention formula:

[0016] in, Represented as adaptive fusion weights, Represented as fusion features, Represented as a multilayer perceptron, This is represented by the sigmoid activation function.

[0017] In one embodiment of the present invention, the step of using the rating prediction module to calculate the rating probability based on the comment semantic interaction features and the sentiment interaction features to obtain the corresponding comment semantic probability distribution and sentiment probability distribution includes: The comment semantic interaction features and the sentiment interaction features are respectively input into the rating prediction module to obtain the first initial predicted rating of the comment semantic interaction and the second initial predicted rating of the sentiment interaction. The first and second initial predicted scores are normalized respectively, and the corresponding comment semantic probability distribution and sentiment probability distribution are obtained.

[0018] In one embodiment of the present invention, generating a task loss value based on the rating, the predicted rating, the comment preference score, and the sentiment preference score includes: Calculate the comment semantic branch prediction loss value based on the rating and the comment preference score; Calculate the sentiment branch prediction loss value based on the rating and the sentiment preference score; Calculate the comment fusion prediction loss value based on the stated score and the predicted score; The task loss value is calculated based on the branch prediction loss value, the sentiment branch prediction loss value, and the comment fusion prediction loss value.

[0019] In one embodiment of the present invention, the user's predicted score is calculated using the following weighted fusion formula:

[0020] in, Represented as the first The user on the first Predicted scores for each project Represented as adaptive fusion weights, Represented as the first The user on the first Comment preference scores for each project Represented as the first The user on the first Sentiment preference scores for each item; The distillation constraint loss value is calculated using the following distillation constraint loss function:

[0021] in, This is expressed as the distillation constraint loss value. Represented as the first The user on the first The semantic probability distribution of comments for each project Represented as the first The user on the first The sentiment probability distribution of each item It is represented as a divergence function.

[0022] The beneficial effects of this invention are as follows: This invention simultaneously utilizes both semantic and sentiment information from reviews, overcoming the problem of insufficient information utilization caused by the single modeling of review information in existing technologies. It can more comprehensively characterize user preferences and item features from different dimensions. By constructing a dual-channel parallel graph learning framework for review and sentiment channels, it achieves collaborative modeling of review semantic and sentiment information, enabling user and item representations to integrate feature information from different sources, thereby improving the accuracy of recommendation results. By introducing a bidirectional knowledge distillation mechanism, a knowledge complementarity relationship is formed between the two channels. It can both utilize sentiment features to mitigate the influence of noise in review text and utilize review semantics to enhance the expressive power of sentiment branches, thereby improving the accuracy, robustness, and stability of recommendation results. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0024] Figure 1 This is a flowchart illustrating a knowledge distillation recommendation model training method that integrates comment and sentiment analysis, provided in one embodiment of the present invention. Figure 2 This is a functional block diagram of a knowledge distillation recommendation model training system that integrates commentary and sentiment, provided in one embodiment of the present invention. Detailed Implementation

[0025] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0026] Please see Figure 1As shown in the embodiment of the present invention, a knowledge distillation recommendation model training method for fusion of comments and sentiment is provided. By constructing a semantic branch and a sentiment branch for comments, the semantic features and sentiment embedding vectors in user comments are co-modeled. Bidirectional knowledge distillation is used to achieve information transfer and complementary enhancement between the two branches, thereby improving the learning effect of user preference representation and item feature representation, and improving the accuracy of rating prediction and model robustness.

[0027] Specifically, the knowledge distillation recommendation model training method that integrates comments and sentiment provided in this embodiment of the invention includes the following steps: S1. Obtain a user set and an item set, and generate a rating embedding vector, a comment embedding vector, and a sentiment embedding vector based on the user set, the item set, and the ratings of users in the user set for items in the item set.

[0028] In this embodiment of the invention, the project set is ,in, This represents the j-th item. Represents the total number of projects; the user set is ,in, This represents the i-th user. This represents the total number of users.

[0029] The user set and the project set also include the i-th user. For the j-th project rating The i-th user For the j-th project Comment text embedding vector The i-th user For the j-th project Emotional characteristics and the i-th user For the j-th project Comment information .

[0030] In this embodiment of the invention, generating rating embedding vectors, comment embedding vectors, and sentiment embedding vectors based on the user set, the item set, and user ratings of items in the item set, respectively, includes: Generate a rating embedding vector based on the ratings of users in the user set for items in the project set; Based on the target comments from users in the user set to the items in the project set, comment embedding vectors and sentiment embedding vectors are generated respectively.

[0031] Specifically, a user-item bipartite graph is constructed based on the user set and item set. ,in, Represents the i-th user For the j-th project The rating; the rating of the i-th user. For the j-th project rating Mapped to the i-th user For the j-th project Rating embedding vector .

[0032] Specifically, the i-th user For the j-th project Comment information The input is fed into the BERT-Whitening model for processing, and the i-th user is generated accordingly. For the j-th project Comment embedding vector The BERT model is used to characterize the semantic features in the comment text. The BERT-whitening model "whitens" the sentence vectors output by the BERT model to generate high-quality semantic embedding vectors that are evenly distributed in space, representing the core semantics of the entire comment.

[0033] Furthermore, the i-th user For the j-th project Comment information The input is processed in a textblob, and the i-th user is generated accordingly. For the j-th project Emotional embedding vector The textblob is used to characterize the sentiment tendency features in comment text. It is a rich Python natural language processing library that is mainly based on sentiment dictionaries and rules. It analyzes each word in the comment information, calculates a score according to a dictionary that is pre-labeled as "positive" or "negative", and uses a weighted average strategy to obtain the final sentiment tendency.

[0034] This invention provides a unified data input for learning the representation of the interaction between users, items, and ratings by structuring the interaction relationship between users, items, and ratings.

[0035] S2. Using the comment graph convolution module and sentiment graph convolution module in the preset knowledge distillation recommendation model, respectively, generate corresponding comment preference embedding features and sentiment preference embedding features based on the rating embedding vector, the comment embedding vector, and the sentiment embedding vector; wherein, the knowledge distillation recommendation model also includes a rating prediction module and a distillation fusion gating module.

[0036] In this embodiment of the invention, a first triplet is constructed based on the rating embedding vector and the comment embedding vector to generate a comment-rating perception map; wherein, the first triplet also includes initialized user-comment embedding features and item-comment embedding features; A second triplet is constructed based on the rating embedding vector and the sentiment embedding vector to generate a sentiment-rating perception map; wherein, the second triplet also includes initialized user-sentiment embedding features and item-sentiment embedding features.

[0037] Specifically, based on the user project bipartite graph Constructing a comment-rating perception graph to understand the interaction between users and projects. ,make Any triplet in the middle is denoted as ,in, Represents the i-th user Comment preference embedding in the comment channel Represents the j-th item Comment preference embedding representation in the comment channel; In constructing a review-rating perception graph The initialization of the triplet in each stage yields the first triplet, which is obtained by randomly initializing the comment preference embeddings of the user node and item node in round 0, denoted as the i-th user node, etc. Initial comment preference embedding representation and the Project Initial comment preference embedding representation Let the current iteration round... The i-th user In the The comment preference embedding after round of iteration updates is represented as follows The j-th project In the The comment preference embedding after round of iteration updates is represented as follows .

[0038] Furthermore, based on the interaction relationships between users and items in the user-item bipartite graph G, an emotion-rating perception graph is constructed. ,make Any triplet in the middle is denoted as ,in, Represents the i-th user Emotional preference embedding representation in the emotional channel Represents the j-th item Emotional preference embedding representation in the emotional channel; In constructing an emotion rating perception map The initialization of the triplet in the first stage yields the second triplet, which is obtained by randomly initializing the sentiment preference embeddings of the user node and the item node in round 0, denoted as ______. individual users Initial sentiment preference embedding representation and the j-th project Initial sentiment preference embedding representation Let the current iteration round The i-th user In the The sentiment preference embedding after rounds of iterative updates is represented as follows The j-th project In the The sentiment preference embedding after rounds of iterative updates is represented as follows .

[0039] This invention constructs comment-rating perception graphs and sentiment-rating perception graphs respectively, enabling the modeling of user preferences and item features under different information sources, providing a structural foundation for subsequent bi-branch representation propagation and knowledge distillation; it extracts semantic and sentiment representations from comment texts and constructs a bi-source graph structure in conjunction with user-item interaction relationships, providing input for subsequent branch representation learning and cross-branch knowledge transfer.

[0040] In this embodiment of the invention, the recommendation model based on bi-branch knowledge distillation includes a comment semantic branch and a sentiment branch. The comment semantic branch is used to learn the preference representations of users and items in the comment semantic space, and the sentiment branch is used to learn the preference representations of users and items in the sentiment space. During model training, distillation constraints are introduced into the rating prediction distributions output by the comment semantic branch and the rating prediction distributions output by the sentiment branch. Branch fusion weights are then generated based on the distilled and enhanced fusion representations to obtain the i-th user's preference. For the j-th project Predicted score .

[0041] Furthermore, the comment graph convolutional module includes a first linear layer, a comment graph convolutional aggregation layer, a second linear layer, and a third linear layer. This comment graph convolutional module processes the comment-rating perception graph. Iterative update processing is performed. Specifically, the first linear layer maps comment text features to the same feature space as the node embeddings; the comment graph convolutional aggregation layer combines neighbor node representations, comment edge features, and normalization coefficients on the comment-rating perception graph to perform message passing and neighborhood aggregation; the second and third linear layers perform linear transformations on the user node aggregation representation and the item node aggregation representation, respectively.

[0042] After M rounds of iterative updates, the user node aggregation representation and item node aggregation representation output from the comment graph convolutional aggregation layer are input into the second linear layer and the third linear layer, respectively, to obtain the i-th user. Comment preference embedding representation in round M and the j-th project Comment preference embedding representation in round M . and The vector representations of the i-th user and the j-th item in the comment semantic branch are formed by the neighborhood interaction relationship on the comment-rating perception graph, the comment text features, and the aggregation of neighborhood information in multiple rounds. They are used to characterize the user's preference features in the comment semantic space and the item's semantic features in the comment semantic space, respectively.

[0043] The sentiment graph convolutional module includes a sentiment graph convolutional aggregation layer, a fourth linear layer, and a fifth linear layer. This module processes the sentiment-rating perception graph. Iterative update processing is performed. Specifically, the sentiment graph convolutional aggregation layer is used to perform message passing and neighborhood aggregation on the sentiment-rating perception graph by combining neighbor node representations, sentiment edge features, and normalization coefficients; the fourth and fifth linear layers are used to perform linear transformations on the user node aggregation representation and the item node aggregation representation, respectively.

[0044] After M rounds of iterative updates, the user node aggregation representation and item node aggregation representation output from the sentiment graph convolutional aggregation layer are input into the fourth linear layer and the fifth linear layer, respectively, to obtain the i-th user. The sentiment preference embedding representation updated in the Mth iteration. and the j-th project Sentiment preference embedding representation updated in the Mth iteration . and This represents the final vector representation of the i-th user and the j-th item in the sentiment branch, formed by the neighborhood interaction relationship, sentiment score, and multi-round neighborhood information aggregation on the sentiment-rating perception graph. It is used to characterize the user's preference features in the sentiment space and the item's sentiment features in the sentiment space, respectively.

[0045] In this embodiment of the invention, the step of generating corresponding comment preference embedding features and sentiment preference embedding features based on the rating embedding vector, the comment embedding vector, and the sentiment embedding vector using the comment graph convolutional module and the sentiment graph convolutional module in the preset knowledge distillation recommendation model includes: The first user neighbor set and the first item neighbor set are determined based on the comment-rating awareness graph corresponding to the comment embedding vector; Obtain the historical user-comment preference embedding features and historical item-comment preference embedding features generated in the previous training round for the first user neighbor set and the first item neighbor set; The comment graph convolutional module generates user-comment preference embedding features based on the first user neighbor set and the corresponding historical item-comment preference embedding features; The comment graph convolutional module generates item-comment preference embedding features based on the first item neighbor set and the corresponding historical user-comment preference embedding features; The user-comment preference embedding feature and the project-comment preference embedding feature are used as comment preference embedding features; The second user neighbor set and the second item neighbor set are determined based on the sentiment-rating perception map corresponding to the sentiment embedding vector. Obtain the historical user-sentiment preference embedding features and historical item-sentiment preference embedding features generated in the previous training round for the second user neighbor set and the second item neighbor set; The sentiment graph convolutional module generates user-sentiment preference embedding features based on the second user neighbor set and the corresponding historical item-sentiment preference embedding features; The sentiment graph convolutional module generates item-sentiment preference embedding features based on the second item neighbor set and the corresponding historical user-sentiment preference embedding features; The user-emotional preference embedding feature and the item-emotional preference embedding feature are used as emotional preference embedding features.

[0046] Specifically, neighborhood information is aggregated on the review-rating perception graph to determine the first user neighbor set and the first item neighbor set, so as to update the representation of user nodes and item nodes in the review semantic branch; neighborhood information is aggregated on the sentiment-rating perception graph to determine the second user neighbor set and the second item neighbor set, so as to update the representation of user nodes and item nodes in the sentiment semantic branch.

[0047] Specifically, the user-review preference embedding feature and the item-review preference embedding feature can be represented by the following formula:

[0048]

[0049] in, This represents the preset comment feature transformation weight matrix. Represents the i-th user For the j-th project Comment text embedding vector, and These represent the comment-rating perception graphs respectively. The i-th user and the j-th project The corresponding first user neighbor set and first project neighbor set, Indicates the first The user-comment preference embedding representation of the round, i.e., the historical user-comment preference embedding feature, Indicates the first The item-comment preference embedding representation of the round is the historical item-comment preference embedding feature.

[0050] By using the weighted aggregation operation of graph convolution, the semantic information of comments is propagated along the user-item interaction edge and aggregated onto the user and item nodes, thereby obtaining the embedded expression of each user and item's comment preferences in the comment channel.

[0051] During each iteration of model training, in the comment-rating perception graph The i-th user was obtained from the above. In the Comment preference embedding representation updated in rounds of iteration and the j-th project In the Comment preference embedding representation updated in rounds of iteration Thus, the i-th user is obtained. Comment preference embedding representation after M-th iteration update and the j-th project Comment preference embedding representation after M-th iteration update ;in, and This represents the final vector representation of the i-th user and j-th item in the comment semantic branch, formed based on the neighborhood interaction relationship on the comment-rating perception graph, comment text features, and multi-round neighborhood information aggregation. It is used to characterize the user's preference features in the comment semantic space.

[0052] Furthermore, the user-sentiment preference embedding feature and the item-sentiment preference embedding feature can be represented by the following formula:

[0053]

[0054] in, Represents the i-th user For the j-th project The emotional characteristics indicate, and These represent the different aspects of the emotion-rating perception map. The i-th user and the j-th project The neighbor set corresponding to the second user neighbor set and the second item neighbor set, Indicates the first The user-sentiment preference embedding representation of the round, i.e., the historical user-sentiment preference embedding feature, Indicates the first The item-sentiment preference embedding representation of the round is the historical item-sentiment preference embedding feature.

[0055] By using the weighted aggregation operation of graph convolution, the sentiment information of comments is propagated along the user-item interaction edge and aggregated to the user and item nodes, thereby obtaining the sentiment preference embedding expression of each user and item in the sentiment channel.

[0056] During each iteration of model training, in the sentiment-rating perception graph The i-th user was obtained from the above. In the Sentiment preference embedding representation updated in rounds of iteration and the j-th project In the Sentiment preference embedding representation updated in rounds of iteration Thus, the i-th user is obtained. Sentiment preference embedding representation updated in the Mth iteration and the j-th project Sentiment preference embedding representation updated in the Mth iteration ;in, and This represents the final vector representation of the i-th user and j-th item in the sentiment branch, formed by the neighborhood interaction relationship on the sentiment-rating perception graph, the sentiment score, and the aggregation of multiple rounds of neighborhood information. It is used to characterize the user's preference features in the sentiment space.

[0057] S3. Calculate the semantic interaction features and comment preference scores based on the comment preference embedding features, and calculate the emotional interaction features and emotional preference scores based on the emotional preference embedding features.

[0058] In this embodiment of the invention, calculating the comment semantic interaction features and the comment preference score based on the comment preference embedding features includes: Element-wise multiplication of the user-comment preference embedding feature and the item-comment preference embedding feature yields the comment semantic interaction feature; The user-comment preference embedding feature and the item-comment preference embedding feature are transposed and multiplied to obtain the comment preference score.

[0059] Specifically, the i-th user output by the comment graph convolution module Comment preference embedding representation after M-th iteration update That is, user-review preference embedding features, and the j-th item Comment preference embedding representation after M-th iteration update In other words, by embedding the item-comment preference features, we can obtain the user-item interaction representation corresponding to the semantic branch of the comment. That is, the semantic interaction features of comments: ,in, This represents the Hadamard product, which is the element-wise multiplication of vectors. Furthermore, the i-th user and the j-th project Comment preference score between Represented as: .

[0060] In this embodiment of the invention, calculating the emotional interaction features and the emotional preference score based on the emotional preference embedding features includes: The emotional interaction features are obtained by multiplying the user-emotional preference embedding features and the item-emotional preference embedding features element by element. The sentiment preference score is obtained by transposing and multiplying the user-sentiment preference embedding feature and the item-sentiment preference embedding feature.

[0061] Specifically, the i-th user output using the sentiment graph convolution module Sentiment preference embedding representation updated in the Mth iteration That is, user-sentiment preference embedding features, and the j-th item Sentiment preference embedding representation updated in the Mth iteration In other words, by embedding the item-sentiment preference features, we can obtain the user-item interaction representation corresponding to the sentiment branch. That is, the characteristics of emotional interaction: ; Furthermore, the i-th user and the j-th project Sentiment preference scores between Represented as: .

[0062] S4. The distillation fusion gating module is used to perform gating fusion calculation based on the semantic interaction features of the comments and the emotional interaction features to obtain adaptive fusion weights.

[0063] In this embodiment of the invention, the step of using the distillation fusion gating module to perform gating fusion calculation based on the semantic interaction features of the comments and the emotional interaction features to obtain adaptive fusion weights includes: The distillation fusion gating module calculates the distillation gating coefficient based on the semantic interaction features and the emotional interaction features of the comment. The distillation gating coefficient is gated and fused with the comment semantic interaction feature and the sentiment interaction feature to obtain the fused feature; Multi-head attention is performed on the fused features to obtain adaptive fusion weights.

[0064] Specifically, a distillation-fusion gating mechanism is introduced, and the distillation gating coefficient is calculated using the following distillation-fusion gating formula:

[0065] in, This is expressed as the distillation gate coefficient. Represented as the first The user on the first The semantic interaction features of comments for each project Represented as the first The user on the first The emotional interaction characteristics of each project Represented as a preset gated transformation weight matrix, Represented as a preset bias vector; The fused features are calculated using the following gated fusion formula:

[0066] in, Represented as a fusion feature, it is used to synthesize semantic and sentiment information of the comments based on the inter-branch distillation constraint, and adaptively adjust the contribution of the two branches in the fusion process. This is expressed as the distillation gate coefficient. Represented as the first The user on the first The semantic interaction features of comments for each project Represented as the first The user on the first Emotional interaction characteristics of each project; The distillation-enhanced fusion features are input into the attention fusion module, and the adaptive fusion weights are calculated using the following multi-head attention formula:

[0067] in, Represented as adaptive fusion weights, Represented as fusion features, Represented as a multilayer perceptron, This is represented by the sigmoid activation function.

[0068] S5. The rating prediction module calculates the rating probability based on the semantic interaction features and the sentiment interaction features of the comments to obtain the corresponding semantic probability distribution and sentiment probability distribution of the comments.

[0069] In this embodiment of the invention, the step of using the rating prediction module to calculate the rating probability based on the comment semantic interaction features and the sentiment interaction features, and obtaining the corresponding comment semantic probability distribution and sentiment probability distribution, includes: The comment semantic interaction features and the sentiment interaction features are respectively input into the rating prediction module to obtain the first initial predicted rating of the comment semantic interaction and the second initial predicted rating of the sentiment interaction. The first and second initial predicted scores are normalized respectively, and the corresponding comment semantic probability distribution and sentiment probability distribution are obtained.

[0070] Specifically, representing user-project interactions Input the rating prediction module and output the predicted score vector corresponding to each rating level. As a semantic branch of comments, for users With the project The first initial predicted score for the interaction; representing the user-item interaction. Input the rating prediction module and output the predicted score vector corresponding to each rating level. As an emotional branch for users With the project The second initial predicted rating for the interaction. The prediction layer structure in the rating prediction modules of the comment semantic branch and sentiment branch is the same, consisting of a multilayer perceptron (MLP) with two linear layers. The layers are nonlinearly transformed using the ReLU activation function, and the final output is a vector with the number of rating levels.

[0071] Furthermore, the first initial predicted score for the comment semantic interaction output by the comment semantic branch. The second initial prediction score based on the sentiment interaction with the sentiment branch output. The corresponding semantic probability distribution of the comments was obtained by performing softmax normalization on each of them. And the probability distribution of emotions :

[0072]

[0073] in, This represents the softmax function. This indicates the preset temperature coefficient.

[0074] S6. Calculate the user's predicted score based on the adaptive fusion weights, the comment preference score, and the sentiment preference score.

[0075] In this embodiment of the invention, the semantic branch prediction score and the sentiment branch prediction score of the comment are weighted and fused, and the i-th user is obtained using the following weighted fusion formula. For the j-th project Predicted score :

[0076] in, Represented as the first The user on the first Predicted scores for each project This is represented as adaptive fusion weights, with values ​​ranging from 0 to 1, dynamically adjusting the contribution of the two branches to the final prediction result. Represented as the first The user on the first Comment preference scores for each project Represented as the first The user on the first Sentiment preference scores for each item.

[0077] This invention first calculates the prediction results for the two branches based on the user representation and item representation output from the comment semantic branch and sentiment branch, respectively. Then, it concatenates the user-item interaction embeddings corresponding to the two branches and inputs them into an attention fusion module to generate adaptive fusion weights for the current user-item sample. Finally, it dynamically weights the prediction results of the two branches using these fusion weights to obtain the final rating prediction result. Since different user-item interaction samples have varying degrees of dependence on comment semantic and sentiment information, the attention fusion mechanism can adaptively allocate the contribution ratio of the two branches according to sample features, improving prediction flexibility and recommendation accuracy.

[0078] S7. Generate a task loss value based on the rating, the predicted rating, the comment preference score, and the sentiment preference score, and generate a distillation constraint loss value based on the comment semantic probability distribution and the sentiment probability distribution.

[0079] In this embodiment of the invention, a task loss function is constructed. The recommendation task loss function includes comment semantic branch prediction loss. Sentiment branch prediction loss and fusion prediction loss This is used to separately constrain the differences between the comment semantic branch prediction results, sentiment branch prediction results, and the final fusion prediction results and the actual scores.

[0080] Specifically, generating a task loss value based on the rating, the predicted rating, the comment preference score, and the sentiment preference score includes: Calculate the comment semantic branch prediction loss value based on the rating and the comment preference score; Calculate the sentiment branch prediction loss value based on the rating and the sentiment preference score; Calculate the comment fusion prediction loss value based on the stated score and the predicted score; The task loss value is calculated based on the branch prediction loss value, the sentiment branch prediction loss value, and the comment fusion prediction loss value.

[0081] Specifically, the comment semantic branch prediction loss, sentiment branch prediction loss, and fusion prediction loss can be expressed by the following formula:

[0082] in, This indicates review, sentiment, or fuse. This represents the loss for predicting semantic branches in comments. Sentiment branch prediction loss and fusion prediction loss , This indicates the comment preference score, sentiment preference score, or predicted score. This indicates the score for genuine review preferences, the score for genuine sentiment preferences, or the genuine rating. This represents a collection of user-project interactions with actual rating records. express Regularization term, This is the preset first hyperparameter, used to control the contribution of the regularization term to the loss function; The task loss value can be expressed by the following formula:

[0083] in, This is a preset second hyperparameter used to control the contribution of the fusion prediction loss to the recommendation task loss function.

[0084] Since the prediction results are explicit rating data, the rating error between users and items is constrained by the task loss function to enhance the model's ability to fit the rating prediction task; the regularization term is designed to suppress excessively large model parameters and prevent overfitting during model training.

[0085] Furthermore, the distillation constraint loss value is calculated using the following distillation constraint loss function:

[0086] in, This is expressed as the distillation constraint loss value. Represented as the first The user on the first The semantic probability distribution of comments for each project Represented as the first The user on the first The sentiment probability distribution of each item It is represented as the Kullback-Leibler divergence function and is used to measure the difference between two probability distributions.

[0087] S8. Optimize the knowledge distillation recommendation model based on the task loss value and the distillation constraint loss value, and return to the step of generating corresponding comment preference embedding features and sentiment preference embedding features using the knowledge distillation recommendation model until the target training conditions are met, and obtain the target knowledge distillation recommendation model.

[0088] In this embodiment of the invention, the total loss value can be calculated based on the task loss value and the distillation constraint loss value: ,in, This is a preset third hyperparameter used to control the contribution of knowledge distillation loss to the overall loss.

[0089] Furthermore, based on the user-item interaction graph, gradient descent is used to iteratively train the knowledge distillation recommendation model that integrates comments and sentiment. Specifically, in each round of training, the comment text features are... and emotional characteristics The user nodes and item nodes obtained from the initialization are input to the model. In the comment semantic branch, the comment-rating awareness graph is used. After propagation and updating by the aforementioned comment graph convolutional module, the first... individual users Comment preference embedding representation and the Project Comment preference embedding representation In the emotion branch, based on the emotion-rating perception graph After propagation and updating by the aforementioned sentiment graph convolutional module, the first... individual users Emotional preference embedding representation and the Project Emotional preference embedding representation .

[0090] Then, based on the user-item interaction representations corresponding to the semantic and sentiment branches of the comments respectively. and This yields the prediction score vectors for each of the two branches. and Rating prediction probability distribution and Distillation constraint terms (i.e., distillation constraint loss value) and adaptive fusion weights Calculate the first individual users For the Project Final prediction score .

[0091] Finally, based on the total loss function Calculate the loss value for the current round, and use gradient descent to backpropagate and iteratively update the parameters in the model. Repeat the above training process until the number of training iterations reaches the set number, or the total loss function is satisfied. If the loss is less than the set loss threshold, i.e. the target training condition, then training stops, and the optimal knowledge distillation recommendation model that integrates comments and sentiment is obtained.

[0092] The optimal knowledge distillation recommendation model is used to predict ratings for target users and candidate items, and the predicted ratings are sorted from highest to lowest. The top-ranked items are then selected. The recommended items are then output to the corresponding users.

[0093] This invention improves the user preference modeling capability and ultimately generates high-quality recommendation results by jointly optimizing the recommendation task loss and knowledge distillation loss, enabling the comment channel and sentiment channel to achieve knowledge transfer and collaborative enhancement while completing the rating prediction task.

[0094] like Figure 2 The diagram shown is a functional block diagram of a knowledge distillation recommendation model training system that integrates comments and sentiment, according to an embodiment of the present invention.

[0095] The knowledge distillation recommendation model training system 200, which integrates comment and sentiment analysis, can be installed in an electronic device. Depending on its functionality, the system may include an embedding vector generation module 201, a graph convolution module 202, a bi-branch feature calculation module 203, a distillation fusion gating module 204, a rating prediction module 205, a predicted rating generation module 206, a constraint loss generation module 207, and a knowledge distillation recommendation model optimization module 208. A module in this invention can also be referred to as a unit, which is a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0096] In this embodiment, the functions of each module / unit are as follows: Embedded vector generation module 201 is used to obtain a user set and an item set, and generate rating embedding vectors, comment embedding vectors and sentiment embedding vectors according to the user set, the item set and the ratings of users in the user set for items in the item set, respectively. The graph convolution module 202 is used to generate corresponding comment preference embedding features and sentiment preference embedding features based on the rating embedding vector, the comment embedding vector, and the sentiment embedding vector, respectively, using the comment graph convolution module and the sentiment graph convolution module in the preset knowledge distillation recommendation model; wherein, the knowledge distillation recommendation model further includes a rating prediction module and a distillation fusion gating module; The dual-branch feature calculation module 203 is used to calculate the comment semantic interaction feature and comment preference score according to the comment preference embedding feature, and to calculate the sentiment interaction feature and sentiment preference score according to the sentiment preference embedding feature. The distillation fusion gating module 204 is used to perform gating fusion calculation based on the semantic interaction features of the comments and the emotional interaction features to obtain adaptive fusion weights. The rating prediction module 205 is used to calculate the rating probability based on the comment semantic interaction features and the sentiment interaction features to obtain the corresponding comment semantic probability distribution and sentiment probability distribution. The prediction rating generation module 206 is used to calculate the user's predicted rating based on the adaptive fusion weights, the comment preference score, and the sentiment preference score. The constraint loss generation module 207 is used to generate a task loss value based on the rating, the predicted rating, the comment preference score, and the sentiment preference score, and to generate a distillation constraint loss value based on the comment semantic probability distribution and the sentiment probability distribution. The knowledge distillation recommendation model optimization module 208 is used to optimize the knowledge distillation recommendation model based on the task loss value and the distillation constraint loss value, and return the steps of generating corresponding comment preference embedding features and sentiment preference embedding features using the knowledge distillation recommendation model until the target training conditions are met, thus obtaining the target knowledge distillation recommendation model.

[0097] In detail, each module in the knowledge distillation recommendation model training system 200 for fusion of comments and sentiment in this invention employs the same technical means as the knowledge distillation recommendation model training method for fusion of comments and sentiment in the accompanying drawings, and can produce the same technical effects, which will not be elaborated here.

[0098] In summary, this invention utilizes both semantic and sentiment information from reviews, overcoming the information inadequacy caused by the single-modeling of review information in existing technologies. It can more comprehensively characterize user preferences and item features from different dimensions. By constructing a dual-channel parallel graph learning framework for review and sentiment channels, it achieves collaborative modeling of review semantic and sentiment information, enabling user and item representations to integrate feature information from different sources, thereby improving the accuracy of recommendation results. Furthermore, by introducing a bidirectional knowledge distillation mechanism, a complementary knowledge relationship is formed between the two channels. This allows for the use of sentiment features to mitigate the impact of noise in review texts and the use of review semantics to enhance the expressive power of sentiment branches, thus improving the accuracy, robustness, and stability of recommendation results.

[0099] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0100] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0101] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0102] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0103] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for training a knowledge distillation recommendation model that integrates comments and sentiment, characterized in that, include: Obtain a user set and an item set, and generate a rating embedding vector, a comment embedding vector, and a sentiment embedding vector based on the user set, the item set, and the user ratings of the items in the item set; The pre-defined knowledge distillation recommendation model utilizes the comment graph convolutional module and the sentiment graph convolutional module in the pre-defined knowledge distillation recommendation model to generate corresponding comment preference embedding features and sentiment preference embedding features based on the rating embedding vector, the comment embedding vector, and the sentiment embedding vector, respectively; wherein, the knowledge distillation recommendation model also includes a rating prediction module and a distillation fusion gating module; Based on the comment preference embedding features, calculate the comment semantic interaction features and comment preference scores respectively; and based on the sentiment preference embedding features, calculate the sentiment interaction features and sentiment preference scores respectively. The distillation fusion gating module is used to perform gating fusion calculations based on the semantic interaction features of the comments and the emotional interaction features to obtain adaptive fusion weights. The rating prediction module calculates the rating probability based on the semantic interaction features and the sentiment interaction features of the comments, thereby obtaining the corresponding semantic probability distribution and sentiment probability distribution of the comments. The user's predicted rating is calculated based on the adaptive fusion weights, the comment preference score, and the sentiment preference score; A task loss value is generated based on the rating, the predicted rating, the comment preference score, and the sentiment preference score; a distillation constraint loss value is generated based on the comment semantic probability distribution and the sentiment probability distribution. The knowledge distillation recommendation model is optimized based on the task loss value and the distillation constraint loss value, and the steps of generating corresponding comment preference embedding features and sentiment preference embedding features using the knowledge distillation recommendation model are returned until the target training conditions are met, thus obtaining the target knowledge distillation recommendation model.

2. The knowledge distillation recommendation model training method based on the fusion of comments and sentiment as described in claim 1, characterized in that, The step of generating rating embedding vectors, comment embedding vectors, and sentiment embedding vectors based on the user set, the item set, and user ratings of items in the item set, respectively, includes: Generate a rating embedding vector based on the ratings of users in the user set for items in the project set; Based on the target comments from users in the user set to the items in the project set, comment embedding vectors and sentiment embedding vectors are generated respectively.

3. The knowledge distillation recommendation model training method based on the fusion of comments and sentiment as described in claim 1, characterized in that, Also includes: A first triplet is constructed based on the rating embedding vector and the comment embedding vector to generate a comment-rating awareness map; wherein, the first triplet also includes initialized user-comment embedding features and item-comment embedding features; A second triplet is constructed based on the rating embedding vector and the sentiment embedding vector to generate a sentiment-rating perception map; wherein, the second triplet also includes initialized user-sentiment embedding features and item-sentiment embedding features.

4. The knowledge distillation recommendation model training method for integrating comments and sentiment as described in claim 3, characterized in that, The step of generating corresponding comment preference embedding features and sentiment preference embedding features based on the rating embedding vector, the comment embedding vector, and the sentiment embedding vector using the pre-defined knowledge distillation recommendation model's comment graph convolutional module and sentiment graph convolutional module respectively includes: The first user neighbor set and the first item neighbor set are determined based on the comment-rating awareness graph corresponding to the comment embedding vector; Obtain the historical user-comment preference embedding features and historical item-comment preference embedding features generated in the previous training round for the first user neighbor set and the first item neighbor set; The comment graph convolutional module generates user-comment preference embedding features based on the first user neighbor set and the corresponding historical item-comment preference embedding features; The comment graph convolutional module generates item-comment preference embedding features based on the first item neighbor set and the corresponding historical user-comment preference embedding features; The user-comment preference embedding feature and the project-comment preference embedding feature are used as comment preference embedding features; The second user neighbor set and the second item neighbor set are determined based on the sentiment-rating perception map corresponding to the sentiment embedding vector. Obtain the historical user-sentiment preference embedding features and historical item-sentiment preference embedding features generated in the previous training round for the second user neighbor set and the second item neighbor set; The sentiment graph convolutional module generates user-sentiment preference embedding features based on the second user neighbor set and the corresponding historical item-sentiment preference embedding features; The sentiment graph convolutional module generates item-sentiment preference embedding features based on the second item neighbor set and the corresponding historical user-sentiment preference embedding features; The user-emotional preference embedding feature and the item-emotional preference embedding feature are used as emotional preference embedding features.

5. The knowledge distillation recommendation model training method based on the fusion of comments and sentiment as described in claim 4, characterized in that, The step of calculating the comment semantic interaction features and comment preference scores based on the comment preference embedding features includes: Element-wise multiplication of the user-comment preference embedding feature and the item-comment preference embedding feature yields the comment semantic interaction feature; The user-comment preference embedding feature and the item-comment preference embedding feature are transposed and multiplied to obtain the comment preference score.

6. The knowledge distillation recommendation model training method based on the fusion of comments and sentiment as described in claim 1, characterized in that, The step of using the distillation fusion gating module to perform gating fusion calculation based on the semantic interaction features of the comments and the emotional interaction features to obtain adaptive fusion weights includes: The distillation fusion gating module calculates the distillation gating coefficient based on the semantic interaction features and the emotional interaction features of the comment. The distillation gating coefficient is gated and fused with the comment semantic interaction feature and the sentiment interaction feature to obtain the fused feature; Multi-head attention is performed on the fused features to obtain adaptive fusion weights.

7. The knowledge distillation recommendation model training method for integrating comments and sentiment as described in claim 6, characterized in that, The distillation gate coefficient is calculated using the following distillation fusion gate formula: in, This is expressed as the distillation gate coefficient. Represented as the first The user on the first The semantic interaction features of comments for each project Represented as the first The user on the first The emotional interaction characteristics of each project Represented as a preset gated transformation weight matrix, Represented as a preset bias vector; The fused features are calculated using the following gated fusion formula: in, Represented as fusion features, This is expressed as the distillation gate coefficient. Represented as the first The user on the first The semantic interaction features of comments for each project Represented as the first The user on the first Emotional interaction characteristics of each project; The adaptive fusion weights are calculated using the following multi-head attention formula: in, Represented as adaptive fusion weights, Represented as fusion features, Represented as a multilayer perceptron, This is represented by the sigmoid activation function.

8. The knowledge distillation recommendation model training method for integrating comments and sentiment according to claim 1, characterized in that, The rating prediction module calculates the rating probability based on the comment semantic interaction features and the sentiment interaction features to obtain the corresponding comment semantic probability distribution and sentiment probability distribution, including: The comment semantic interaction features and the sentiment interaction features are respectively input into the rating prediction module to obtain the first initial predicted rating of the comment semantic interaction and the second initial predicted rating of the sentiment interaction. The first and second initial predicted scores are normalized respectively, and the corresponding comment semantic probability distribution and sentiment probability distribution are obtained.

9. The knowledge distillation recommendation model training method based on the fusion of comments and sentiment as described in claim 1, characterized in that, The step of generating a task loss value based on the rating, the predicted rating, the comment preference score, and the sentiment preference score includes: Calculate the comment semantic branch prediction loss value based on the rating and the comment preference score; Calculate the sentiment branch prediction loss value based on the rating and the sentiment preference score; Calculate the comment fusion prediction loss value based on the stated score and the predicted score; The task loss value is calculated based on the branch prediction loss value, the sentiment branch prediction loss value, and the comment fusion prediction loss value.

10. The knowledge distillation recommendation model training method for integrating comments and sentiment according to claim 1, characterized in that, The predicted user score is calculated using the following weighted fusion formula: in, Represented as the first The user on the first Predicted scores for each project Represented as adaptive fusion weights, Represented as the first The user on the first Comment preference scores for each project Represented as the first The user on the first Sentiment preference scores for each item; The distillation constraint loss value is calculated using the following distillation constraint loss function: in, This is expressed as the distillation constraint loss value. Represented as the first The user on the first The semantic probability distribution of comments for each project Represented as the first The user on the first The sentiment probability distribution of each item It is represented as a divergence function.

Images