A graph neural network recommendation method for review text

Through the graph neural network recommendation method for comment text, a two-part graph between users and products is constructed. Graph convolution and comparison learning technology is used to solve the noise problem in the existing recommendation method, improve the recommendation performance and accuracy, and achieve the improvement Delicate user preference for learning.

CN114723522BActive Publication Date: 2024-11-26HEFEI UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210345556.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-31
Publication Date
2024-11-26
Estimated Expiration
2042-03-31

AI Technical Summary

Technical Problem

The existing comment-based recommendation method ignores the two-part graph model of user interaction with products, resulting in the introduction of noise in node characterization learning, which is difficult to reflect real user preferences and product characteristics, limiting the recommendation effect.

Method used

Using the graph neural network recommendation method for comment text, by constructing a two-part graph of the user and the product, using graph convolution and graph comparison learning technology, extracting scores and comments as interactive features, calculating the representation of users and products, and enhancing the representation robustness. Sex and accuracy.

Benefits of technology

Improve recommendation performance, and improve the accuracy and robustness of score prediction by more accurately capturing the collaborative signals between users and products and delicate user preferences, avoiding noise in node characterization, and enhancing the effect of recommendation system. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114723522B_ABST
    Figure CN114723522B_ABST
Patent Text Reader

Abstract

The present invention discloses a graph neural network recommendation method for review text, including: 1. Using users' comments and ratings on products to construct a bipartite graph of users and products, including: user node set, product node set, rating matrix, and review feature tensor. 2. Constructing a graph convolution method for review text, using comments and ratings as edge features to participate in graph convolution, and encoding user and product representations. 3. Using graph contrast learning to enhance user and product representations. 4. Constructing an interaction layer to encode interaction vectors from user and product representations. 5. Using contrast learning to bring the distribution of interaction representations and comment vectors closer. 6. Predicting ratings based on interaction representations to achieve product recommendations. The present invention uses comments and ratings as interaction features between users and products, adaptively learns the mutual influence between the two, learns more accurate node representations and more delicate user preferences, and thus can improve recommendation performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of personalized recommendation, and specifically to a graph neural network recommendation method for review texts. Background Art

[0002] Personalized recommendation systems analyze user historical behaviors and mine user interests and preferences, thereby recommending products that users are interested in. The large amount of user review information on e-commerce websites helps to further mine user preferences and improve recommendation results. The current mainstream method of using review texts regards historical reviews as description texts of users and products, and extracts semantic features from them to enhance the representation of users and products.

[0003] Although existing review-based recommendation methods have achieved more accurate recommendation results than traditional recommendation models, these methods ignore the fact that user-product interactions can be naturally modeled using a bipartite graph: users and products are nodes, and ratings and review texts reflect the edge features between nodes. From the perspective of a bipartite graph, existing review-based recommendation methods use edge review texts directly as node attributes, which inevitably introduces noise in node representation learning, making it difficult for node representations to reflect real user preferences and product features, thereby limiting the recommendation effect. Summary of the invention

[0004] The present invention aims to solve the shortcomings of the existing review-based recommendation model and propose a graph neural network recommendation method for review text, in order to more fully mine the reviews and achieve more accurate modeling of user and product representations, thereby improving the recommendation performance.

[0005] The present invention adopts the following technical solutions to solve the technical problems:

[0006] The graph neural network recommendation method for comment text of the present invention is characterized in that it is performed according to the following steps:

[0007] Step 1: Construct a bipartite graph of users and products based on users’ ratings and comments on products, including: User set Product Collection Rating Matrix and the comments tensor Among them, represents the i-th user, M represents the user set The total number of users, represents the jth product, N represents the total number of products; the element r in the i-th row and j-th column of the rating matrix R ij represents the i-th user u i For the jth product v j Ratings, represents the set of all rating values; let represents the i-th user ui For the jth product v j The review vector of the user is thus composed of the rating matrix R and the review tensor E, which together form the interaction set ε={R,E} between the user and the product, and the bipartite graph Represents the interaction between users and products;

[0008] Step 2: Perform feature propagation through graph convolution on the comment text to obtain user and product representations:

[0009] Step 2.1: Get the user’s embedding matrix through one-hot encoding and the product embedding matrix And respectively as the initial representation of the corresponding node; among them, the user's embedding matrix The i-th row vector of represents the i-th user u i The representation vector of the product; similarly, the embedding matrix of the product The j-th row vector of represents the jth product v j The embedding vector of

[0010] There are L layers of graph convolution defined;

[0011] Step 2.2: In the lth convolutional layer, use formula (1) to calculate the jth product v j To the i-th user u i The characteristic information propagation vector

[0012]

[0013] In formula (1), represents the jth product v obtained by the l-1th convolutional layer j Characterization vector; when l = 1, let and Respectively represent the j-th product v j and the i-th user u i The neighbor node set of |·| indicates the number of elements in the calculation set; and Represents the score r in the lth convolutional layer ij Specify two parameter vectors, and Represents the score r in the lth convolutional layer ij The two parameter matrices specified; σ(·) represents the Sigmoid activation function; represents transpose;

[0014] Step 2.3: In the lth convolutional layer, use formula (2) to calculate the i-th user ui To the jth product v j The characteristic information propagation vector

[0015]

[0016] In formula (2), represents the i-th user u obtained by the l-1th convolutional layer i Characterize the vector; when l = 1, let

[0017] Step 2.4: Use formula (3) to aggregate the information propagation vector of the lth convolutional layer, so as to obtain the i-th user u i and the jth product v j The representation vector at the lth convolutional layer and

[0018]

[0019] In formula (3), W (l) is the parameter matrix of the lth convolutional layer;

[0020] Step 2.5: After assigning l+1 to l, return to step 2.2 and continue until L layers of graph convolution are completed, and the node representation of the last layer is used as the final user representation. and product characterization

[0021] Step 3: Use graph contrast learning to enhance the representation vectors of users and products:

[0022] Step 3.1: Enhance user representation through contrastive learning:

[0023] For the user-item bipartite graph The product nodes in the graph are randomly discarded with probability ρ for secondary perturbation, thus obtaining two user subgraphs and And process according to the process of step 2.2 to step 2.5, so as to obtain two sets of user representation vectors and Represents the subgraph Get the i-th user u i The representation vector of Represents the subgraph Get the i-th user u i The representation vector of

[0024] According to formula (4), the user's contrastive learning loss function is established

[0025]

[0026] In formula (4), Represents a collection of users Randomly sample user u i Seek expectations; Indicates that the user collection First sample user u i , and then from the user collection Sample user u i' Then find the expectation of the two samples; Indicates calculating the similarity of feature vectors of two nodes;

[0027] Step 3.2: Enhance product representation vector:

[0028] For the user-item bipartite graph According to the probability ρ, user nodes in the graph are randomly discarded for secondary perturbation, thereby obtaining two product subgraphs and processing them according to the process of steps 2.2 to 2.5 to obtain two sets of product representation vectors and Represents the subgraph Get the jth user v j The representation vector of Represents the subgraph Get the jth user v j The representation vector of .

[0029] Use formula (4) to establish the contrastive learning loss function of the product

[0030]

[0031] In formula (4), Represents a collection of products Randomly sample user v j Seek expectations; Indicates that the product collection First sample user v j , and then from the product collection Sample user v j' , find the expectation of two samplings;

[0032] Step 3.3: Use formula (6) to get the contrast loss function of node enhancement

[0033]

[0034] Step 4: Use formula (7) to calculate the i-th user u i and the jth product vj The interaction representation vector h ij :

[0035]

[0036] In formula (7), MLP represents multi-layer perceptron, [,] represents vector concatenation operation;

[0037] Step 5: Calculate the interactive contrast loss according to formula (8)

[0038]

[0039] In formula (8), represents sampling of user-product pairs (u i ,v j ) to calculate the expectation, represents sampling user-product pairs (u i ,v j ), and then sample user-product pairs (u i' ,v j' ) extracts negative sample pairs (i', j') and calculates the expectation; e i′j′ represents the i'th user u i' For the j'th product v j' 's comment vector;

[0040] Step 6: According to formula (9), we can get the i-th user u i For the jth product v j Prediction score of

[0041]

[0042] In formula (9), w is the parameter vector;

[0043] Step 6.1: Calculate the loss function of rating prediction according to formula (10):

[0044]

[0045] In formula (10), represents the set of all user-product pairs;

[0046] Step 6.2: Construct the total loss function according to formula (11)

[0047]

[0048] In formula (11), α and β are hyperparameters used to balance the three loss functions;

[0049] Step 6.3: Use the gradient descent method to optimize all parameters so that the loss function Converge to the minimum, thus obtaining the optimal recommendation model with the smallest prediction score error, which is used to recommend products.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. Aiming at the shortcoming that the existing recommendation model modeling does not conform to the review as an interactive feature, the present invention proposes a graph neural network recommendation method for review text, constructs a user-product bipartite graph through step 1, and uses the review text and rating as the edge features. This modeling method is more consistent with the attributes of the review, thereby avoiding the introduction of irrelevant noise in the node representation.

[0052] 2. The present invention targets the user-product bipartite graph and introduces the rating and comment features on the edges in the graph convolution through step 2 to calculate the node and user representations. This modeling conforms to the actual situation of ratings and comments as user-product interactions. It adaptively learns the mutual influence between products and users through comments and ratings, more accurately captures the collaborative signals between the two, and learns more accurate node representations, thereby improving the recommendation performance.

[0053] 3. The present invention enhances the representation of users and products through the graph contrast learning technology in step 3. Constraining the consistency of node representation under different graph perturbations forces the model to learn the most relevant information about the node from the bipartite graph, which not only alleviates the noise of node representation but also improves the robustness of node representation, thereby effectively improving the accuracy of rating prediction.

[0054] 4. The present invention uses contrastive learning and takes the review text as a supervisory signal. During training, it not only takes the target rating as the training target but also learns more detailed and rich review information. Specifically, the present invention not only learns the user's rating but also learns why the corresponding rating is given, thereby obtaining better model parameters and more accurate recommendation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flowchart of a graph neural network recommendation method for review texts of the present invention. DETAILED DESCRIPTION

[0056] In this embodiment, a graph neural network recommendation method for comment text takes into account the fact that the existing comment-based recommendation model does not model comment text in a way that is appropriate for comments as interactive features. By using comments as edge features in graph convolution to model users and products, comments are used to more accurately calculate user and product representations. On this basis, graph contrastive learning is introduced to enhance the robustness of node representations, and contrastive learning is used to use comments as supervisory signals to guide the model to learn more detailed user preferences, thereby improving recommendation performance. Specifically, Figure 1 As shown, the method is carried out according to the following steps.

[0057] Step 1: Construct a bipartite graph of users and products based on users’ ratings and comments on products, including: User set Product Collection Rating Matrix and the comments tensor Among them, represents the i-th user, M represents the user set The total number of users, represents the jth product, N represents the total number of products; the element r in the i-th row and j-th column of the rating matrix R ij represents the i-th user u i For the jth product v j Ratings, represents the set of all rating values; let represents the i-th user u i For the jth product v j The review vector of the user is thus composed of the rating matrix R and the review tensor E, which together form the interaction set ε={R,E} between the user and the product, and the bipartite graph Represents the interaction between users and products;

[0058] Step 2: Perform feature propagation through graph convolution on the comment text to obtain user and product representations:

[0059] Step 2.1: Get the user’s embedding matrix through one-hot encoding and the product embedding matrix And respectively as the initial representation of the corresponding node; among them, the user's embedding matrix The i-th row vector of represents the i-th user u i The representation vector of the product; similarly, the embedding matrix of the product The j-th row vector of represents the jth product v j The embedding vector of

[0060] There are L layers of graph convolution defined;

[0061] Step 2.2: In the lth convolutional layer, use formula (1) to calculate the jth product v j To the i-th user u i The characteristic information propagation vector

[0062]

[0063] In formula (1), represents the jth product v obtained by the l-1th convolutional layer j Characterization vector; when l = 1, let and Respectively represent the j-th product v j and the i-th user u i The neighbor node set of |·| indicates the number of elements in the calculation set; and Represents the score r in the lth convolutional layer ij Specify two parameter vectors, and Represents the score r in the lth convolutional layer ij The two parameter matrices specified; σ(·) represents the Sigmoid activation function; represents transpose;

[0064] This method effectively integrates the bipartite graph structure and comment information in information dissemination, improving model representation. Specifically, the traditional method uses scores to measure the influence of information transmission vectors, while the present invention uses detailed comments to learn the influence of products on users. Similarly, for comments on the edge, the present invention uses the same method to introduce comment features in the dissemination to enrich information features.

[0065] Step 2.3: In the lth convolutional layer, use formula (2) to calculate the i-th user u i To the jth product v j The characteristic information propagation vector

[0066]

[0067] In formula (2), represents the i-th user u obtained by the l-1th convolutional layer i Characterize the vector; when l = 1, let

[0068] Step 2.4: Use formula (3) to aggregate the information propagation vector of the lth convolutional layer, so as to obtain the i-th user u i and the jth product v j The representation vector at the lth convolutional layer and

[0069]

[0070] In formula (3), W (l) is the parameter matrix of the lth convolutional layer;

[0071] Step 2.5: After assigning l+1 to l, return to step 2.2 and continue until L layers of graph convolution are completed, and the node representation of the last layer is used as the final user representation. and product characterization

[0072] Step 3: Use graph contrast learning to enhance the representation vectors of users and products:

[0073] Step 3.1: Enhance user representation through contrastive learning:

[0074] For the user-item bipartite graph The product nodes in the graph are randomly discarded with probability ρ for secondary perturbation, thus obtaining two user subgraphs and And process according to the process of step 2.2 to step 2.5, so as to obtain two sets of user representation vectors and Represents the subgraph Get the i-th user u i The representation vector of Represents the subgraph Get the i-th user u i The representation vector of .

[0075] According to formula (4), the user's contrastive learning loss function is established

[0076]

[0077] In formula (4), Represents a collection of users Randomly sample user u i Seek expectations; Indicates that the user collection First sample user u i , and then from the user collection Sample user u i' , find the expectation of two samples. Used to calculate the similarity of feature vectors of two nodes.

[0078] This method constrains the representation of the same node to be consistent in different subgraphs. Specifically, this method forces the common information features extracted from different neighbor nodes and comments to more accurately reflect the user's preferences, such as a user's preference for hard science fiction movies. Therefore, this step can effectively improve the accuracy and robustness of node features, thereby improving recommendation performance.

[0079] Step 3.2: Enhance product representation vector:

[0080] For the user-item bipartite graph According to the probability ρ, user nodes in the graph are randomly discarded for secondary perturbation, thereby obtaining two product subgraphs and processing them according to the process of steps 2.2 to 2.5 to obtain two sets of product representation vectors and Represents the subgraph Get the jth user v j The representation vector of Represents the subgraph Get the jth user v j The representation vector of .

[0081] Use formula (4) to establish the contrastive learning loss function of the product

[0082]

[0083] In formula (4), Represents a collection of products Randomly sample user v j Seek expectations; Indicates that the product collection First sample user v j , and then from the product collection Sample user v j' , find the expectation of two samplings.

[0084] Step 3.3: Use formula (6) to get the contrast loss function of node enhancement

[0085]

[0086] Step 4: Use formula (7) to calculate the i-th user u i and the jth product v j The interaction representation vector h ij :

[0087]

[0088] In formula (7), MLP represents multi-layer perceptron, [,] represents vector concatenation operation;

[0089] Step 5: Calculate the interactive contrast loss according to formula (8)

[0090]

[0091] In formula (8), represents sampling of user-product pairs (u i ,v j ) to calculate the expectation, It means that we first sample user-product pairs (u i ,v j ), and then sample user-product pairs (u i' ,v j' ) extracts negative samples to find the expectation of (i',j'); e i′j′ represents the i'th user u i' For the j'th product v j' Comments vector.

[0092] Traditional recommendation models train model parameters with ratings as the goal. This invention introduces comments as the goal through contrastive learning, so that the model not only learns how to accurately predict numerical ratings during training, but also learns the detailed reasons given by users in their comments. Because additional effective supervision signals are provided, the rating prediction performance of the model can be further improved.

[0093] Step 6: According to formula (9), we can get the i-th user u i For the jth product v j Prediction score of

[0094]

[0095] In formula (9), w is the parameter vector;

[0096] Step 6.1: Calculate the loss function of rating prediction according to formula (10):

[0097]

[0098] In formula (10), represents the set of all user-product pairs;

[0099] Step 6.2: Construct the total loss function according to formula (11)

[0100]

[0101] Among them, α and β are hyperparameters used to balance the three loss functions;

[0102] Step 6.3: Use the gradient descent method to optimize all parameters so that the loss function Converge to the minimum, thus obtaining the optimal recommendation model with the smallest prediction score error, which is used to recommend products.

[0103] Experimental example:

[0104] In order to verify the effectiveness of the present method, experiments are conducted on five public datasets commonly used in recommendation systems: Digital Music, Toys and Games, Clothing, CDs and Yelp, and the recommendation performance of the method of the present invention (RGCL) is compared with 10 existing recommendation methods: SVD, NCF, DeepCONN, NARRE, DAML, SDNet, TransNet, GC-MC, RMG and SSG. The MSE commonly used in recommendation systems is used as the evaluation index. The smaller the MSE, the smaller the error of the score prediction and the higher the recommendation accuracy.

[0105] Table 1

[0106] Recommended Method Digital Music Toys and Games Clothing CDs Yelp SVD 0.8523 0.8086 1.1167 0.8662 1.1939 NCF 0.8403 0.8078 1.1094 0.8781 1.1896 DeepCoNN 0.8378 0.8028 1.1184 0.8621 1.1877 NARRE 0.8172 0.7962 1.1064 0.8495 1.1862 DAML 0.8237 0.7936 1.1065 0.8483 1.1793 SDNet 0.8331 0.8006 1.108 0.8654 1.1837 TransNets 0.8273 0.798 1.1141 0.844 1.1855 GC-MC 0.809 0.7986 1.1088 0.8404 1.1737 RMG 0.8074 0.7901 1.1064 0.8425 1.1705 SSG 0.8218 0.8064 1.1228 0.8458 1.1807 RGCL 0.7735 0.7771 1.0858 0.818 1.1609

[0107] Table 1 is a comparison of the recommended effects of the method of the present invention and the comparative method. From the experimental results, it can be seen that the method proposed by the present invention is superior to the existing method in terms of MSE index.

[0108] In summary, this method uses comments and ratings as interactive features between users and products, adaptively learns the mutual influence between the two, learns more accurate node representations and more delicate user preferences, and thus improves recommendation performance.

Claims

1. A graph neural network recommendation method for review text, characterized in that: The steps are as follows: Step 1: Construct a bipartite graph of users and products based on users’ ratings and comments on products, including: User set , Product Collection , Rating Matrix and the comments tensor ; Among them, let Indicates Users, Represents a user collection The total number of users, Indicates products, Represents the total number of products; rating matrix Middle Line Column Elements Indicates Users For Products Ratings, represents the set of all rating values; let Indicates Users For Products The review vector of and the comments tensor Together they form a collection of interactions between users and products , and use the bipartite graph Represents the interaction between users and products; Step 2: Perform feature propagation through graph convolution on the review text to obtain user and product representations: Step 2.1: Get the user’s embedding matrix through one-hot encoding and the product embedding matrix And respectively as the initial representation of the corresponding node; among them, the user's embedding matrix No. Row vector Indicates Users The representation vector of the product; similarly, the embedding matrix of the product No. Row vector Indicates Products The embedding vector of Definition Layer graph convolution; Step 2.2, in In the convolutional layer, formula (1) is used to calculate the Products To Users The characteristic information propagation vector : (1) In formula (1), Indicates The convolutional layer obtains Products The characterization vector of season ; and Respectively represent Products and Users The set of neighbor nodes of Indicates the number of elements in a collection. and Indicated in Scoring in convolutional layer Specify two parameter vectors, and Indicated in Scoring in convolutional layer The two parameter matrices specified; Represents the Sigmoid activation function; represents transpose; Step 2.3, in In the convolutional layer, formula (2) is used to calculate the Users To Products The characteristic information propagation vector : (2) In formula (2), Indicates The convolutional layer obtains Users Characterize the vector; when season ; Step 2.4: Use formula (3) to polymerize The information propagation vector of the convolutional layer is obtained accordingly. Users and Products In the Representation vector of convolutional layer and : (3) In formula (3), For the The parameter matrix of the convolutional layer; Step 2.5: Assign to After that, return to step 2.2 and execute until After the layer graph convolution calculation, the node representation of the last layer is used as the final user representation and product characterization ; Step 3: Use graph contrast learning to enhance the representation vectors of users and products: Step 3.1: Enhance user representation through contrastive learning: For the user-item bipartite graph According to probability Randomly discard product nodes in the graph to perform secondary perturbation, thereby obtaining two user subgraphs and And process according to the process of step 2.2 to step 2.5, so as to obtain two sets of user representation vectors and ; Represents the subgraph Get the Users The representation vector of Represents the subgraph Get the Users The representation vector of According to formula (4), the user's contrastive learning loss function is established : (4) In formula (4), Represents a collection of users Randomly sample users Seek expectations; Indicates that the user collection First sample users , and then from the user collection Sampling users Then find the expectation of the two samples; Indicates calculating the similarity of feature vectors of two nodes; Step 3.2: Enhance product representation vector: For the user-item bipartite graph According to probability Randomly discard the user nodes in the graph to perform secondary perturbation, thereby obtaining two product subgraphs and processing them according to the process of steps 2.2 to 2.5 to obtain two sets of product representation vectors and ; Represents the subgraph Get the Users The representation vector of Represents the subgraph Get the Users The representation vector of Use formula (5) to establish the contrastive learning loss function of the product : (5) In formula (5), Represents a collection of products Randomly sample users Seek expectations; Indicates that the product collection First sample users , and then from the product collection Sampling users , find the expectation of two samplings; Step 3.3: Use formula (6) to get the contrast loss function of node enhancement : (6) Step 4: Use formula (7) to calculate Users and Products The interaction representation vector : (7) In formula (7), MLP represents multi-layer perceptron, Represents a vector concatenation operation; Step 5: Calculate the interactive contrast loss according to formula (8) : (8) In formula (8), Represents the interaction set Sample user-product pairs Calculate the expectation, Indicates that we start with the interaction set Sample user-product pairs , and then from the interaction set Sample user-product pairs Extract negative samples from After that, we seek expectations; Indicates Users For Products 's comment vector; Step 6: According to formula (9), we get Users For Products Prediction score of : (9) In formula (9); is the parameter vector; Step 6.1: Calculate the loss function of rating prediction according to formula (10): : (10) In formula (10), represents the set of all user-product pairs; Step 6.2: Construct the total loss function according to formula (11) : (11) In formula (11), and is a hyperparameter used to balance the three loss functions; Step 6.3: Use the gradient descent method to optimize all parameters so that the loss function Converge to the minimum, thus obtaining the optimal recommendation model with the smallest prediction score error, which is used to recommend products.