A review recommendation method based on time sequence and polarity

Abstract

This invention belongs to the field of information processing technology, specifically relating to a comment recommendation method based on time sequence and polarity. The method includes: acquiring and preprocessing user comments and item comments to obtain user comment embeddings, item comment embeddings, user attribute sets, and item attribute sets; calculating user fusion embeddings based on user comment embeddings and user attribute sets, and calculating item fusion embeddings based on item comment embeddings and item attribute sets; processing the user fusion embeddings and item fusion embeddings separately according to different polarities using a sequence model to obtain user sequence vectors and item sequence vectors; processing the user sequence vectors and item sequence vectors according to different polarities using a cross-attention mechanism to obtain final user embeddings and final item embeddings; fusing the final user embeddings and final item embeddings to obtain a fusion embedding; and inputting the fusion embedding into a fully connected layer for processing to obtain the item recommendation result. This invention has high recommendation accuracy and good application prospects.

Description

Technical Field

[0001] This invention belongs to the field of information processing technology, specifically relating to a comment recommendation method based on time sequence and polarity. Background Technology

[0002] In recent years, review recommendation systems have flourished. Traditional recommendation systems typically model user-product interactions in a static way, capturing only general user preferences. However, in real life, users possess not only behavioral data but also a wealth of textual data such as reviews. This data provides a more granular supplement to user profile modeling. Many state-of-the-art recommendation models consider mining attributes from user reviews to add to them, making the review model more comprehensive. Thus, review recommendation systems have emerged.

[0003] Given its significant practical value, research on review recommendation problems is booming. Regarding the mining of user reviews, traditional recommendation algorithms typically derive user-item representations from a rating matrix linking users and items, leading to well-known algorithms such as collaborative filtering. However, ratings are inherently coarse-grained (e.g., 1 to 5 stars), and this simplicity hinders the depth of user preference learning. Early recommendations relied primarily on matrix factorization or topic modeling. In recent years, with the rapid development of deep learning, its models have increasingly become the preferred choice for processing such data.

[0004] However, existing models fail to consider the balance between the polarity and quantity of user reviews in practice. Review ratings tend to be concentrated between high and low scores, and reviews can exhibit opposite polarities based on ratings. Furthermore, many models neglect the fact that the sequential relationships among multiple user reviews can reflect changes in user preferences. These issues can mislead recommender systems into making incorrect recommendations related to user preference modeling. Based on these problems, this invention proposes a review recommendation method based on time sequence and polarity. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a review recommendation method based on time sequence and polarity. The method includes: acquiring user reviews and item reviews and preprocessing them; inputting the preprocessed data into a trained review recommendation model to obtain item recommendation results.

[0006] The training process of the comment recommendation model includes:

[0007] S1: Obtain user comments and project comments and preprocess them to obtain user comment embeddings, project comment embeddings, user attribute sets, and project attribute sets;

[0008] S2: Calculate the user fusion embedding based on the user comment embedding and the user attribute set; calculate the project fusion embedding based on the project comment embedding and the project attribute set.

[0009] S3: Use a sequence model to process the user fusion embedding and item fusion embedding according to different polarities to obtain user sequence vectors and item sequence vectors;

[0010] S4: Use cross-attention mechanism to process user sequence vectors and item sequence vectors according to different polarities to obtain the final user embedding and the final item embedding;

[0011] S5: Merge the final user embedding and the final item embedding to obtain the merged embedding; input the merged embedding into the fully connected layer for processing to obtain the item recommendation result;

[0012] S6: Calculate the total loss of the model and adjust the model parameters based on the total loss to obtain a well-trained comment recommendation model.

[0013] Preferably, the process of calculating user fusion embedding based on user comment embedding and user attribute set includes:

[0014] The attention weight vector is obtained by linearly mapping the user attribute vector in the user attribute set and then using the softmax function.

[0015] The attention weight vector and user attribute vector are weighted and fused, and the fused parameters are linearly mapped to obtain the weighted user attribute embedding.

[0016] Adding weighted user attribute embeddings to user comment embeddings results in user fusion embeddings.

[0017] Preferably, the process of obtaining the user sequence vector includes:

[0018] User fusion embeddings are divided into active fusion vectors and passive fusion vectors; both types of vectors are position-encoded, and the position codes are added to the corresponding vectors.

[0019] The two types of vectors with added position encoding are input into the sequence model Transformer to obtain the user sequence vectors.

[0020] Preferably, the process of obtaining the final user embedding includes:

[0021] User sequence vectors are divided into positive sequence vectors and negative sequence vectors; the attention weights of negative sequence vectors on positive sequence vectors and the weights of positive sequence vectors on negative sequence vectors are calculated respectively.

[0022] Calculate the negative collaboration vector based on the attention weights of the positive sequence vectors on the negative sequence vectors and the negative sequence vectors; calculate the positive collaboration vector based on the attention weights of the negative sequence vectors on the positive sequence vectors and the positive sequence vectors.

[0023] Calculate the negative embedding of the user based on the negative collaboration vector and the negative sequence vector; calculate the positive embedding of the user based on the positive collaboration vector and the positive sequence vector.

[0024] By combining negative and positive user embeddings, the final user embedding is obtained.

[0025] Furthermore, the formula for calculating the attention weights of the positive sequence vector on the negative sequence vector is as follows:

[0026]

[0027] Where, α i,j This represents the attention weight of the positive sequence vector i on the negative sequence vector j. Let j represent the negative sequence vector. Let i represent the positive sequence vector, and n represent the number of positive sequence vectors.

[0028] Furthermore, the formula for calculating the negative cooperation vector is:

[0029]

[0030] in, α represents the negative cooperative vector of negative sequence vector j. i,j This represents the attention weight of the positive sequence vector i on the negative sequence vector j. Let j represent the negative sequence vector, and n represent the number of positive sequence vectors.

[0031] Furthermore, the formula for calculating user negative embedding is:

[0032]

[0033] in, This represents the negative embedding of user u. This indicates negative embedding by intermediate users. Let j represent the negative sequence vector. δ represents the negative cooperative vector of the negative sequence vector, m represents the negative weight, and m represents the number of negative sequence vectors.

[0034] Furthermore, the formula for integrating negative and positive user embeddings is as follows:

[0035]

[0036] Among them, M uThis represents the final user embedding, where n represents the number of positive sequence vectors and m represents the number of negative sequence vectors. This represents the negative and positive embedding of user u. This represents the negative embedding of user u.

[0037] Preferably, the formula for calculating the total loss of the model is:

[0038]

[0039] Where N represents the logarithm of user comments to project comments, r u,i This represents the actual rating that user u gave to item i. This represents the predicted score that user u would give to item i, as output by the model; λ1 and λ2 represent the first and second hyperparameters, respectively; W a W b and W c Indicates the first, second, and third training weights.

[0040] The beneficial effects of this invention are as follows: It combines user comment embeddings and multiple comment attribute embeddings through a comment attribute attention module; it utilizes a serialization module to learn the sequence information of comments under different polarities; to address the problem of imbalanced polarity in the number of comments, a polarity balancing module is designed, and polarity balancing is achieved based on comment vectors of opposite polarities through a collaborative attention mechanism; finally, it uses updated user and item embeddings to generate recommendation scores; compared with existing technologies, this invention solves the problem of imbalanced polarity that existing comment-based recommendation systems ignore, and effectively captures changes in user preferences by combining the temporal features of comment text, thus effectively improving recommendation accuracy. Attached Figure Description

[0041] Figure 1 This is a flowchart of the training process for the comment recommendation model in this invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] This invention proposes a comment recommendation method based on time sequence and polarity, the method comprising the following:

[0044] We acquire and preprocess user reviews and project reviews, then input the preprocessed data into a trained review recommendation model to obtain project ratings, which are the project recommendation results.

[0045] like Figure 1 As shown, the training process of the comment recommendation model includes:

[0046] S1: Obtain user comments and project comments and preprocess them to obtain user comment embeddings, project comment embeddings, user attribute sets, and project attribute sets.

[0047] Given user u i (u i ∈U), with user u i Interactive project v j (v j ∈V), obtain the corresponding user comments and project comments. The preprocessing process for user comments and project comments includes:

[0048] Since this paper only uses direct embedding of comment text to complete subsequent recommendation tasks, it does not require strict distinction between different paragraphs and sentences. Therefore, each user and item comment text only needs to be concatenated with a special label [CLS], and sliced ​​or zero-paded according to the set text length to obtain the input sequence for BERT. This sequence is then fed into the BERT model to obtain user comment embeddings and item comment embeddings. Taking user comment embedding as an example:

[0049] t u,i =BERT([[CLS];w1,w2,…,w d ])

[0050] Among them, t u,i w represents the i-th comment by user u. d This represents the d-th word element in a user's comment.

[0051] Comment attribute mining is performed on different comments to generate a set of comment attributes corresponding to each comment (a set of user attributes and a set of project attributes); taking the set of user comment attributes as an example, it can be represented as: The number of vectors in the user comment attribute vector set is the same as the size of the user comment vector set. For each user comment attribute vector, preferably, nine attributes about the comment are obtained through data preprocessing: comment timestamp, comment length, number of sentiment words, proportion of positive sentiment words, proportion of negative sentiment words, average length of user comments, rating, number of likes, and sentiment analysis value. The value of each attribute is normalized and mapped to a range of 0 to 1. Each attribute vector is a 9-dimensional vector representing these nine comment attributes, which can be represented as follows:

[0052] S2: Calculate the user fusion embedding based on the user comment embedding and the user attribute set, and calculate the project fusion embedding based on the project comment embedding and the project attribute set.

[0053] The user fusion embedding is calculated based on user comment embeddings and user attribute sets, specifically:

[0054] For the i-th comment, firstly through W a Its attribute vector a u,i Mapping to the same dimensional space as the BERT embedding, and then applying the softmax function to obtain the attention weight vector f. u,i :

[0055] f u,i =softmax(W a ·a u,i )

[0056] Specifically, the softmax function is applied to each element of the vector to normalize the weights, so that the sum of the weights of all attributes is 1. u,i It is a vector of length 9, where each element represents the attention weight of the attribute corresponding to the current position. Specifically, f u,i,j The j-th element has the following relationship:

[0057]

[0058] Among them, (W) a ·a u,i ) j This represents the j-th element of the vector after matrix multiplication.

[0059] The attention weight vector and user attribute vector are weighted and fused together, W b (W b ∈R dk The fused parameters are mapped to the same dimensional space as the BERT embedding, which is like a W... a The learnable parameter matrices are identical, and their sizes are exactly the same. However, W a It is mainly used to perform dimensionality transformation of attribute vectors to calculate softmax weight scores, while W b The main task is to perform dimensional transformation on the rating matrix in order to learn how to better integrate it with the original review vectors.

[0060] a′ u,i =W b ·(a u,i ⊙f u,i )

[0061] Where ⊙ represents element-wise multiplication (Hadamard product), a′ u,iThis indicates weighted user attribute embedding, a′ u,i It is a with t u,i Vectors of the same dimension, containing attribute information weighted based on attention weights.

[0062] Adding the weighted user attribute embedding to the user comment embedding yields the user fused embedding, which adds information about the corresponding comment attribute i:

[0063] t′ u,i =t u,i +a′ u,i

[0064] The final user fusion embedding set T′ u Represented as:

[0065]

[0066] The method for calculating project fusion embedding based on project comment embedding and project attribute set is the same as the method for calculating user fusion embedding described above, and will not be repeated here.

[0067] S3: Use a sequence model to process the user fusion embedding and item fusion embedding according to different polarities to obtain user sequence vectors and item sequence vectors.

[0068] The process of obtaining the user sequence vector includes:

[0069] User fusion embedding is divided into active fusion vectors. and negative fusion vector Both vectors are positionally encoded, and the positional encoding is added to the corresponding vectors. Specifically:

[0070] The comments are divided into two groups: positive comments and negative comments (1-2 points for negative, 3-5 points for positive). The goal is to input both positive and negative comments as sequences into the Transformer, allowing the comment vectors to capture additional information about the sequence. However, the Transformer model itself does not directly process sequence order information, thus requiring positional encoding to supplement this. In the previous layer, we mentioned that we had already extracted the comment attributes for each comment, normalized and standardized the timestamps of each comment, and integrated them into each comment using an attention mechanism. Therefore, it is unnecessary to use specific time node information to fuse them back into the comment vectors. This invention will employ absolute positional encoding to encode the recall vectors within the sequence.

[0071] Absolute position encoding is typically calculated using sine and cosine functions, resulting in a unique encoded vector for each position. Positive and negative comments are processed identically; here, we take positive comments as an example, for position (p) and dimension (2k) or (2k+1) (k ranging from 0 to...). ), Location code PE pos(i) The calculation formula is as follows:

[0072]

[0073] Among them PE (pos,2k) and PE (pos,2k+1) These represent the positional encoding values ​​of position (p) in dimensions (2k) and (2k+1), respectively.

[0074] Add position encoding to the corresponding embedding:

[0075] x u,i =t′ u,i +PE pos(i)

[0076] Where, x u,i This indicates that the location encoding information will be added to T′. u The vector following the i-th comment vector. Here, after processing all comments, we obtain a vector with a count of |T′. u A set of vectors, defining these sequence of comment sets with added position encoding as...

[0077] The two vectors after adding position encoding The inputs are fed into the sequence model Transformer, resulting in a user sequence vector whose size and dimensions remain unchanged.

[0078] V u =transformer(X) u )

[0079]

[0080] The process of obtaining the project sequence vector is the same as the method for obtaining the user sequence vector, and will not be repeated here.

[0081] S4: Use cross-attention mechanism to process user sequence vectors and item sequence vectors according to different polarities to obtain the final user embedding and the final item embedding.

[0082] For a user sequence vector, assuming the user has n positive reviews and m negative reviews, it is divided into positive sequence vectors. and negative sequence vector

[0083] The attention weights of negative sequence vectors on positive sequence vectors and positive sequence vectors on negative sequence vectors are calculated separately; the calculation methods for both attention weights are the same. Specifically, when calculating the attention weights of positive sequence vectors on negative sequence vectors, a cross-attention mechanism is used to establish a linear relationship between the user's n positive comment vectors and each negative comment vector, and a softmax-like operation is used to perform the weighted calculation. The formula is as follows:

[0084]

[0085] Where, α i,j This represents the attention weight of the positive sequence vector i on the negative sequence vector j. Let j represent the negative sequence vector. Let i represent the positive sequence vector, and n represent the number of positive sequence vectors.

[0086] The negative collaboration vector is calculated based on the attention weights of the positive sequence vector on the negative sequence vector and the negative sequence vector itself.

[0087]

[0088] in, α represents the negative cooperative vector of negative sequence vector j. i,j This represents the attention weight of the positive sequence vector i on the negative sequence vector j. Let j represent the negative sequence vector, and n represent the number of positive sequence vectors.

[0089] The above operation completes the mapping from the user's positive preference direction embedding space to the user's negative preference direction embedding space, similar to a negative collaborative vector. This invention also uses similar operations to generate positive cooperation vectors.

[0090] The negative embedding of a user is calculated based on the negative collaboration vector and the negative sequence vector, while the positive embedding is calculated based on the positive collaboration vector and the positive sequence vector. The calculation methods for both parameters are similar. The formula for calculating the negative embedding is as follows:

[0091]

[0092] in, This represents the negative embedding of user u. δ represents the negative embedding of intermediate users; δ represents the negative weight, which is used to control the influence between the original vector and the cooperative vector; m represents the number of negative sequence vectors.

[0093] Computing user-positive embedding The formula is:

[0094]

[0095] The user's final embedding vector needs to be fused from two polarity embedding vectors. The fusion is weighted based on the number of comment polarities to obtain the final user embedding:

[0096]

[0097] Among them, M u This represents the final user embedding, where n represents the number of positive sequence vectors and m represents the number of negative sequence vectors. This represents the negative and positive embedding of user u. This represents the negative embedding of user u.

[0098] The final embedded M of the project is obtained i The method described above is the same as the method used to obtain the final embedded M by the user. u The method is the same, so it will not be repeated here.

[0099] S5: Merge the final user embedding and the final project embedding to obtain the merged embedding; input the merged embedding into the fully connected layer for processing to obtain the project recommendation result.

[0100] The final embedding and the item's final embedding are concatenated, and a fully connected layer is used to perform a linear transformation to obtain the predicted score for the item:

[0101]

[0102] in, W represents the predicted rating that user u would give to item i, as shown in the model output. c b represents the weight of the fully connected layer. c Denotes the bias, where W c and b c These are all hyperparameters that can be learned.

[0103] S6: Calculate the total loss of the model and adjust the model parameters based on the total loss to obtain a well-trained comment recommendation model.

[0104] In the comment recommendation model, key attribute information of a single comment is integrated into the comment itself, while also considering information features from multiple comment sequences. Furthermore, a cross-attention module is used to address the polarity imbalance problem among comments; thus, the objective function can be defined as:

[0105]

[0106] That is, the total loss of the model is:

[0107]

[0108] Where N represents the logarithm of user comments to project comments, r u,i This represents the actual rating that user u gave to item i. This represents the model output's predicted rating for item i that user u would give; minimizing the objective function allows for efficient updating of all model parameters via gradient descent. The parameters involved include the parameter matrix W from the comment attribute focus layer. a This matrix is ​​used to map attribute vectors to the same space as comment vectors. And the parameter matrix W comes from the comment attribute focus layer. b This is used to learn the fusion of the original comment vector and the comment vector after adding attribute vectors. The final parameter matrix W c Derived from the polarity balancing layer, it is used to learn the predicted score after linear transformation through the fully connected layer. ‖·‖1 and ‖·‖2 represent the regularization norms of l1 and l2, respectively, and have corresponding hyperparameters λ1 and λ2. The regularization operation is to achieve better generalization and prevent uncontrollable values ​​of these parameters and overfitting.

[0109] Backward gradient propagation is performed based on the total model loss to adjust the model parameters. Training stops when the loss function converges or reaches the maximum preset number of iterations, and the model parameters are saved, resulting in a trained review recommendation model. User reviews and item reviews are acquired and preprocessed. The preprocessed data is then input into the trained review recommendation model to obtain predicted ratings for items. Item recommendations are then made based on these predicted ratings. For example, suitable products can be recommended to users based on their past reviews and product reviews.

[0110] In summary, this invention uses the Transformer sequence model to process the sets of positive and negative user reviews and project reviews respectively, obtaining review vectors with sequence information. Positive and negative reviews are input into a pre-designed polarity balancing module. Each review is combined with all reviews of different polarities to generate a co-vector of that review vector, which is then fused to supplement the information transmitted by opposite polarities. Finally, the user's positive and negative review vectors are fused using a neural network, and the same process is applied to project reviews, obtaining the final user review vector and project review vector. A fully connected layer is then used to obtain the predicted rating. This invention addresses the issue of balancing the polarity and quantity of user reviews, improving the model's recommendation accuracy and demonstrating promising application prospects.

[0111] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A review recommendation method based on time sequence and polarity, characterized in that, include: Obtain user reviews and project reviews and preprocess them. Then, input the preprocessed data into the trained review recommendation model to obtain project recommendation results. The training process of the comment recommendation model includes: S1: Obtain user comments and project comments and preprocess them to obtain user comment embeddings, project comment embeddings, user attribute sets, and project attribute sets; S2: Calculate the user fusion embedding based on the user comment embedding and the user attribute set; calculate the project fusion embedding based on the project comment embedding and the project attribute set. S3: Using a sequence model, user fusion embeddings and item fusion embeddings are processed separately according to different polarities to obtain user sequence vectors and item sequence vectors; the process of obtaining user sequence vectors includes: User fusion embeddings are divided into active fusion vectors and passive fusion vectors; both types of vectors are position-encoded, and the position codes are added to the corresponding vectors. The two types of vectors with added position encoding are input into the sequence model Transformer to obtain the user sequence vector; S4: Use a cross-attention mechanism to process the user sequence vector and item sequence vector according to different polarities to obtain the final user embedding and the final item embedding; the process of obtaining the final user embedding includes: User sequence vectors are divided into positive sequence vectors and negative sequence vectors; the attention weights of negative sequence vectors on positive sequence vectors and the weights of positive sequence vectors on negative sequence vectors are calculated respectively. Calculate the negative collaboration vector based on the attention weights of the positive sequence vectors on the negative sequence vectors and the negative sequence vectors; calculate the positive collaboration vector based on the attention weights of the negative sequence vectors on the positive sequence vectors and the positive sequence vectors. Calculate the negative embedding of the user based on the negative collaboration vector and the negative sequence vector; calculate the positive embedding of the user based on the positive collaboration vector and the positive sequence vector. By combining negative and positive user embeddings, the final user embedding is obtained; S5: Merge the final user embedding and the final item embedding to obtain the merged embedding; input the merged embedding into the fully connected layer for processing to obtain the item recommendation result; S6: Calculate the total loss of the model and adjust the model parameters based on the total loss to obtain a well-trained comment recommendation model.

2. The comment recommendation method based on time sequence and polarity according to claim 1, characterized in that, The process of calculating user fusion embeddings based on user comment embeddings and user attribute sets includes: The attention weight vector is obtained by linearly mapping the user attribute vector in the user attribute set and then using the softmax function. The attention weight vector and user attribute vector are weighted and fused, and the fused parameters are linearly mapped to obtain the weighted user attribute embedding. Adding weighted user attribute embeddings to user comment embeddings results in user fusion embeddings.

3. The comment recommendation method based on time sequence and polarity according to claim 1, characterized in that, The formula for calculating the attention weights of the positive sequence vector on the negative sequence vector is: ； in, This represents the attention weight of the positive sequence vector i on the negative sequence vector j. Let j represent the negative sequence vector. Represents the positive sequence vector i, This represents the number of positive sequence vectors.

4. The comment recommendation method based on time sequence and polarity according to claim 1, characterized in that, The formula for calculating the negative cooperation vector is: ； in, This represents the negative cooperation vector of the negative sequence vector j. This represents the attention weight of the positive sequence vector i on the negative sequence vector j. Let j represent the negative sequence vector. This represents the number of positive sequence vectors.

5. The comment recommendation method based on time sequence and polarity according to claim 1, characterized in that, The formula for calculating negative user embeddings is: ； ； in, This represents the negative embedding of user u. This indicates negative embedding by intermediate users. Let j represent the negative sequence vector. The negative cooperation vector represents the negative sequence vector. Indicates negative weight. This represents the number of negative sequence vectors.

6. The comment recommendation method based on time sequence and polarity according to claim 1, characterized in that, The formula for combining negative and positive user embeddings is: ； in, This indicates that the user ultimately embeds it. Indicates the number of positive sequence vectors. This represents the number of negative sequence vectors. This represents the negative and positive embedding of user u. This represents the negative embedding of user u.

7. The comment recommendation method based on time sequence and polarity according to claim 1, characterized in that, The formula for calculating the total loss of the model is: ； in, This represents the logarithm of user comments versus project comments. This represents the actual rating that user u gave to item i. This indicates the model output's predicted rating for item i that user u would give; , Indicates the first and second hyperparameters; , and This represents the first, second, and third training weights.

Images