A recommendation method integrating knowledge graph and sentiment analysis

Abstract

The present invention discloses a recommendation method integrating knowledge graph and sentiment analysis, the method comprising the following steps: constructing a data set, processing comment text data in the data set to obtain triples, processing the triples by linear transformation to obtain user embedding vectors and project embedding vectors, encoding the comment text data by a pre-trained language learning model to obtain a probability distribution value of sentiment analysis, fusing the user embedding vector and the project embedding vector, obtaining a contextual representation of the comment text data and connecting it with the corresponding probability distribution value of sentiment analysis to obtain a matching score between the user and the project, performing matching prediction verification on the matching score between the user and the project, and obtaining the accuracy of the matching score between the user and the project. The present invention introduces sentiment analysis, which can help identify the user's emotional attitude towards a project or content, thereby more accurately capturing the user's preference and improving the matching degree and user satisfaction of the recommendation.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and specifically to a recommendation method integrating knowledge graph and sentiment analysis. Background Art

[0002] As a structured semantic knowledge base, the knowledge graph stores a large amount of information in the form of entity and relationship triples, providing rich background knowledge for the recommendation system. Knowledge graph embedding technologies, such as knowledge graph embedding TransE and knowledge graph embedding TransR, effectively capture the semantic relationship between entities by mapping entities and relationships into continuous vector spaces, and have been widely used in recommendation systems to enhance the understanding of complex user interests and item attributes.

[0003] Attention Mechanism is a key technology that enables the model to dynamically allocate attention based on the importance of input elements. By calculating scores, assigning weights, and aggregating information, the attention mechanism improves the representation of relationships between users and items in the recommendation system, allowing the model to focus on the most critical parts of the data for the recommendation task.

[0004] In the recommendation system that integrates knowledge graphs, knowledge graph embedding is combined with attention mechanism to improve the recommendation effect. Although they are effective, they still have some inherent limitations. Traditional knowledge graph embedding recommendation methods mainly focus on structured knowledge, but user preferences and item features in recommendation systems often contain a lot of unstructured information (such as user comments, item descriptions, etc.), which may not be fully mined and utilized in traditional knowledge graph embedding, resulting in poor performance in actual recommendation tasks.

[0005] Sentiment analysis is a key technology in recommendation systems. By integrating sentiment analysis, recommendation models can gain in-depth insights into users' attitudes and emotions toward products or services. This in-depth understanding helps recommendation systems more accurately capture user preferences and sentiment trends, and thus provide more customized recommendation results. Using sentiment analysis, recommendation models can filter out products or content with negative sentiments, and prioritize items with high positive reviews, thereby improving the overall effectiveness of recommendations.

[0006] However, the accuracy of sentiment analysis is limited by the performance of the algorithm itself. If the accuracy of the sentiment analysis model is not high, the quality of the recommendations that rely on its results will be affected accordingly. Sentiment analysis models may find it difficult to fully grasp the complex language features in comments, such as context, sarcasm, and puns, which may cause the recommendation system to misunderstand the user's true intentions. When dealing with the cold start problem of new users or new projects, due to the limited amount of available data, effective sentiment analysis and knowledge graph embedding may not be possible, which will also affect the accuracy of the recommendation system. Each user or project will obtain the emotional features contained in its comments by processing the comment text, but the comments contain not only emotional information, but also the user's personal preference information, and not every comment information is equally important to the user, so sentiment analysis cannot be used directly.

[0007] In order to overcome the above defects and shortcomings, the present invention proposes a recommendation method that integrates knowledge graph and sentiment analysis. The system makes more comprehensive use of structured and unstructured information, improves the accuracy of sentiment analysis and the efficiency of the recommendation system, and effectively solves the cold start problem, providing users with more accurate and personalized recommendation services. Summary of the invention

[0008] In view of the deficiencies in the prior art, the present invention provides a method for solving the problems mentioned in the background technology.

[0009] To achieve the above object, the present invention provides the following technical solution: a recommendation method integrating knowledge graph and sentiment analysis, comprising the following steps:

[0010] Step S1: construct a data set, process missing values ​​and standardized comment text data in the data set, extract entities and relations from the processed data set, and convert entities and relations into triples;

[0011] Step S2: linear transformation is used to map the triples to different vector spaces to obtain the scoring function of the triples, the scoring function of the triples is trained using the knowledge graph embedding learning model to obtain the features of the triples, and the features of the triples are calculated and filtered to obtain the filtered attention score values, and the filtered attention score values ​​are processed using the multi-head self-attention mechanism to obtain the user embedding vector and the item embedding vector;

[0012] Step S3: converting the comment text data into a pre-trained word unit embedding vector through a pre-trained language learning model, combining the word unit embedding vector with the paragraph identification vector and the position identification vector corresponding to the word unit embedding vector, constructing a comprehensive vector of the input sequence, inputting the comprehensive vector into the pre-trained language learning model for processing, obtaining a vector of contextual semantic information, inputting the vector of contextual semantic information into a bidirectional long short-term memory network for processing, and obtaining a probability distribution value of sentiment analysis;

[0013] Step S4: The user embedding vector and the project embedding vector are interactively fused by dot product, and the result of the dot product fusion is used as a query in the multi-head self-attention mechanism to extract the contextual representation of the comment text data in the interaction between the user embedding vector and the project embedding vector; the contextual representation of the comment text data is connected with the corresponding probability distribution value of the sentiment analysis to obtain the sentiment connection value, and based on the sentiment connection value, a multi-head attention mechanism is used to calculate the importance of the contextual representation of the comment text data in the interaction to the user, and the attention head merging result is obtained, and the attention head merging result is processed by a normalization function to obtain the sentiment feature embedding of the user or project, and the matching score of the user and the project is obtained by analyzing the sentiment feature embedding of the user or project;

[0014] Step S5: construct a recommendation module; perform matching prediction verification on the matching scores of the user and the project in step S4 through the recommendation module to obtain the accuracy of the matching scores of the user and the project.

[0015] Furthermore, in step S1, entities and relations are converted into triples. The specific process is as follows:

[0016] The data on the interactions between users and projects are constructed into a dataset, and the dataset is processed to remove invalid data, process missing values, and standardize comment text data. Entities and relations are extracted from the processed dataset and converted into triples.

[0017] Furthermore, users and items are converted into entities in triples, and the interaction between users and items is defined as the relationship in the triples; the triples are composed of the head entity, the tail entity, and the relationship between the head entity and the tail entity; for the entity matrix , Represents the form in which entity f is stored in the entity matrix. Entity f includes the head entity and the tail entity; the relationship matrix R , Represents the form in which the relation r is stored in the relation matrix;

[0018] The triple The head entity h, tail entity t and relation r in are obtained by random initialization. , , ; , , They represent the head entity embedding vector, the tail entity embedding vector, and the relationship embedding vector respectively.

[0019] Further, in step S2, the user embedding vector and the item embedding vector are obtained. The specific process is as follows:

[0020] Step S21: Use linear transformation to map the head entity embedding vector, tail entity embedding vector and relationship embedding vector in the triple to different vector spaces through the knowledge graph embedding model to obtain the scoring function of the triple, which is expressed as:

[0021] (1);

[0022] In the formula, Represents the scoring function calculation for triples; A weight matrix representing a linear transformation of a relationship; Represents the square operation of the L2 norm;

[0023] Step S22: Then use the knowledge graph embedding learning model to train the scoring function of the triples, expressed as:

[0024] (2);

[0025] In the formula, Represents the training loss function of the knowledge graph embedding learning model; Represents a triple Negative samples of Indicates that an entity different from h is randomly selected from the entity matrix as the head entity of the negative sample; Randomly select entities different from t from the entity set representing the entity matrix as the tail entity of the negative sample; Represents the hyperparameter that controls the interval; True refers to the valid triple set that actually exists in the knowledge graph; max means taking the largest value of two numbers;

[0026] After the training process of step S22, the knowledge graph embedding learning model adjusts the head entity embedding vector, the tail entity embedding vector and the relationship embedding vector, and finally the head entity embedding vector, the tail entity embedding vector and the relationship embedding vector tend to be stable, forming representative adjusted entity and relationship embedding vectors; and finally outputs the adjusted entity embedding vector matrix , Represents the form in which the adjusted entity f embedding vector is stored in the adjusted entity embedding vector matrix; the adjusted entity f embedding vector includes the adjusted head entity embedding vector and the adjusted tail entity embedding vector; the adjusted relationship embedding vector matrix , The adjusted relation r embedding vector is represented by the form in which the adjusted relation embedding vector is stored in the matrix;

[0027] Step S23: concatenate the adjusted head entity embedding vector, the adjusted tail entity embedding vector, and the adjusted relationship embedding vector to calculate the features of the triple, expressed as:

[0028] (3);

[0029] In the formula, Represents the characteristics of a triple; A weight matrix representing a trained linear transformation matrix; , and They represent the adjusted head entity h embedding vector, the adjusted relation r embedding vector and the adjusted tail entity t embedding vector respectively;

[0030] Step S24: In the features of the triplet, the adjusted embedding vector of the head entity h has multiple neighbor entities, and the attention score value of the triplet formed by the adjusted embedding vector of the head entity h and the neighbor entities is calculated, which is expressed as:

[0031] (4);

[0032] In the formula, Represents a triple The attention score value; represents the activation function to introduce nonlinear elements; The weight matrix representing the learned linear transformation;

[0033] Step S25: Filter the attention score values ​​of the triples, and then normalize the attention score values ​​of the filtered triples to obtain the filtered attention score values, which are expressed as:

[0034] (5);

[0035] In the formula, Represents the filtered attention score value; exp means converting the original score into a positive number; represents the set of all neighbor entities of the adjusted embedding vector of the head entity h; v represents a neighbor entity of the adjusted embedding vector of the entity h; P represents the set of all relations between the adjusted embedding vector of the head entity h and the neighbor entities; u represents a relation embedding vector between the adjusted embedding vector of the entity h and the neighbor entity; represents the attention score threshold; Represents a triple The attention score value;

[0036] Step S26: Use M independent attention heads in the multi-head self-attention mechanism to capture different features and relationships in the filtered attention score values ​​respectively; by merging the outputs of the M independent attention heads, a comprehensive head entity is obtained, which is expressed as:

[0037] (6);

[0038] In the formula, Represents a comprehensive header entity; represents the activation function; Represents the neighbor entity of the adjusted head entity h; Represents the feature representation value in the Mth independent attention head; represents the attention score value in the Mth independent attention head; m represents the Mth independent attention head;

[0039] Step S27: A multi-layer information propagation mechanism is used to learn the high-order neighbor information of the comprehensive head entity; the first layer of information propagation focuses on the comprehensive head entity and the directly connected entities, and the second layer of information propagation is based on the first layer and expands the learning of entities adjacent to the comprehensive head entity and the neighbor entities; in each layer of information propagation, the attention mechanism is used to perform weighted aggregation of the information of the neighboring entities and update the representation of the comprehensive head entity;

[0040] The concatenation operation is used to aggregate the comprehensive header entities of all layers of information propagation in the multi-layer information propagation mechanism into a single vector, which is expressed as:

[0041] (7);

[0042] In the formula, represents a single vector; represents the obtained first-layer comprehensive head entity embedding; represents the obtained comprehensive head entity embedding at the Zth level; and Represent the weight matrix and bias term of the trained linear transformation respectively;

[0043] Step S28: Use a multi-layer information propagation mechanism to train users and items in the same training method as step S27 to obtain user embedding vector U and item embedding vector I.

[0044] Further, the probability distribution value of sentiment analysis in step S3, the specific process is:

[0045] First, the pre-trained language learning model converts the review text data into pre-trained word embedding vectors; the word embedding vectors are combined with the paragraph identification vectors and position identification vectors corresponding to the word embedding vectors to construct a comprehensive vector of the input sequence;

[0046] The comprehensive vector is input into the multi-layer transformer network of the embedding layer of the pre-trained language learning model. Each layer of the multi-layer transformer network conducts in-depth analysis of the relationship between the comprehensive vectors through a multi-head self-attention mechanism. After continuous processing by the multi-layer transformer network, a vector containing rich contextual semantic information is generated for each comprehensive vector in the input sequence, which is expressed as:

[0047] (8);

[0048] In the formula, A vector representing contextual semantic information; It means that it has been processed by a multi-layer converter network; represents the comprehensive vector representation of the input;

[0049] Pass the vector of contextual semantic information to a bidirectional long short-term memory network; the bidirectional long short-term memory network consists of forward and backward long short-term memory network units;

[0050] The forward LSTM network unit processes the input sequence from the start to the end, while the backward LSTM network unit processes from the end to the start. The forward and backward LSTM network units process the vector of contextual semantic information through the input gate, forget gate, output gate, and maintenance and update of the cell state to obtain the hidden state of the current time step a of the bidirectional LSTM network, which is expressed as:

[0051] (9);

[0052] In the formula, represents the hidden state of the current time step a; represents the output gate; represents the hyperbolic tangent activation function; Indicates cell status;

[0053] The fully connected layer is used to linearly transform the hidden state of the current time step a of the bidirectional long short-term memory network, which is expressed as:

[0054] (10);

[0055] In the formula, Represents the output vector of the fully connected layer; and Represent the weight matrix weights and bias terms of the linear transformation of the fully connected layer respectively;

[0056] Then use the activation function to map the output vector of the fully connected layer into the probability distribution value of sentiment analysis , expressed as:

[0057] (11);

[0058] In the formula, and Represent the weight matrix and bias term of the linear transformation of the output layer respectively; Represents the activation function.

[0059] Further, in step S4, the matching score between the user and the project is obtained, and the specific process is as follows:

[0060] The user embedding vector U and the item embedding vector I interact and fuse them through the dot product method. The result of the dot product fusion is used as the query in the multi-head self-attention mechanism. The context representation of the i-th comment text data in the interaction between the user embedding vector U and the item embedding vector I is taken out and recorded as q; the context representation q of the i-th comment text data is compared with the probability distribution value of the corresponding sentiment analysis Connect them together to get the emotional connection value B, expressed as:

[0061] (12);

[0062] In the formula, is the function used to connect the dot product result with the probability distribution value of sentiment analysis; Represents prime-level multiplication; Represents the concatenation operation of vectors;

[0063] After obtaining the sentiment connection value B, a multi-head attention mechanism is used to calculate the importance of the contextual representation of the i comment text data in the interaction to the user, and obtain the query ,key Sum , expressed as:

[0064] (13);

[0065] (14);

[0066] (15);

[0067] In the formula, Generate query Represents the weight matrix of the learned linear transformation; Indicates the generated key Represents the weight matrix of the learned linear transformation; Represents the generated value represents the weight matrix of the learned linear transformation; , and Respectively represent queries ,key Sum The bias term of

[0068] Using Query ,key Sum Calculate the attention score and get the attention head merging result, expressed as:

[0069] (16);

[0070] In the formula, Indicates the result of attention head merging; Represents the total number of attention heads; Represents a transpose operation; represents the total number of attention heads; represents the dimension of the key vector;

[0071] All the comment text data of the attention head merging results are weighted and integrated through the normalization function, and a probability weight is assigned to the contextual representation of the i-th comment text data in the interaction to obtain the sentiment feature embedding of the user or project, which is expressed as:

[0072] (17);

[0073] In the formula, Represent the sentiment feature embedding of users or items; The weight matrix representing the learned linear transformation of the attention head merging result; The weight matrix representing the learned linear transformation of the activation function; express The bias term of express The bias term of Represents the total number of contextual representations of the comment text data corresponding to the user embedding vector U and the item embedding vector I; represents the activation function; represents the first Contextual representation of review text data;

[0074] By performing sentiment analysis on the sentiment feature embedding of users or projects, we can obtain the user's sentiment feature embedding and the sentiment feature embedding of the items , embedding the user's emotional features and the sentiment feature embedding of the items Connect with the user embedding vector U and the item embedding vector I respectively to get the final representation of the user and item, expressed as:

[0075] (18);

[0076] (19);

[0077] In the formula, represents the final representation of the user; Represents the sentiment feature embedding of the user embedding vector U; represents the final representation of the project; represents the sentiment feature embedding of the item embedding vector I;

[0078] Finally, the matching score between the user and the item is obtained by calculating the inner product of the final representation of the user and the item. , expressed as:

[0079] (20).

[0080] Further, in step S5, the accuracy of the matching score between the user and the item is obtained, and the specific process is as follows:

[0081] The mean square error loss is used to measure the matching score between the predicted user and the item. The difference between the actual user-item matching score and the accuracy of the user-item matching score is obtained, which is expressed as:

[0082] (twenty one);

[0083] In the formula, represents the loss function; The weight matrix representing the learned linear transformation; Indicates the actual matching score between users and items.

[0084] Compared with the existing technology, the present invention has the following beneficial effects:

[0085] (1) This invention combines knowledge graph embedding with the attention mechanism to dynamically capture the relationship between entities and emphasize the influence of important neighbors. At the same time, the multi-layer information propagation mechanism can learn the high-order interactions of entities, significantly improving the generalization ability of representation. In addition, the information filtering module effectively reduces noise interference and enhances the expressiveness of the model.

[0086] (2) The present invention introduces sentiment analysis, which can help identify users' emotional attitudes towards projects or content, thereby more accurately capturing users' preferences. By analyzing users' emotional feedback, the recommendation module can recommend new content with similar emotional tendencies, thereby improving the matching degree of recommendations and user satisfaction.

[0087] (3) The present invention obtains the probability distribution value of sentiment analysis by fusing sentiment analysis with knowledge graph embedding vectors. This method can process implicit feedback, provide context-aware sentiment drive for recommendation results, and improve the robustness of recommendations. Through multimodal information fusion, a more comprehensive project and user representation is provided, thereby achieving more refined recommendation matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0089] like Figure 1 As shown, the present invention provides a technical solution: a recommendation method integrating knowledge graph and sentiment analysis, comprising the following steps:

[0090] Step S1: construct a data set, process missing values ​​and standardized comment text data in the data set, extract entities and relations from the processed data set, and convert entities and relations into triples;

[0091] Step S2: linear transformation is used to map the triples to different vector spaces to obtain the scoring function of the triples, the scoring function of the triples is trained using the knowledge graph embedding learning model to obtain the features of the triples, and the features of the triples are calculated and filtered to obtain the filtered attention score values, and the filtered attention score values ​​are processed using the multi-head self-attention mechanism to obtain the user embedding vector and the item embedding vector;

[0092] Step S3: converting the comment text data into a pre-trained word unit embedding vector through a pre-trained language learning model, combining the word unit embedding vector with the paragraph identification vector and the position identification vector corresponding to the word unit embedding vector, constructing a comprehensive vector of the input sequence, inputting the comprehensive vector into the pre-trained language learning model for processing, obtaining a vector of contextual semantic information, inputting the vector of contextual semantic information into a bidirectional long short-term memory network for processing, and obtaining a probability distribution value of sentiment analysis;

[0093] Step S4: The user embedding vector and the project embedding vector are interactively fused by dot product, and the result of the dot product fusion is used as a query in the multi-head self-attention mechanism to extract the contextual representation of the comment text data in the interaction between the user embedding vector and the project embedding vector; the contextual representation of the comment text data is connected with the corresponding probability distribution value of the sentiment analysis to obtain the sentiment connection value, and based on the sentiment connection value, a multi-head attention mechanism is used to calculate the importance of the contextual representation of the comment text data in the interaction to the user, and the attention head merging result is obtained, and the attention head merging result is processed by a normalization function to obtain the sentiment feature embedding of the user or project, and the matching score of the user and the project is obtained by analyzing the sentiment feature embedding of the user or project;

[0094] Step S5: construct a recommendation module; perform matching prediction verification on the matching scores of the user and the project in step S4 through the recommendation module to obtain the accuracy of the matching scores of the user and the project.

[0095] In step S1, entities and relations are converted into triples. The specific process is as follows:

[0096] The data on the interactions between users and projects are constructed into a dataset, and the dataset is processed to remove invalid data, process missing values, and standardize comment text data. Entities and relations are extracted from the processed dataset and converted into triples.

[0097] Among them, users and items are converted into entities in triples, and the interaction between users and items is defined as the relationship in triples; triples are composed of head entities, tail entities, and the relationship between the corresponding head entities and tail entities; for the entity matrix , Represents the form in which entity f is stored in the entity matrix. Entity f includes the head entity and the tail entity; the relationship matrix R , Represents the form in which the relation r is stored in the relation matrix;

[0098] The triple The head entity h, tail entity t and relation r in are obtained by random initialization. , , ; , , They represent the head entity embedding vector, the tail entity embedding vector, and the relationship embedding vector respectively.

[0099] Among them, the user embedding vector and the item embedding vector are obtained in step S2, and the specific process is:

[0100] Step S21: Use linear transformation to map the head entity embedding vector, tail entity embedding vector and relationship embedding vector in the triple to different vector spaces through the knowledge graph embedding model to obtain the scoring function of the triple, which is expressed as:

[0101] (1);

[0102] In the formula, Represents the scoring function calculation of triples, triples The distance between the head entity h after being transformed by the relation r and the tail entity t. When the score value is small, it means that the relationship between h and t is stronger, and it is more likely to be a true relationship, and vice versa; The weight matrix representing the linear transformation of the relationship, used to transform the head entity embedding vector to a space closer to the tail entity embedding vector; Represents the square operation of the L2 norm;

[0103] Step S22: Then use the knowledge graph embedding learning model to train the scoring function of the triples, expressed as:

[0104] (2);

[0105] In the formula, Represents the training loss function of the knowledge graph embedding learning model; Represents a triple Negative samples of Indicates that an entity different from h is randomly selected from the entity matrix as the head entity of the negative sample; Randomly select entities different from t from the entity set representing the entity matrix as the tail entity of the negative sample; It represents the hyperparameter of the control interval. Its setting can help the knowledge graph embedding learning model distinguish between true triples and negative samples. True refers to the set of valid triples that actually exist in the knowledge graph. Max means taking the larger value of the two numbers.

[0106] After the training process of step S22, the knowledge graph embedding learning model adjusts the head entity embedding vector, the tail entity embedding vector and the relationship embedding vector, and finally the head entity embedding vector, the tail entity embedding vector and the relationship embedding vector tend to be stable, forming representative adjusted entity and relationship embedding vectors; and finally outputs the adjusted entity embedding vector matrix , Represents the form in which the adjusted entity f embedding vector is stored in the adjusted entity embedding vector matrix; the adjusted entity f embedding vector includes the adjusted head entity embedding vector and the adjusted tail entity embedding vector; the adjusted relationship embedding vector matrix , The adjusted relation r embedding vector is represented by the form in which the adjusted relation embedding vector is stored in the matrix;

[0107] Step S23: concatenate the adjusted head entity embedding vector, the adjusted tail entity embedding vector, and the adjusted relationship embedding vector to calculate the features of the triple, expressed as:

[0108] (3);

[0109] In the formula, Represents the characteristics of a triple; A weight matrix representing a trained linear transformation matrix; , and They represent the adjusted head entity h embedding vector, the adjusted relation r embedding vector and the adjusted tail entity t embedding vector respectively;

[0110] Step S24: In the features of the triplet, the adjusted embedding vector of the head entity h has multiple neighbor entities, and the attention score value of the triplet formed by the adjusted embedding vector of the head entity h and the neighbor entities is calculated, which is expressed as:

[0111] (4);

[0112] In the formula, Represents a triple The attention score value; Represents the activation function to introduce nonlinear elements and avoid gradient disappearance; The weight matrix representing the linear transformation used for learning is used to adjust the importance of features;

[0113] Step S25: After calculating the attention score of the triple, since the neighbor entities with low correlation of the adjusted head entity h embedding vector bring noise influence, an information filtering module is added to filter the attention score of the triple, so as to reduce the interference of noise; then the attention score of the filtered triple is normalized to obtain the filtered attention score, which is expressed as:

[0114] (5);

[0115] In the formula, represents the filtered attention score value, which represents the influence weight of the triplet on the adjusted head entity h embedding vector; exp represents converting the original score into a positive number; represents the set of all neighbor entities of the adjusted embedding vector of the head entity h; v represents a neighbor entity of the adjusted embedding vector of the head entity h; P represents the set of all relations between the adjusted embedding vector of the head entity h and the neighbor entities; u represents a relation embedding vector between the adjusted embedding vector of the entity h and the neighbor entity; represents the attention score threshold; Represents a triple The attention score value;

[0116] Step S26: Use M independent attention heads in the multi-head self-attention mechanism to capture different features and relationships in the filtered attention score values ​​respectively; by merging the outputs of the M independent attention heads, a comprehensive head entity is obtained, which is expressed as:

[0117] (6);

[0118] In the formula, Represents a comprehensive header entity; represents the activation function; Represents the neighbor entity of the adjusted head entity h; Represents the feature representation value in the Mth independent attention head; represents the attention score value in the Mth independent attention head; m represents the Mth independent attention head;

[0119] Step S27: Repeat the same method in step S26 to update the relationship embedding vector to obtain a new adjusted relationship embedding matrix ; By updating the adjusted relation embedding vector, the knowledge graph embedding learning model is helped to capture and optimize the semantic relationship between entities, and the prediction accuracy and generalization ability of the knowledge graph embedding learning model are improved; The adjusted relation embedding vector matrix Each row in represents an adjusted relation embedding vector. Update to get the new adjusted relation embedding matrix , expressed as:

[0120] ;

[0121] In the formula, Represents a weight matrix of a trained linear transformation matrix, used to adjust The value of

[0122] Step S28: A multi-layer information propagation mechanism is used to learn the high-order neighbor information of the comprehensive head entity; the first layer of information propagation focuses on the comprehensive head entity and the directly connected entities, and the second layer of information propagation is based on the first layer to expand the learning of entities adjacent to the comprehensive head entity and the neighbor entities; in each layer of information propagation, the attention mechanism is used to perform weighted aggregation of the information of the neighboring entities, and the representation of the comprehensive head entity is updated;

[0123] The concatenation operation is used to aggregate the comprehensive header entities of all layers of information propagation in the multi-layer information propagation mechanism into a single vector, which is expressed as:

[0124] (7);

[0125] In the formula, represents a single vector; represents the obtained first-layer comprehensive head entity embedding; represents the obtained comprehensive head entity embedding at the Zth level; and Represent the weight matrix and bias term of the trained linear transformation respectively;

[0126] Step S29: Use a multi-layer information propagation mechanism to train users and items in the same training method as step S28 to obtain user embedding vector U and item embedding vector I.

[0127] Among them, the probability distribution value of sentiment analysis in step S3, the specific process is:

[0128] First, the pre-trained language learning model (Bert model) converts the review text data into pre-trained word embedding vectors; the word embedding vectors are combined with the paragraph identification vectors and position identification vectors corresponding to the word embedding vectors to construct a comprehensive vector of the input sequence;

[0129] The paragraph identifier vector is used to distinguish different text segments such as questions and answers, while the position identifier vector encodes the relative position information of the word embedding vector in the sequence; the comprehensive vector is input into the multi-layer transformer network of the embedding layer of the pre-trained language learning model. Each layer of the multi-layer transformer network conducts in-depth analysis of the mutual relationship of the comprehensive vectors through a multi-head self-attention mechanism; after continuous processing by the multi-layer transformer network, a vector containing rich contextual semantic information is generated for each comprehensive vector in the input sequence, which is expressed as:

[0130] (8);

[0131] In the formula, A vector representing contextual semantic information; It means that it has been processed by a multi-layer converter network; represents the comprehensive vector representation of the input;

[0132] Pass the vector of contextual semantic information to a bidirectional long short-term memory network (BiLSTM); the bidirectional long short-term memory network consists of forward and backward long short-term memory network units;

[0133] The forward LSTM network unit processes the input sequence from the start to the end, while the backward LSTM network unit processes from the end to the start. The forward and backward LSTM network units process the vector of contextual semantic information through the input gate, forget gate, output gate, and maintenance and update of the cell state to obtain the hidden state of the current time step a of the bidirectional LSTM network, which is expressed as:

[0134] (9);

[0135] In the formula, represents the hidden state of the current time step a; represents the output gate; represents the hyperbolic tangent activation function; Indicates cell status;

[0136] Sentiment prediction: A fully connected layer is used to linearly transform the hidden state of the current time step a of the bidirectional long short-term memory network, which is expressed as:

[0137] (10);

[0138] In the formula, Represents the output vector of the fully connected layer; and Represent the weight matrix weights and bias terms of the linear transformation of the fully connected layer respectively;

[0139] Then use the activation function Map the output vector of the fully connected layer into the probability distribution value of sentiment analysis , expressed as:

[0140] (11);

[0141] In the formula, and They represent the weight matrix and bias term of the linear transformation of the output layer respectively.

[0142] In step S4, the matching score between the user and the project is obtained, and the specific process is as follows:

[0143] In order to learn the importance of the comment text data in the interaction between the user embedding vector U and the project embedding vector I to the user, a multi-head self-attention mechanism is introduced; the user embedding vector U and the project embedding vector I are interacted and fused by dot product, and the result of the dot product fusion is used as the query in the multi-head self-attention mechanism, and the context representation of the i-th comment text data in the interaction between the user embedding vector U and the project embedding vector I is taken out and recorded as q; the context representation q of the i-th comment text data is compared with the probability distribution value of the corresponding sentiment analysis Connect them together to get the emotional connection value B, expressed as:

[0144] (12);

[0145] In the formula, is the function used to connect the dot product result with the probability distribution value of sentiment analysis; Represents prime-level multiplication; Represents the concatenation operation of vectors;

[0146] After obtaining the sentiment connection value B, a multi-head attention mechanism is used to calculate the importance of the contextual representation of the i comment text data in the interaction to the user, and obtain the query ,key Sum ; Query It is used to extract information related to the current output from the input sequence; key Used to determine the relevance of each element in the input sequence to the current query; the value Contains information from the input sequence, which will be selectively aggregated to generate the final output representation:

[0147] (13);

[0148] (14);

[0149] (15);

[0150] In the formula, Generate query Represents the weight matrix of the learned linear transformation; Indicates the generated key The weight matrix of the learnable linear transformation represented by ; Represents the generated value Represents the weight matrix of the learned linear transformation; , and Respectively represent queries ,key Sum The bias term of

[0151] Using Query ,key Sum Calculate the attention score and get the attention head merging result, expressed as:

[0152] (16);

[0153] In the formula, Indicates the result of attention head merging; Represents the total number of attention heads; Represents a transpose operation; represents the total number of attention heads; Represents the dimension of the key vector, which is used to scale the attention scores to avoid numerical stability issues during gradient descent;

[0154] Emotional feature embedding: All comment text data of the attention head merging results are weighted and integrated through the normalization function, and a probability weight is assigned to the contextual representation of the i-th comment text data in the interaction to obtain the emotional feature embedding of the user or project, which is expressed as:

[0155] (17);

[0156] In the formula, Represent the sentiment feature embedding of users or items; The weight matrix representing the learned linear transformation of the attention head merging result; The weight matrix representing the learned linear transformation of the activation function; express The bias term of express The bias term of Represents the total number of contextual representations of the comment text data corresponding to the user embedding vector U and the item embedding vector I; represents the activation function; represents the first Contextual representation of review text data;

[0157] Prediction module: By performing sentiment analysis on the sentiment feature embedding of users or items, the user’s sentiment feature embedding is obtained. and the sentiment feature embedding of the items , embedding the user's emotional features and the sentiment feature embedding of the items Connect with the user embedding vector U and the item embedding vector I respectively to get the final representation of the user and item, expressed as:

[0158] (18);

[0159] (19);

[0160] In the formula, represents the final representation of the user; Represents the sentiment feature embedding of the user embedding vector U; represents the final representation of the project; represents the sentiment feature embedding of the item embedding vector I;

[0161] Finally, the matching score between the user and the item is obtained by calculating the inner product of the final representation of the user and the item. , expressed as:

[0162] (20).

[0163] Among them, the loss function: use the mean square error loss to measure the matching score between the predicted user and the item The difference between the actual user-item matching score and the accuracy of the user-item matching score is obtained, which is expressed as:

[0164] (twenty one);

[0165] In the formula, represents the loss function; The weight matrix representing the learned linear transformation is used to adjust the contribution of different samples to the total loss; Represents the matching score between the actual user and the item;

[0166] Embedding the knowledge graph into the training loss function of the learning model and the loss for the recommendation task As the final objective function, it is expressed as:

[0167] ;

[0168] In the formula, represents the final objective function; It means regularization in Adopted Parameterized L2 regularization to prevent overfitting, represents the regularization coefficient; express Parameters.

[0169] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A recommendation method integrating knowledge graph and sentiment analysis, characterized in that: The steps include: Step S1: construct a data set, process missing values ​​and standardized comment text data in the data set, extract entities and relations from the processed data set, and convert entities and relations into triples; Step S2: linear transformation is used to map the triples to different vector spaces to obtain the scoring function of the triples, the scoring function of the triples is trained using the knowledge graph embedding learning model to obtain the features of the triples, and the features of the triples are calculated and filtered to obtain the filtered attention score values, and the filtered attention score values ​​are processed using the multi-head self-attention mechanism to obtain the user embedding vector and the item embedding vector; Step S3: converting the comment text data into a pre-trained word unit embedding vector through a pre-trained language learning model, combining the word unit embedding vector with the paragraph identification vector and the position identification vector corresponding to the word unit embedding vector, constructing a comprehensive vector of the input sequence, inputting the comprehensive vector into the pre-trained language learning model for processing, obtaining a vector of contextual semantic information, inputting the vector of contextual semantic information into a bidirectional long short-term memory network for processing, and obtaining a probability distribution value of sentiment analysis; Step S4: The user embedding vector and the project embedding vector are interacted and fused by dot product to obtain a fused vector, the fused vector is input into the multi-head self-attention mechanism for query, and the contextual representation of the comment text data in the interaction between the user embedding vector and the project embedding vector is taken out; the contextual representation of the comment text data is connected with the probability distribution value of the corresponding sentiment analysis to obtain a connection vector, the fused vector and the connection vector are connected to obtain a sentiment connection value, based on the sentiment connection value, a multi-head attention mechanism is used to calculate the importance of the contextual representation of the comment text data in the interaction to the user, and an attention head merging result is obtained, the attention head merging result is processed by a normalization function to obtain the sentiment feature embedding of the user and the project, and the matching score of the user and the project is obtained by analyzing the sentiment feature embedding of the user and the project; Step S5: construct a recommendation module; The matching scores between the user and the project in step S4 are matched and predicted and verified by the recommendation module to obtain the accuracy of the matching scores between the user and the project.

2. The recommendation method integrating knowledge graph and sentiment analysis according to claim 1, characterized in that: In step S1, entities and relations are converted into triples. The specific process is as follows: The data on the interactions between users and projects are constructed into a dataset, and the dataset is processed to remove invalid data, process missing values, and standardize comment text data. Entities and relations are extracted from the processed dataset and converted into triples.

3. The recommendation method integrating knowledge graph and sentiment analysis according to claim 2, characterized in that: Users and items are converted into entities in triples. The interaction between users and items is defined as the relationship in the triple. The triple consists of the head entity, the tail entity and the relationship between the head entity and the tail entity. For the entity matrix , Represents the form in which entity f is stored in the entity matrix. Entity f includes the head entity and the tail entity; the relationship matrix R , Represents the form in which the relation r is stored in the relation matrix; The triple The head entity h, tail entity t and relation r in are obtained by random initialization. , , ; , , They represent the head entity embedding vector, the tail entity embedding vector, and the relationship embedding vector respectively.

4. The recommendation method integrating knowledge graph and sentiment analysis according to claim 3, characterized in that: In step S2, the user embedding vector and the item embedding vector are obtained. The specific process is as follows: Step S21: Use linear transformation to map the head entity embedding vector, tail entity embedding vector and relationship embedding vector in the triple to different vector spaces through the knowledge graph embedding model to obtain the scoring function of the triple, which is expressed as: （1）； In the formula, Represents the scoring function calculation for triples; A weight matrix representing a linear transformation of a relationship; Represents the square operation of the L2 norm; Step S22: Then use the knowledge graph embedding learning model to train the scoring function of the triples, expressed as: （2）； In the formula, Represents the training loss function of the knowledge graph embedding learning model; Represents a triple Negative samples of Indicates that an entity different from h is randomly selected from the entity matrix as the head entity of the negative sample; Randomly select entities different from t from the entity set representing the entity matrix as the tail entity of the negative sample; Represents the hyperparameter that controls the interval; True refers to the valid triple set that actually exists in the knowledge graph; max means taking the largest value of two numbers; After the training process of step S22, the knowledge graph embedding learning model adjusts the head entity embedding vector, the tail entity embedding vector and the relationship embedding vector, and finally the head entity embedding vector, the tail entity embedding vector and the relationship embedding vector tend to be stable, forming representative adjusted entity and relationship embedding vectors; and finally outputs the adjusted entity embedding vector matrix , Represents the form in which the adjusted entity f embedding vector is stored in the adjusted entity embedding vector matrix; The adjusted entity f embedding vector includes the adjusted head entity embedding vector and the adjusted tail entity embedding vector; the adjusted relationship embedding vector matrix , The adjusted relation r embedding vector is represented by the form in which the adjusted relation embedding vector is stored in the matrix; Step S23: concatenate the adjusted head entity embedding vector, the adjusted tail entity embedding vector, and the adjusted relationship embedding vector to calculate the features of the triple, which is expressed as: （3）； In the formula, Represents the characteristics of a triple; A weight matrix representing a trained linear transformation matrix; , and They represent the adjusted head entity h embedding vector, the adjusted relation r embedding vector and the adjusted tail entity t embedding vector respectively; Step S24: In the features of the triplet, the adjusted embedding vector of the head entity h has multiple neighbor entities, and the attention score value of the triplet formed by the adjusted embedding vector of the head entity h and the neighbor entities is calculated, which is expressed as: （4）； In the formula, Represents a triple The attention score value; represents the activation function to introduce nonlinear elements; The weight matrix representing the learned linear transformation; Step S25: Filter the attention score values ​​of the triples, and then normalize the attention score values ​​of the filtered triples to obtain the filtered attention score values, which are expressed as: （5）； In the formula, Represents the filtered attention score value; exp means converting the original score into a positive number; represents the set of all neighbor entities of the adjusted embedding vector of the head entity h; v represents a neighbor entity of the adjusted embedding vector of the entity h; P represents the set of all relations between the adjusted embedding vector of the head entity h and the neighbor entities; u represents a relation embedding vector between the adjusted embedding vector of the entity h and the neighbor entity; represents the attention score threshold; Represents a triple The attention score value; Step S26: Using the multi-head self-attention mechanism The independent attention heads capture different features and relationships in the filtered attention scores. By merging The output of the independent attention heads is obtained by obtaining the comprehensive head entity, which is expressed as: （6）； In the formula, Represents a comprehensive header entity; represents the activation function; Indicated in The feature representation values ​​in the independent attention heads; Indicated in The attention scores in the individual attention heads; Indicates independent attention heads; Step S27: adopting a multi-layer information propagation mechanism to learn high-order neighbor information of the comprehensive head entity; The first layer of information propagation focuses on the comprehensive head entity and directly connected entities, while the second layer of information propagation is based on the first layer and expands the learning of entities adjacent to the comprehensive head entity and neighboring entities. In each layer of information propagation, the attention mechanism is used to perform weighted aggregation of information on neighboring entities and update the representation of the comprehensive head entity. The concatenation operation is used to aggregate the comprehensive header entities of all layers of information propagation in the multi-layer information propagation mechanism into a single vector, which is expressed as: （7）； In the formula, represents a single vector; represents the obtained first-layer comprehensive head entity embedding; represents the obtained comprehensive head entity embedding at the Zth level; and Represent the weight matrix and bias term of the trained linear transformation respectively; Step S28: Use a multi-layer information propagation mechanism to train users and items in the same training method as step S27 to obtain user embedding vector U and item embedding vector I.

5. The recommendation method integrating knowledge graph and sentiment analysis according to claim 4, characterized in that: The probability distribution value of sentiment analysis in step S3, the specific process is: First, the pre-trained language learning model converts the review text data into pre-trained word embedding vectors; the word embedding vectors are combined with the paragraph identification vectors and position identification vectors corresponding to the word embedding vectors to construct a comprehensive vector of the input sequence; The comprehensive vector is input into the multi-layer transformer network of the embedding layer of the pre-trained language learning model. Each layer of the multi-layer transformer network conducts in-depth analysis of the relationship between the comprehensive vectors through a multi-head self-attention mechanism. After continuous processing by the multi-layer transformer network, a vector containing rich contextual semantic information is generated for each comprehensive vector in the input sequence, which is expressed as: （8）； In the formula, A vector representing contextual semantic information; It means that it has been processed by a multi-layer converter network; represents the comprehensive vector representation of the input; Pass the vector of contextual semantic information to a bidirectional long short-term memory network; the bidirectional long short-term memory network consists of forward and backward long short-term memory network units; The forward LSTM network unit processes the input sequence from the start to the end, while the backward LSTM network unit processes from the end to the start. The forward and backward LSTM network units process the vector of contextual semantic information through the input gate, forget gate, output gate, and maintenance and update of the cell state to obtain the hidden state of the current time step a of the bidirectional LSTM network, which is expressed as: （9）； In the formula, represents the hidden state of the current time step a; represents the output gate; represents the hyperbolic tangent activation function; Indicates cell status; The fully connected layer is used to linearly transform the hidden state of the current time step a of the bidirectional long short-term memory network, which is expressed as: （10）； In the formula, Represents the output vector of the fully connected layer; and Represent the weight matrix weights and bias terms of the linear transformation of the fully connected layer respectively; Then use the activation function to map the output vector of the fully connected layer into the probability distribution value of sentiment analysis , expressed as: （11）； In the formula, and Represent the weight matrix and bias term of the linear transformation of the output layer respectively; Represents the activation function.

6. The recommendation method integrating knowledge graph and sentiment analysis according to claim 5, characterized in that: In step S4, the matching score between the user and the project is obtained. The specific process is as follows: The user embedding vector U and the project embedding vector I interact and fuse them through dot product to obtain the fusion vector, which is input into the multi-head self-attention mechanism for query, and the context representation of the i-th comment text data in the interaction between the user embedding vector U and the project embedding vector I is taken out and recorded as q; q is compared with the probability distribution value of the corresponding sentiment analysis Connect them to get the connection vector, connect the fusion vector and the connection vector to get the emotional connection value B, which is expressed as: （12）； In the formula, represents the function that connects the dot product result with the probability distribution value of sentiment analysis; Represents prime-level multiplication; Represents the concatenation operation of vectors; After obtaining the sentiment connection value B, a multi-head attention mechanism is used to calculate the importance of the contextual representation of the i comment text data in the interaction to the user, and obtain the query ,key Sum , expressed as: （13）； （14）； （15）； In the formula, Generate query represents the weight matrix of the learned linear transformation; Indicates the generated key represents the weight matrix of the learned linear transformation; Represents a generated value represents the weight matrix of the learned linear transformation; , and Respectively represent queries ,key Sum The bias term of Using Query ,key Sum Calculate the attention score and get the attention head merging result, expressed as: （16）； In the formula, Indicates the result of attention head merging; Represents the total number of attention heads; Represents a transpose operation; represents the total number of attention heads; represents the dimension of the key vector; All the comment text data of the attention head merging results are weighted and integrated through the normalization function, and a probability weight is assigned to the contextual representation of the i-th comment text data in the interaction to obtain the sentiment feature embedding of the user and the project, which is expressed as: （17）； In the formula, Represent the sentiment feature embedding of users and items; The weight matrix representing the learned linear transformation of the attention head merging result; The weight matrix representing the learned linear transformation of the activation function; express The bias term of express The bias term of Represents the total number of contextual representations of the comment text data corresponding to the user embedding vector U and the item embedding vector I; represents the activation function; represents the first Contextual representation of review text data; By performing sentiment analysis on the sentiment feature embedding of users and projects, we can get the user's sentiment feature embedding Embedding with sentiment features of items , embedding the user's emotional features Embedding with sentiment features of items Connect with the user embedding vector U and the item embedding vector I respectively to get the final representation of the user and item, expressed as: （18）； （19）； In the formula, represents the final representation of the user; Represents the sentiment feature embedding of the user embedding vector U; represents the final representation of the project; represents the sentiment feature embedding of the item embedding vector I; Finally, the matching score between the user and the item is obtained by calculating the inner product of the final representation of the user and the item. , expressed as: （20）。 7. The recommendation method integrating knowledge graph and sentiment analysis according to claim 6, characterized in that: In step S5, the accuracy of the matching score between the user and the item is obtained. The specific process is as follows: The mean square error loss is used to measure the matching score between the predicted user and the item. The difference between the actual user-item matching score and the accuracy of the user-item matching score is obtained, which is expressed as: （21）； In the formula, represents the loss function; The weight matrix representing the learned linear transformation; Indicates the actual matching score between users and items.

Images