Inplausible friend recommendation method based on intra-layer and inter-layer attention
By using the method based on inter-layer attention within the layer, combined with user social networks and attribute networks, the problems of low computing efficiency, insufficient data sparsity and interpretation in the existing recommendation system are solved, and the generation of high-quality recommendation lists and users are realized. Improved satisfaction.
Patent Information
- Application Number
- CN202411050500.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-15
AI Technical Summary
The existing interpretability recommendation methods have low computational efficiency, large differences in input data, and data sparsity make it difficult for recommendation systems to accurately capture user interests, and lack effective interpretability and recommendation performance.
Using the method based on inter-layer attention in the layer, the user attribute network is constructed through user attribute text, combined with user social network and attribute network for joint learning, and the in-layer attention mechanism is used to extract features and inter-layer attention mechanism fusion information to generate users Embed for improved recommendation accuracy and interpretability.
It improves the accuracy and user satisfaction of the recommendation system, generates high-quality recommendation lists, has a certain degree of interpretability, and effectively improves user satisfaction.
Smart Images

Figure CN118967347A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of graph neural networks and recommendation technologies, and specifically to an explainable friend recommendation method based on intra-layer and inter-layer attention. Background Art
[0002] In today's digital age, users are faced with an increasing number of choices, making the research on recommendation systems a hot topic. As a powerful tool to alleviate information overload, recommendation systems usually recommend items or commodities that users may be interested in based on their historical behavior, thereby saving time and cost for people. Traditional recommendation systems are mainly divided into the following three categories: content-based recommendation algorithms, which use feature engineering to construct item features, use user historical data to learn user preferences, and recommend items that users may be interested in; recommendation algorithms based on collaborative filtering, which recommend items that similar users like to users based on the similarities between users; hybrid recommendation algorithms, which mix multiple recommendation techniques to make up for their respective shortcomings and obtain better recommendation performance.
[0003] Explainable recommendation is a very important research direction in recommendation systems. It aims to ensure the accuracy of recommendations while providing corresponding explanations for the recommendation results. It not only helps to improve the transparency, persuasiveness, effectiveness and credibility of recommendation systems, but also assists system designers in debugging and improving recommendation algorithms and improving the overall performance of recommendation systems.
[0004] At present, explainable recommendation methods can be roughly divided into methods based on topic modeling, graph-based methods, deep learning-based methods, and rule mining-based methods. The topic modeling-based method takes text information as input, builds a topic word cloud, and mines the content that users are most interested in. This method is widely used in e-commerce; the graph-based method regards the interaction between users and items as an adjacency matrix, constructs a bipartite or tripartite graph of users and items, and then uses graph information propagation or clustering methods to give explanations; the deep learning-based method uses different deep learning technologies, such as CNN, RNN, GNN, etc., to learn user and item embedding, and automatically give explainable sentences based on natural language generation models; the rule mining-based method uses frequent pattern mining, association rules, etc. to establish connections between different items and generate intuitive explanations for users. Although explainable recommendation methods are different, they also have shortcomings. First, the computational efficiency is low. Explainable recommendation methods must consider user preferences, explore user intrinsic interests, analyze user behavior, and give reasons for recommendations, so the design is relatively complex. Second, the input data varies greatly, including text data, audio data, video data, etc. The difference in data types poses a great challenge to data processing. Finally, there is the problem of data sparsity. The sparsity of data makes it difficult for the recommendation system to accurately capture user interests and give convincing reasons for recommendations. Summary of the invention
[0005] The purpose of the present invention is to provide an explainable friend recommendation method based on intra-layer and inter-layer attention to solve the above defects.
[0006] In order to achieve the above object, the present invention provides the following technical solutions:
[0007] An explainable friend recommendation method based on intra-layer and inter-layer attention includes the following steps:
[0008] S1. Let U be the set of users, where U = {u1,u2,…,u N}; Let T be the user attribute text set, where T = {t1, t2, …, t N}, t i is the attribute text of user i, t i = {g p ,g q ,…,g r}, g p , g q ,…,g r are attributes of user i respectively. The attributes in the attribute texts of different users may be different. The user attribute network is constructed through attribute clustering, user attribute reduction, similarity matrix calculation, attribute edge establishment and other steps.
[0009] S2, joint learning of user social network and user attribute network, where S is the social network between users, S = (U, E1), U is the user set, E1 is the set of social edges, E1 = {(u i ,u j )|u i ∈U,u j ∈U}, where (u i ,u j ) represents user u i and user u j There is an undirected social edge between them; A is the attribute network between users, where A = (U, E2), U is the set of users, E2 is the set of attribute edges, and E2 = {(u k ,u l )|u k ∈U,u l ∈U}, where (u k ,u l ) represents user u k and user u l There is an undirected attribute edge between them; through the structures such as embedding layer, diffusion layer, prediction layer, etc., user embedding is generated and the user recommendation list is output.
[0010] Preferably, the step S1 includes the following detailed steps:
[0011] S11. Based on the user attribute text, obtain the attribute set G in all attribute texts = {g1, g2, ..., g M}, where g1,g2,…,g M are different attributes. We use the trained word embedding model BERT to obtain attribute embedding, denoted as v = {v1, v2, …, v M}, where v1,v2,…,v M They are attributes g1, g2, ..., g M The embedding vector of is used to cluster the attributes using the k-means method;
[0012] S12. Divide the importance of attributes in different clusters by manual screening. For each user, he has the following three types of attributes: the attributes of the cluster center, which are retained because they are extremely representative; more important attributes, such as research direction, interests, etc., are also retained; ordinary attributes are replaced with the corresponding attributes of the cluster center; thus completing the reduction of user attributes;
[0013] S13. Let T′ be the set of user attribute texts after attribute reduction, where T′={t1′,t2′,…,t′ N}, t i ′ is the attribute text of user i after attribute reduction; let G′ be the attribute set after attribute reduction, where G′={g1′,g′2,…,g′ p}, g1′, g′2, …, g′ p is the remaining attribute after attribute reduction; given user u i and attribute text t i ′, attribute g l ′∈G′ for user u i The attribute text t i The calculation formula of the importance of ′ is as follows:
[0014]
[0015]
[0016]
[0017] Where TF is the attribute g l ′ frequency of occurrence, Is the attribute text t i Attribute g in ′ l ′Occurrence count, |t i ′| is the number of attributes in the attribute text, and IDF is the inverse attribute g l′ is the frequency of occurrence, N is the number of attribute texts, f l is the attribute g l 'The number of attribute texts that appear, is the attribute g l ′ for user u i The attribute text t i 'The importance of
[0018] Calculate the importance of different attributes to the attribute text in turn and get the value of user u i The attribute text t i TF-IDF vector of ′ in Represents attributes g1′, g′2, …, g′ p Attribute text t for user i i ′, and the TF-IDF vectors of other attribute texts can be obtained in the same way. The set of TF-IDF vectors of all user attribute texts is denoted as X = {x1, x2, …, x N}, x i Indicates attribute text t i ′TF-IDF vector; the similarity between different attribute texts is calculated by defining the following cosine similarity to obtain the similarity matrix of attribute texts. The similarity matrix represents the similarity matrix between user i and user j. The calculation of cosine similarity is as follows:
[0019]
[0020] In the formula, x i and x j are the embedding vectors of user i and user j respectively;
[0021] S14. Set the threshold μ. For any two users i and j, if s ij >μ, then user i and user j are considered similar, and an edge is used to connect the two, thus constructing an attribute-based user relationship network.
[0022] Preferably, the step S2 includes the following detailed steps:
[0023] S21, in the embedding layer, real number encoding is used to represent user embedding;
[0024] Let P∈R N×D represents the initialized user embedding matrix, N is the number of users, D is the embedding dimension, and the i-th row of the matrix P represents the embedding vector of user i; through linear transformation, a trainable weight matrix W1∈R is learned D×D , get the user embedding matrix E, so that the embedding matrix has practical meaning;
[0025] For a given user ui , embedded as shown in formula (5):
[0026]
[0027] In the formula, p i represents the user's original embedding, represents the user embedding after linear transformation;
[0028] S22. In the diffusion layer, user embedding is used as the input of the user social network and user attribute network, neighbor node information is obtained based on the intra-layer attention mechanism, and different network information is fused based on the inter-layer attention mechanism to generate the user's embedding vector;
[0029] The diffusion layer is a three-layer structure, each layer is basically the same, the specific operation is as follows:
[0030] S221, let l be 1, 2, 3 in sequence, and repeat steps S222 to S223;
[0031] S222, Feature extraction based on intra-layer attention mechanism:
[0032] First, the corresponding graph attention layers are generated for the user social network and the user attribute network respectively. The adjacency matrices of different networks are input into different graph attention layers respectively, and the user embedding is input at the same time, which is mapped into high-dimensional features through linear transformation to obtain sufficient expression ability.
[0033] Then, the attention weights of neighbor users are obtained through the attention mechanism of a single-layer feedforward neural network, and all attention weights are normalized, as shown in formulas (6) to (9):
[0034]
[0035]
[0036]
[0037]
[0038] In the formula, sl and al represent the lth layer of the user social network and the user attribute network respectively. and represents the importance of the features of user j in different networks to user i, || represents the concatenation of the two matrices, W sl and W al is the shared weight matrix, and represents user u at layer l-1 i and user u j The output embedding of and is the weight vector, LeakyRLU is the activation function for nonlinear changes, N i and For user u i The neighbor user set of and is the attention coefficient normalized by the softmax function;
[0039] Finally, the embedding vectors of neighbor users are averaged based on the attention weights to obtain the output embedding of the target user. The calculation formula for the user output embedding is:
[0040]
[0041]
[0042] In the formula, and is the user output embedding of the lth layer of different networks, σ is the activation function, S i and A i are neighbors of user i in different networks, and is the attention coefficient of different networks, W sl and W al is the weight matrix, and is the embedding of neighbor users;
[0043] S223. Feature fusion based on inter-layer attention mechanism:
[0044] After the processing in the previous part, the user embeddings of different networks are obtained; then, the embeddings of different networks are connected with the input embeddings and sent to the multi-layer MLP. The output dimension is one-dimensional, representing the score of this layer of network, as shown in formulas (12) and (13):
[0045]
[0046]
[0047] in, and is the attention coefficient of different networks, MLP is a multi-layer perceptron, [·] is a connection operation;
[0048] Then the scores of different networks are normalized to obtain the corresponding attention coefficients and output user embedding, as shown in formulas (14) to (16):
[0049]
[0050]
[0051]
[0052] In the formula, is the output embedding after fusing the embedding vectors of different network layers l, and is the attention coefficient of different networks;
[0053] S23. In the prediction layer, for any user i, obtain the embedding set of user i, denoted as Although the output of the latter layer contains the information of the previous layers, the output of each layer mainly focuses on the neighbor information of different orders. Therefore, different embeddings are connected as the final output, as shown in formula (17):
[0054]
[0055] In the formula, [·] is a concatenation operation, which can obtain the predicted score of user i for user j, as shown in formula (18):
[0056]
[0057] In the formula, ⊙ represents the vector dot product;
[0058] For user i, the predicted scores of other users by the user are arranged in descending order, and then several users with the highest scores are recommended to user i.
[0059] S24. Design of loss function. Select BPR loss as the loss function, as shown in formula (19):
[0060]
[0061] In the formula, R + is a set of positive samples, i.e., user pairs with social relations (u i ,u j ), R - is a set of negative samples, i.e., user pairs without social relations (u i ,u k ), σ(x) is the sigmoid function, and θ is the regularization parameter of the model.
[0062] Preferably, in step S14, the threshold μ is an adjustable parameter, and μ=0.6 is set here.
[0063] Preferably, in step S21, D is an adjustable parameter, and D=64 is set here.
[0064] The beneficial effects of the present invention are:
[0065] The present invention provides an explainable friend recommendation method based on intra-layer and inter-layer attention. A user attribute network is constructed according to user attribute text, and the user attribute network is obtained through attribute clustering, user attribute reduction, establishment of attribute edges and other steps. The accuracy of recommendation is improved and a high-quality recommendation list is generated through joint learning of user social network and user attribute network. An effective attention mechanism is adopted, including feature extraction based on the intra-layer attention mechanism and feature fusion based on the inter-layer attention mechanism. It is precisely because of the use of the intra-layer and inter-layer attention mechanism that the algorithm has a certain degree of explainability and can effectively improve user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 : Flow chart of step S1 in the present invention;
[0067] Figure 2 : Flowchart of step S2 in the present invention. DETAILED DESCRIPTION
[0068] The present invention is further described below in conjunction with the embodiments. It should be noted that these are merely examples and illustrations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should be deemed to fall within the protection scope of the present invention.
[0069] Embodiment 1:
[0070] An explainable friend recommendation method based on intra-layer and inter-layer attention, such as Figures 1-2 As shown, the following steps are included:
[0071] S1. Let U be the set of users, where U = {u1,u2,…,u N}; Let T be the user attribute text set, where T = {t1, t2, …, t N}, t i is the attribute text of user i, t i = {g p ,g q ,…,g r}, g p , g q ,…,g r are attributes of user i, such as gender, work unit, etc. The attributes in different user attribute texts may be different. Through attribute clustering, user attribute reduction, similarity matrix calculation, attribute edge establishment and other steps, a user attribute network is constructed. The detailed steps are as follows:
[0072] S11. Based on the user attribute text, obtain the attribute set G in all attribute texts = {g1, g2, ..., g M}, where g1,g2,…,g M are different attributes. We use the trained word embedding model BERT to obtain the corresponding attribute embedding, denoted as v = {v1, v2, …, v M}, where v1,v2,…,v M They are attributes g1, g2, ..., g M The embedding vector of user attributes is processed using the k-means algorithm. The elbow method is used to determine the number of clusters, that is, the value of k. At the same time, cosine similarity is used to measure the similarity between attribute i and attribute j. The calculation formula is:
[0073]
[0074] In the formula, v i and v j are the embedding vectors of attribute i and attribute j respectively.
[0075] S12. Divide the importance of attributes in different clusters by manual screening. For each user, he may have the following three types of attributes: the attributes of the cluster center, which are retained because of their excellent representativeness; more important attributes, such as research direction, interests, etc., are also retained; ordinary attributes are replaced with the corresponding attributes of the cluster center; thus completing the reduction of user attributes.
[0076] For example, assume that t1 = {g1, g2, g6}, t2 = {g3, g4, g7}, t3 = {g2, g3, g5}, t4 = {g3, g4, g5}, there are 4 attribute texts and 7 attributes in total, T is the user attribute text set, denoted by T = {t1, t2, t3, t4}, G is the set of all attributes, denoted by G = {g1, g2, …, g7}, use the trained word embedding model BERT to obtain all attribute embeddings, denoted by v = {v1, v2, …, v7}, use the k-means algorithm to cluster user attributes, and obtain the following three clusters {v1, v4, v6}, {v3, v7}, {v2, v5}, which correspond to the following three groups of attributes {g1, g4, g6}, {g3, g7}, {g2, g5}. The importance of attributes in different clusters is divided by manual screening. The cluster center attribute in {g1, g4, g6} is g1, the important attribute is g4, and the common attribute is g6. The cluster center attribute in {g3, g7} is g3, and the common attribute is g7. The cluster center attribute in {g2, g5} is g2, and the important attribute is g5. For the attribute text t1 = {g1, g2, g6}, g1 and g2 are attributes of different cluster centers, which are retained. g6 is a common attribute, and the attribute g1 corresponding to the cluster center is used for classification. The attribute text t1 is replaced by rows, thereby completing the attribute reduction of the attribute text t1. The reduction of other attribute texts can be completed similarly. The attribute texts after attribute reduction are t1={g1,g2,g1}, t2={g3,g4,g3}, t3={g2,g3,g5}, t4={g3,g4,g5}. After renumbering, they are recorded as t1′={g1′,g′2,g1′}, t2′={g′3,g′4,g′3}, t3′={g′2,g′3,g′5}, t4′={g′3,g′4,g′5}.
[0077] S13. Let T′ be the set of user attribute texts after attribute reduction, where T′={t1′,t2′,…,t′ N}, t i ′ is the attribute text of user i after attribute reduction; let G′ be the attribute set after attribute reduction, where G′={g1′,g′2,…,g′ p}, g1′, g′2, …, g′ p is the remaining attribute after attribute reduction; given user u i and attribute text t i ′, attribute g l ′∈G′ for user u i The attribute text t i The calculation formula of the importance of ′ is as follows:
[0078]
[0079]
[0080]
[0081] Where TF is the attribute g l ′ frequency of occurrence, Is the attribute text t i Attribute g in ′ l ′Occurrence count, |t i ′| is the number of attributes in the attribute text, and IDF is the inverse attribute g l ′ is the frequency of occurrence, N is the number of attribute texts, f l is the attribute g l 'The number of attribute texts that appear, Is the attribute a l ′ for user u i The attribute text t i 'The importance of
[0082] Calculate the importance of different attributes to the attribute text in turn and get the value of user u i The attribute text t i TF-IDF vector of ′ in Represents attributes g1′, g′2, …, g′ p Attribute text t for user i i ′, and the TF-IDF vectors of other attribute texts can be obtained in the same way, denoted as X = {x1, x2, …, x N}, x i Indicates attribute text t i ′’s TF-IDF vector.
[0083] For example, assuming that t1′={g1′,g′2,g1′}, t2′={g′3,g′3,g′4}, t3′={g′2,g′3,g′5}, t4′={g′3,g′4,g′5}, there are 4 attribute texts and 5 attributes. For the attribute text t1′, there are 2 attributes, which appear 2 times and 1 time respectively. The importance of g1′ to t1′ is calculated as follows:
[0084]
[0085] The importance of g′2 to t1′ is calculated as follows:
[0086]
[0087] Since there are no other attributes in t1′, the TF-IDF vector x1 of t1′ is {0.6778, 0.1382, 0, 0, 0}. The TF-IDF vectors of other texts can be obtained similarly.
[0088] The similarity between different attributed texts is calculated by defining the following cosine similarity to obtain the similarity matrix of the attributed texts. The similarity matrix represents the similarity matrix between user i and user j. The calculation of cosine similarity is as follows:
[0089]
[0090] In the formula, x i and x j are the embedding vectors of user i and user j respectively.
[0091] S14, set a threshold μ, which is an adjustable parameter. In this embodiment, μ=0.6. For any two users i and j, if s ij >μ, then user i and user j are considered similar, and an edge is used to connect the two, thus constructing an attribute-based user relationship network.
[0092] S2, joint learning of user social network and user attribute network, where S is the social network between users, S = (U, E1), U is the user set, E1 is the set of social edges, E1 = {(u i ,u j )|u i ∈U,u j ∈U}, where (u i ,u j ) represents user u i and user u j There is an undirected social edge between them; A is the attribute network between users, where A = (U, E2), U is the set of users, E2 is the set of attribute edges, and E2 = {(u k ,u l )|u k ∈U,u l ∈U}, where (u k ,u l ) represents user u k and user u l There is an undirected attribute edge between them. Through the structures of embedding layer, diffusion layer, prediction layer, etc., user embedding is generated and the user recommendation list is output. The detailed steps are as follows:
[0093] S21, in the embedding layer, real number encoding is used to represent user embedding;
[0094] Let P∈R N×Drepresents the initialized user embedding matrix, N is the number of users, D is the embedding dimension, and D is an adjustable parameter. In this embodiment, D=64 is set. The i-th row of the matrix P represents the embedding vector of user i. Through linear transformation, a trainable weight matrix W1∈R is learned. D×D , generate the user embedding matrix E, so that the embedding matrix has practical meaning.
[0095] For a given user u i , embedded as shown in formula (6):
[0096]
[0097] In the formula, p i represents the initialized embedding vector of user i, represents the user embedding vector after linear transformation, and W1 is the trainable weight matrix;
[0098] S22. In the diffusion layer, user embedding is used as the input of the user social network and user attribute network. The neighbor node information is obtained based on the intra-layer attention mechanism. The different network information is fused based on the inter-layer attention mechanism to generate the user's embedding vector.
[0099] The diffusion layer is a three-layer structure, each layer is basically the same, the specific operation is as follows:
[0100] S221. Let l be 1, 2, and 3 in sequence, and repeat steps S222 to S223.
[0101] S222, Feature extraction based on intra-layer attention mechanism:
[0102] First, corresponding graph attention layers are generated for the user social network and user attribute network respectively. The adjacency matrices of different networks are input into different graph attention layers respectively, and the user embedding is input at the same time, which is mapped into high-dimensional features through linear transformation to obtain sufficient expressive power.
[0103] Then, the user embedding and the neighbor embedding are connected and then dot-multiplied with the weight vector. Through nonlinear activation, the attention scores between the target user and the neighbor users are obtained. All attention scores are normalized and all attention weights are normalized, as shown in formulas (7) to (10):
[0104]
[0105]
[0106]
[0107]
[0108] In the formula, sl and al represent the lth layer of the user social network and the user attribute network respectively. and represents the importance of the features of user j in different networks to user i, || represents the concatenation of the two matrices, W sl and W al is the shared weight matrix, and represents user u at layer l-1 i and user u j The output embedding of and is the weight vector, LeakyRLU is the activation function for nonlinear changes, N i and For user u i The neighbor user set of and is the attention coefficient normalized by the softmax function.
[0109] Finally, the embedding vectors of neighbor users are weighted averaged based on the attention weights to obtain the output embedding vector of the target user. The calculation formula for the user output embedding is:
[0110]
[0111]
[0112] In the formula, and is the output embedding of users in layer l of different networks, σ is the activation function, S i and A i are neighbors of user i in different networks, and is the attention coefficient of different networks, W sl and W al is the weight matrix, and is the embedding vector of neighbor users.
[0113] S223. Feature fusion based on inter-layer attention mechanism:
[0114] After the processing in the previous part, the user embeddings of different networks are obtained; then, the embeddings of different networks are connected with the input embeddings and sent to the multi-layer MLP (multi-layer perceptron). The output dimension is one-dimensional, representing the score of this layer of network, as shown in formulas (13) and (14):
[0115]
[0116]
[0117] in, and is the attention coefficient of different networks, MLP is a multi-layer perceptron, [·] is a connection operation, is the output embedding of target user i after fusing the embedding vectors of different network layers l-1, and are the output embeddings of users at layer l of different networks.
[0118] Then the scores of different networks are normalized to obtain the corresponding attention coefficients and output user embedding, as shown in formulas (15) to (17):
[0119]
[0120]
[0121]
[0122] In the formula, is the output embedding after fusing the embedding vectors of different network layers l, and is the attention coefficient of different networks, and are the output embeddings of users at layer l of different networks.
[0123] S23. In the prediction layer, for any user i, obtain the embedding set of user i, denoted as Although the output of the latter layer contains the information of the previous layers, the output of each layer mainly focuses on the neighbor information of different orders. Therefore, different embeddings are connected as the final output, as shown in formula (18):
[0124]
[0125] In the formula, [·] is a concatenation operation, which can obtain the predicted score of user i for user j, as shown in formula (19):
[0126]
[0127] Where ⊙ represents the vector dot product.
[0128] For user i, the predicted scores of other users by the user are arranged in descending order, and then several users with the highest scores are recommended to user i.
[0129] S24. Design of loss function. Select BPR loss as the loss function, as shown in formula (20):
[0130]
[0131] In the formula, R + is a set of positive samples, i.e., user pairs with social relations (u i ,u j ), R - is a set of negative samples, i.e., user pairs without social relations (u i ,u k ), σ(x) is the sigmoid function, and θ is the regularization parameter of the model, such as the weight matrix W1 of the embedding layer and the weight matrix W of the diffusion layer. sl and W al wait.
[0132] The present invention provides an explainable friend recommendation method based on intra-layer and inter-layer attention. A user attribute network is constructed according to user attribute text, and the user attribute network is obtained through attribute clustering, user attribute reduction, establishment of attribute edges and other steps. The accuracy of recommendation is improved and a high-quality recommendation list is generated through joint learning of user social network and user attribute network. An effective attention mechanism is adopted, including feature extraction based on the intra-layer attention mechanism and feature fusion based on the inter-layer attention mechanism. It is precisely because of the use of the intra-layer and inter-layer attention mechanism that the algorithm has a certain degree of explainability and can effectively improve user satisfaction.
[0133] The above is an exemplary description of the invention. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as such non-substantial improvements are made by adopting the method concept and technical solution of the present invention, or the concept and technical solution of the invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.
Claims
1. An explainable friend recommendation method based on intra-layer and inter-layer attention, It is characterized in that The following steps are involved: S1. Let U be the set of users, where U = {u 1 ,u 2 ,…,u N }; Let T be the user attribute text set, where T = {t 1 ,t 2 ,…,t N }, t i is the attribute text of user i, t i = {g p ,g q ,…,g r }, g p , g q ,…,g r are attributes of user i respectively. The attributes in the attribute texts of different users may be different. The user attribute network is constructed through attribute clustering, user attribute reduction, similarity matrix calculation, attribute edge establishment and other steps. S2, joint learning of user social network and user attribute network, where S is the social network between users, S = (U, E 1 ), U is the user set, E 1 is the set of social edges, E 1 ={(u i ,u j )|u i ∈U,u j ∈U}, where (u i ,u j ) represents user u i and user u j There is an undirected social edge between them; A is the attribute network between users, where A = (U, E 2 ), U is the user set, E 2 is the set of attribute edges, E 2 ={(u k ,u l )|u k ∈U,u l ∈U}, where (u k ,u l ) represents user u k and user u l There is an undirected attribute edge between them; through the structures such as embedding layer, diffusion layer, prediction layer, etc., user embedding is generated and the user recommendation list is output.
2. According to claim 1, an explainable friend recommendation method based on intra-layer and inter-layer attention, It is characterized in that The detailed steps of step S1 are as follows: S11. Based on the user attribute text, obtain the set of different attributes in the attribute text of all users G = {g 1 ,g 2 ,…,g M }, where g 1 ,g 2 ,…,g M are different attributes. We use the trained word embedding model BERT to obtain the attribute embedding vector, denoted as v = {v 1 ,v 2 ,…,v M }, where v 1 ,v 2 ,…,v M They are attributes g 1 ,g 2 ,…, the embedding vector of gM, the attributes are clustered using the k-means method; S12. Divide the importance of attributes in different clusters by manual screening. For each user, he has the following three types of attributes: the attributes of the cluster center, which are retained because they are extremely representative; more important attributes, such as research direction, interests, etc., are also retained; ordinary attributes are replaced with the corresponding attributes of the cluster center; thus completing the reduction of user attributes; S13, let T′ be the user attribute text set after attribute reduction, where T′={t′ 1 ,t′ 2 ,…,t′ N }, t′ i is the attribute text of user i after attribute reduction; let G′ be the attribute set after attribute reduction, where G′={g′ 1 ,g′ 2 ,…,g′ p }, g′ 1 ,g′ 2 ,…,g′ p is the remaining attribute after attribute reduction; given user u i and attribute text t′ i , attribute g′ l ∈G′ for user u i The attribute text t′ i The importance of is calculated as follows: Where TF is the attribute g′ l Frequency of occurrence, is the attribute text t′ i The middle attribute g′ l Number of occurrences, |t′ i | is the number of attributes in the attribute text, and IDF is the inverse attribute g′ l Occurrence frequency, N is the number of attribute texts, f l is the attribute g′ l The number of occurrences of the attribute text, is the attribute g′ l For user u i The attribute text t′ i The importance of Calculate the importance of different attributes to the attribute text in turn and get the value of user u i The attribute text t′ i TF-IDF vector in Represents the attribute g′ 1 ,g′ 2 ,…,g′ p Attribute text t′ for user i i The importance of the text attribute, the TF-IDF vector of other attribute texts can be obtained in the same way. The set of TF-IDF vectors of all user attribute texts is recorded as X = {x 1 ,x 2 ,…,x N }, x i Represents attribute text t′ i TF-IDF vector; the similarity between different attribute texts is calculated by defining the following cosine similarity to obtain the similarity matrix of attribute texts. The similarity matrix represents the similarity matrix between user i and user j. The calculation of cosine similarity is as follows: In the formula, x i and x j are the embedding vectors of user i and user j respectively; S14. Set the threshold μ. For any two users i and j, if s ij >μ, then user i and user j are considered similar, and an edge is used to connect the two, thus constructing an attribute-based user relationship network.
3. According to claim 1, an explainable friend recommendation method based on intra-layer and inter-layer attention, It is characterized in that The detailed steps of step S2 are as follows: S21, in the embedding layer, real number encoding is used to represent user embedding; Let P∈R N×D represents a randomly initialized user embedding matrix, N is the number of users, D is the embedding dimension, and the i-th row of the matrix P represents the embedding vector of user i; Through linear transformation, learn a trainable weight matrix W 1 ∈R D×D , get the user embedding matrix E, so that the embedding matrix has practical meaning; For a given user u i , embedded as shown in formula (5): In the formula, p i represents the initialized embedding vector of user i, represents the user embedding vector after linear transformation; S22. In the diffusion layer, user embedding is used as the input of the user social network and user attribute network, neighbor node information is obtained based on the intra-layer attention mechanism, and different network information is fused based on the inter-layer attention mechanism to generate the user's embedding vector; The diffusion layer is a three-layer structure, each layer is basically the same, the specific operation is as follows: S221, let l be 1, 2, 3 in sequence, and repeat steps S222 to S223; S222, Feature extraction based on intra-layer attention mechanism: First, the corresponding graph attention layers are generated for the user social network and the user attribute network respectively. The adjacency matrices of different networks are input into different graph attention layers respectively, and the user embedding is input at the same time, which is mapped into high-dimensional features through linear transformation to obtain sufficient expression ability. Then, the attention weights of neighbor users are obtained through the attention mechanism of a single-layer feedforward neural network, and all attention weights are normalized, as shown in formulas (6) to (9): In the formula, sl and al represent the lth layer of the user social network and the user attribute network respectively. and represents the importance of the features of user j in different networks to user i, || represents the concatenation of the two matrices, W sl and W al is the shared weight matrix, and represents user u at layer l-1 i and user u j The output embedding of and is the weight vector, LeakyRLU is the activation function for nonlinear changes, N i and For user u i The neighbor user set of and is the attention coefficient normalized by the softmax function; Finally, the embedding vectors of neighbor users are averaged based on the attention weights to obtain the output embedding of the target user. The calculation formula for the user output embedding is: In the formula, and is the output embedding of users in layer l of different networks, σ is the activation function, S i and A i are neighbors of user i in different networks, and is the attention coefficient of different networks, W sl and W al is the weight matrix, and is the embedding of neighbor users; S223. Feature fusion based on inter-layer attention mechanism: After the processing in the previous part, the user embeddings of different networks are obtained; then, the embeddings of different networks are connected with the input embeddings and sent to the multi-layer MLP. The output dimension is one-dimensional, representing the score of this layer of network, as shown in formulas (12) and (13): in, and is the attention coefficient of different networks, MLP is a multi-layer perceptron, [·] is a connection operation; Then the scores of different networks are normalized to obtain the corresponding attention coefficients and output the user embedding vector, as shown in formulas (14) to (16): In the formula, is the output embedding after fusing the embedding vectors of different network layers l, and is the attention coefficient of different networks; S23. In the prediction layer, for any user i, obtain the embedding set of user i, denoted as Although the output of the latter layer contains the information of the previous layers, the output of each layer mainly focuses on the neighbor information of different orders. Therefore, different embeddings are connected as the final output, as shown in formula (17): In the formula, [·] is a concatenation operation, which can obtain the predicted score of user i for user j, as shown in formula (18): In the formula, ⊙ represents the vector dot product; For user i, the predicted scores of other users by the user are arranged in descending order, and then several users with the highest scores are recommended to user i. S24. Design of loss function. Select BPR loss as the loss function, as shown in formula (19): In the formula, R + is a set of positive samples, i.e., user pairs with social relations (u i ,u j ), R - is a set of negative samples, i.e., user pairs without social relations (u i ,u k ), σ(x) is the sigmoid function, and θ is the regularization parameter of the model.
4. The explainable friend recommendation method based on intra-layer and inter-layer attention according to claim 2, It is characterized in that In step S14, the threshold μ is an adjustable parameter, and μ=0.6 is set here.
5. The explainable friend recommendation method based on intra-layer and inter-layer attention according to claim 3, It is characterized in that In step S21, D is an adjustable parameter, and D=64 is set here.
Citation Information
Cited By
Graph attention mechanism-based map optimization and path recommendation method
CN120316340A
