A graph neural network model recommendation method integrating attention mechanism

Abstract

The present invention discloses a graph neural network model recommendation method integrating an attention mechanism, the method comprising the following steps: S1: input data preprocessing, the input data preprocessing is divided into two parts, the user item model and the user social model. In view of the problem that the current recommendation system has low accuracy and cannot accurately utilize social network information and time series information, this method can take into account the time series information of users and items, and can integrate the user's social network information. This technical solution makes full use of time series information and social network information, and when adding a multi-head attention mechanism to recommend to users, it focuses on the influence of friends on users in different aspects, rather than considering the mutual influence between friends, so that the algorithm has better performance.

Description

Technical Field

[0001] The present invention relates to a graph neural network model recommendation method integrating an attention mechanism, which can be used in the technical field of recommendation algorithms. Background Art

[0002] In the recommendation system, the recommendation algorithm based on graph neural network (GNN) is one of the popular algorithms for recommendation system applications. In 2009, graph neural network was proposed, and the emergence of GCN model in 2017 made graph neural network develop more rapidly. The NGCF model introduced GCN into the recommendation algorithm, and modeled the expression of high-order connectivity in the user-item graph, so as to effectively inject the collaborative signal into the embedding process in an explicit way; the BGANR algorithm uses the attention mechanism and adds bias to better capture the high-order connectivity between nodes. The GraphRec model proposes a principled method based on GNN, which can capture the interactions and opinions in the user-item graph at the same time, model the two graphs at the same time, and model the heterogeneous strength.

[0003] As a new type of extended neural network inspired by convolutional neural network (CNN) and graph embedding ideas, graph neural network can extract and represent data features in the graph field. It is an efficient and easily scalable new neural network structure, and has shown powerful functions in learning graph data. Compared with traditional deep learning methods, graph neural network can reflect entities and their connections by building graph models. With the publication of literature on attention mechanism, researchers have also begun to consider using attention mechanism to improve existing related social recommendation algorithms. CHENG et al. proposed to integrate attention mechanism into the recommendation system in CHENGZ Y, DING Y, HE XN, et al. A^3NCF Anadaptive aspect attention model for rating prediction [C] / / Proceedings of the 27th International Joint Conference on Artificial Intelligence, Louisiana, Feb 2-Feb 7, 2018. Menlo Park: AAAI, 2018: 3748-3754., and carried out fruitful research work, verifying the good effect of attention mechanism in various social recommendation tasks. Summary of the invention

[0004] The purpose of this invention is to solve the above problems existing in the prior art and to propose a graph neural network model recommendation method that integrates an attention mechanism.

[0005] The purpose of the present invention is achieved through the following technical solution: a method for recommending a graph neural network model integrating an attention mechanism, the method comprising the following steps:

[0006] S1: Input data preprocessing, which is divided into two parts: user-item model and user-social model;

[0007] S2: In the user-item model obtained in step S1, calculate the user u i The set N of neighboring nodes on the set V V ( u i) =Neigh v (ui), N V ( u i) Item node v in the collection j Arrange in ascending order by time t, and obtain an item sequence S (u i )=[v1,v2,…,v L ]; for the sequence S(u i ) in the item element v j Encode it using onehot encoding and then enter the embedding layer to get the corresponding embedding vector And the corresponding score The embedding representation of Will and Splice and input into the perceptron dimensionality reduction to obtain an item information representation e′ containing the rating value j And update the item sequence S;

[0008] S3: Take the updated item sequence S in step S2 as the input of the gated recurrent unit, and input the initial hidden state h0 at the same time, and obtain the sequence of length L composed of the final output of the gated recurrent unit at each moment Where k is the number of stacked layers of gated recurrent units;

[0009] The gated recurrent unit performs the following calculation at time t:

[0010]

[0011] r t =σ(W ir x t +b ir +W hr h (t-1) +b hr ) (2)

[0012] z t =σ(Wiz x t +b iz +W hz h (t-1) +b hz ) (3)

[0013] n t =tanh(W in x t +b in +W hn h (t-1) +b hn ) (4)

[0014] h t =(1-z t )⊙n t +z t ⊙h (t-1) (5)

[0015] Among them, δ represents the Dropout function, σ represents the Sigmoid activation function, and ⊙ represents the element product; x t Indicates the input at the current moment, h (t-1) Represents the hidden state of the GRU unit at the previous moment;

[0016] S4: Splicing h' and After the activation of the linear layer and the nonlinear ReLU function, we get As the neighborhood set N v (ui) aggregation result; the aggregation result Embedding representation with the vertex itself Splicing, linear affine and nonlinear activation, to get the nth iteration representation of the user node

[0017] S5: In the user social model, for all users and friends, the gated network is first used to embed the users and their friends, and the fused embedding vector representation is obtained after feature fusion. After obtaining the fused vector S, the attention vector is learned from the attention weight matrix Q.

[0018] Note that each element of vector a is defined as: Then, use the softmax function to normalize a and get the final result:

[0019]

[0020] Friends Embed U (i，j) Expanded to the matrix via the memory matrix Q:

[0021] F l =u (i，j) ⊙Ql (7)

[0022] where ⊙ represents the element-wise product of vectors; the matrix F represents the friend preferences in different latent aspects;

[0023] Finally, to generate the friend vector representation, the attention scores are used to compute a weighted representation of F:

[0024] f (i，j) =∑a l F l (8)

[0025] The output is a specific relation vector f (i，j) , which can be regarded as the influence vector of user i’s j-th friend on this user’s preference; let f (i，1) , f (i，2) …f (i，j) The friend relationship vector of user j generated by the attention memory module;

[0026] S6: Using the multi-head attention mechanism, for each user (Q), compare it with the input friend vector (K) obtained from the upper layer to obtain the output vector under the influence of different friends. The formula is as follows:

[0027]

[0028] p (i，j) =MultiHeadAttention(Q,K,V)=Concat(head,...head) (10)

[0029] S7: The output of the upper layer is integrated with the user itself to obtain the user's comprehensive interest preference; under the influence of the gating mechanism, the gate G and the final user are influenced by different aspects of their friends I i The calculation formula is:

[0030]

[0031]

[0032] Preferably, in step S3, the number k of stacking layers of gated recurrent units is 2.

[0033] Preferably, in the step S3, the dropout probability of the Dropout function is 0.5.

[0034] Preferably, in the step S4, the number of iterations n of the user node is 5.

[0035] Preferably, in the step S4, the initial embedding dimension is 64.

[0036] Preferably, in step S6, the multi-head attention mechanism linearly maps Q, K and V, performs a scaled dot product attention operation on the mapping result, and then splices the results of the scaled dot product attention and inputs them into the linear mapping layer, and finally obtains the result of the multi-head attention mechanism; Represents the characteristic dimension of vectors Q and K, which is used to avoid excessive inner product values.

[0037] Preferably, in the step S3, the gated recurrent unit converts x t and h (t-1) The two are concatenated and passed through an independent fully connected layer to obtain r t Controlled reset gating factor and z t The forgetting gating factor controlled by t )⊙n t : Indicates the selective "forgetting" of the original hidden state; z t ⊙h (t-1) : Indicates the selective "memory" of the information containing the current node.

[0038] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: In view of the problem that the current recommendation system has low accuracy and cannot accurately utilize social network information and time series information, this method can take into account the time series information of users and items, and can integrate the social network information of users. This technical solution makes full use of time series information and social network information, adds a multi-head attention mechanism to recommend users, and focuses on considering the influence of friends on users in different aspects, rather than considering the mutual influence between friends, so that the algorithm has better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of a method for recommending a graph neural network model that integrates an attention mechanism according to the present invention. DETAILED DESCRIPTION

[0040] The purpose, advantages and features of the present invention will be illustrated and explained by the non-limiting description of the following preferred embodiments. These embodiments are only typical examples of the application of the technical solution of the present invention, and any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection claimed by the present invention.

[0041] The present invention discloses a graph neural network model recommendation method integrating attention mechanism, such as Figure 1 As shown, the method comprises the following steps:

[0042] S1: Input data preprocessing, which is divided into two parts: user-item model and user-social model;

[0043] S2: In the user-item model obtained in step S1, calculate the user u i The set N of neighboring nodes on the set V V (ui) =Neigh v (ui), N V (ui) Item node v in the collection j Arrange in ascending order by time t, and obtain an item sequence S (u i )=[v1,v2,…,v L ]; for the sequence S(u i ) in the item element v j Encode it using onehot encoding and then enter the embedding layer to get the corresponding embedding vector And the corresponding score The embedding representation of Will and Splice and input into the perceptron dimensionality reduction to obtain an item information representation e′ containing the rating value j And update the item sequence S;

[0044] S3: Take the updated item sequence S in step S2 as the input of the gated recurrent unit, and input the initial hidden state h0 at the same time, and obtain the sequence of length L composed of the final output of the gated recurrent unit at each moment Where k = 2 is the number of stacked layers of gated recurrent units;

[0045] The gated recurrent unit performs the following calculation at time t:

[0046]

[0047] r t =σ(W ir x t +b ir +W hr h (t-1) +b hr ) (14)

[0048] z t =σ(W iz x t +b iz +W hz h (t-1) +b hz ) (15)

[0049] n t =tanh(W in x t +b in +Whn h (t-1) +b hn ) (16)

[0050] h t =(1-z t )⊙n t +z t ⊙h (t-1) (17)

[0051] Among them, δ represents the Dropout function with a dropout probability of 0.5, σ represents the Sigmoid activation function, and ⊙ represents the element product; x t Indicates the input at the current moment, h (t-1) Represents the hidden state of the GRU unit at the previous moment; the two are concatenated and passed through an independent fully connected layer to obtain r t Controlled reset gating factor and z t The forgetting gating factor controlled by t )⊙n t : Indicates the selective "forgetting" of the original hidden state; z t ⊙h (t-1) : Indicates the selective "memory" of the information containing the current node;

[0052] S4: Splicing h' and After the activation of the linear layer and the nonlinear ReLU function, we get As the neighborhood set N v (ui) aggregation result; the aggregation result Embedding representation with the vertex itself Splicing, linear affine and nonlinear activation, to get the nth iteration representation of the user node When the user node is iterated 5 times, the effect is the best and no overfitting occurs;

[0053] S5: In the user social model, for all users and friends, the gated network is first used to embed the users and their friends, and the fused embedding vector representation is obtained after feature fusion, where the initial embedding dimension is 64; after obtaining the fused vector S, the attention vector is learned from the attention weight matrix Q;

[0054] Note that each element of vector a is defined as: Then, use the softmax function to normalize a and get the final result:

[0055]

[0056] Friends Embed U (i，j) Expanded to the matrix via the memory matrix Q:

[0057] F l =u (i，j) ⊙Q l (19)

[0058] where ⊙ represents the element-wise product of vectors; the matrix F represents the friend preferences in different latent aspects;

[0059] Finally, to generate the friend vector representation, the attention scores are used to compute a weighted representation of F:

[0060] f (i，j) =∑a l F l (20)

[0061] The output is a specific relation vector f (i，j) , which can be regarded as the influence vector of user i’s j-th friend on this user’s preference; let f (i，1) , f (i，2) …f (i，j) The friend relationship vector of user j generated by the attention memory module;

[0062] S6: Using the multi-head attention mechanism, for each user (Q), compare it with the input friend vector (K) obtained from the upper layer to obtain the output vector under the influence of different friends. The formula is as follows:

[0063]

[0064] p (i，j) =MultiHeadAttention(Q,K,V)=Concat(head,...head) (22)

[0065] The multi-head attention mechanism linearly maps Q, K, and V, performs a scaled dot product attention operation on the mapping results, and then concatenates the results of the scaled dot product attention and inputs them into the linear mapping layer to finally obtain the result of the multi-head attention mechanism; Represents the characteristic dimension of vectors Q and K, which is used to avoid excessive inner product values;

[0066] S7: The output of the upper layer is integrated with the user itself to obtain the user's comprehensive interest preference; under the influence of the gating mechanism, the gate G and the final user are influenced by different aspects of their friends I i The calculation formula is:

[0067]

[0068]

[0069] The present invention uses graph neural networks to perform deep feature learning and extract user feature representation when processing user ratings. In processing social information, the multi-head attention mechanism is used to more comprehensively consider the influence of friends on users in different aspects, deeply highlighting the important influence of relevant important friends in a certain aspect, and at the same time, the gated network is used for fusion to improve the accuracy of the final recommendation.

[0070] The method includes using a gated recurrent unit to forget and memorize the time series information of the project, thereby enhancing the abstraction ability of the time series information; using a gated neural network to fuse the input user and friends to obtain a rich interactive embedding representation; using an attention memory network to obtain the influence of friends on the user in different aspects; using a multi-head attention mechanism to adjust the influence between friends; and finally, integrating the influence of friends based on the user's own interests and hobbies, and then recommending the project. The present invention can improve the accuracy of the recommendation results.

[0071] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit and essential features of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting from any point of view, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0072] In addition, it should be understood that although this specification is described in accordance with the implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation modes that can be understood by those skilled in the art. The present invention has many implementation modes, and all technical solutions formed by equivalent transformation or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A graph neural network model recommendation method integrating attention mechanism, characterized by: The method comprises the following steps: S1: Input data preprocessing, which is divided into two parts: user-item model and user-social model; S2: In the user-item model obtained in step S1, calculate the user u i The set of neighboring nodes on the set V Item node v in the collection j Arrange in ascending order by time t, and obtain an item sequence S (u i )=[v1,v2,…,v L ]; for the sequence S(u i ) in the item element v j Encode it using onehot encoding and then enter the embedding layer to get the corresponding embedding vector And the corresponding score The embedding representation of Will and Splice and input into the perceptron dimensionality reduction to obtain an item information representation e containing the rating value j τ and update the item sequence S; S3: Take the updated item sequence S in step S2 as the input of the gated recurrent unit, and input the initial hidden state h0 at the same time, and obtain the sequence of length L composed of the final output of the gated recurrent unit at each moment Where k is the number of stacked layers of gated recurrent units; The gated recurrent unit performs the following calculation at time t: r t =σ(W ir x t +b ir +W hr h (t-1) +b hr ) (2) z t =σ(W iz x t +b iz +W hz h (t-1) +b hz ) (3) n t =tanh(W in x t +b in +W hn h (t-1) +b hn ) (4) h t =(1-z t )⊙n t +z t ⊙h (t-1) (5) Among them, δ represents the Dropout function, σ represents the Sigmoid activation function, and ⊙ represents the element product; x t Indicates the input at the current moment, h (t-1) Represents the hidden state of the GRU unit at the previous moment; S4: Splicing h' and After the activation of the linear layer and the nonlinear ReLU function, we get As the neighborhood set N v (ui) aggregation result; the aggregation result Embedding representation with the vertex itself Splicing, linear affine and nonlinear activation, to get the nth iteration representation of the user node S5: In the user social model, for all users and friends, the gated network is first used to embed the users and their friends, and the fused embedding vector representation is obtained after feature fusion. After obtaining the fused vector S, the attention vector is learned from the attention weight matrix Q. Note that each element of vector a is defined as: Then, use the softmax function to normalize a and get the final result: Friends Embed U (i , j) Expanded to the matrix via the memory matrix Q: F l =u (i , j) ⊙Q l (7)where ⊙ represents the element-wise product of vectors; the matrix F represents the friend preferences in different latent aspects; Finally, to generate the friend vector representation, the attention scores are used to compute a weighted representation of F: f (i , j) =Σa l F l (8) The output is a specific relationship vector f (i，j) , as the influence vector of user i’s j-th friend on this user’s preference; let f (i，1) , f (i，2) …f (i，j) The friend relationship vector of user j generated by the attention memory module; S6: Using the multi-head attention mechanism, for each user Q, compare it with the input friend vector K obtained in the upper layer to obtain the output vector under the influence of different friends. The formula is as follows: p (i，j) =MultiHeadAttention(Q,K,V)=Concat(head,…head) (10) S7: The output of the upper layer is integrated with the user itself to obtain the user's comprehensive interest preference; under the influence of the gating mechanism, the gate G and the final user are influenced by different aspects of their friends I i The calculation formula is:

2. According to claim 1, a graph neural network model recommendation method integrating attention mechanism is characterized in that: In the step S3, the number k of stacking layers of the gated recurrent unit is 2.

3. According to claim 1, a graph neural network model recommendation method integrating attention mechanism is characterized in that: In the step S3, the dropout probability of the Dropout function is 0.

5.

4. The method for recommending a graph neural network model integrating an attention mechanism according to claim 1, characterized in that: In the step S4, the number of iterations n of the user node is 5.

5. The method for recommending a graph neural network model integrating an attention mechanism according to claim 1, characterized in that: In the S4 step, the initial embedding dimension is 64.

6. The method for recommending a graph neural network model integrating an attention mechanism according to claim 1, characterized in that: In the step S6, the multi-head attention mechanism linearly maps Q, K and V, performs a scaled dot product attention operation on the mapping results, and then splices the results of the scaled dot product attention and inputs them into the linear mapping layer, and finally obtains the result of the multi-head attention mechanism; Represents the characteristic dimension of vectors Q and K, which is used to avoid excessive inner product values.

7. The method for recommending a graph neural network model integrating an attention mechanism according to claim 1, characterized in that: In the step S3, the gated recurrent unit converts x t and h (t-1) The two are concatenated and passed through an independent fully connected layer to obtain r t Controlled reset gating factor and z t The forgetting gating factor controlled by t )⊙n t : Indicates the selective "forgetting" of the original hidden state; z t ⊙h (t-1) : Indicates selective "memory" of the information containing the current node.

Images