Long-tail sequence recommendation algorithm based on knowledge perception self-supervised learning
By fusing the user-item interaction two-part graph with the knowledge graph to construct and train the extended sequence of long-tail sequences, the problem of difficulty in processing long-tail sequences is solved, and high-precision recommendation effect is achieved.
Patent Information
- Application Number
- CN202410940177.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-12
AI Technical Summary
The existing sequence recommendation algorithms are not effective when processing long-tail sequences, resulting in inaccurate recommendation results and poor user experience, because there is less user behavior data in long-tail sequences and short sequence length, which is difficult to effectively process through conventional algorithms.
By fusing the user-item interaction two-part graph with the knowledge graph to form a collaborative knowledge graph, an extended sequence of long-tail sequences is constructed, and a Transformer encoder is used for embedding encoding, and high-quality embedding is obtained through self-supervised learning training, thereby improving recommendation accuracy .
It effectively improves the recommendation accuracy of long-tail sequences and significantly improves the user experience. By using entity information in the knowledge graph to build an extended sequence, it provides self-supervised signals for long-tail sequences, and obtains high-precision recommendation results.
Smart Images

Figure CN118939873A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of recommendation technology, and in particular to a long-tail sequence recommendation algorithm based on knowledge-aware self-supervised learning. Background Art
[0002] Traditional recommendation algorithms mainly explore users' static interests, and there are many problems such as difficulty in capturing users' real-time interest changes, which leads to poor results in practical applications and difficulty in meeting users' personalized needs. Unlike traditional recommendation algorithms, sequence recommendation algorithms can analyze users' behaviors in different time periods and adjust recommended content in real time. Its ability to dynamically adjust and capture user interests in real time makes the recommendation results more accurate and diversified, significantly improving the user experience. However, existing sequence recommendation algorithms perform well when processing long sequences, but face challenges when processing short sequences. Since there is less user behavior data in long-tail sequences, the sequence length is short, and long-tail sequences occupy the vast majority of all sequences, it is difficult to effectively process them through conventional sequence recommendation algorithms, resulting in inaccurate recommendation results and poor user experience. Summary of the invention
[0003] In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides a long-tail sequence recommendation algorithm based on knowledge-aware self-supervised learning.
[0004] The technical solution of the present invention is: a long-tail sequence recommendation algorithm based on knowledge-aware self-supervised learning, which is carried out in the following steps:
[0005] Step 1. Transform the user-item interaction bipartite graph G 1 ={(u, i)|u∈U, i∈I} and knowledge graph G 2 ={(h, r, t)|h, t∈ε, r∈R} to form a collaborative knowledge graph G 3 ={(h, r, t)|h, t∈ε′, r∈R′};
[0006] The user-item interaction bipartite graph G 1 In the example, (u, i) indicates that user u has interacted with item i, and U = {u 1 ,u 2 , ..., u |U|} represents the user set, I={i 1 ,i 2 , ..., i |I|} represents a collection of items;
[0007] The knowledge graph G 2 In the above example, (h, r, t) represents a triple consisting of the head entity h and the tail entity t through the relation r, and ε = {e 1 , e2 , ..., e |ε|} represents an entity set, R = {r 1 , r 2 ,…,r |R|} represents a relationship set;
[0008] In the collaborative knowledge graph, ε′=ε∪U, R′=R∪{interaction}, |U| represents the total number of users, |I| represents the total number of items, |ε| represents the total number of entities, and |R| represents the total number of relationships;
[0009] Step 2. Order represents the interaction sequence of user u, |s u | represents the number of items contained in the interaction sequence of user u∈U. All users’ interaction sequences are sorted in descending order according to the number of items in the sequence. The number of items in the sequence is accumulated starting from the shortest interaction sequence. When the accumulated number reaches 20% of the total number of items, this part of the sequence is defined as a long-tail sequence, and a long-tail sequence set is obtained. Starting from the shortest interaction sequence in the long-tail sequence set, the interaction sequence s of user u is used as the starting point. u As input, it is encoded by the Transformer encoder to obtain the original interaction sequence embedding h of user u as shown in formula (1): u :
[0010] h u =TransEnc(s u ) (1)
[0011] Step 3. Based on the interaction sequence s of user u in the long-tail sequence set u Construct K extended sequences from the items in where e m and e n is the connection interaction sequence s in the knowledge graph u Entity nodes between items, where K is the connection interaction sequence s in the knowledge graph u The number of paths between items;
[0012] Step 4. Each extended sequence obtained As input, it is encoded by the Transformer encoder to obtain the extended sequence embedding of user u as shown in formula (2):
[0013]
[0014] Step 5. Use self-supervised learning to train the original interaction sequence embedding h u and extended sequence embedding Use formula (3) as the self-supervised loss function
[0015]
[0016] in, and Represents the embedding of different extended sequences generated by the same user u, which are used as positive sample pairs during training; and Represents different users u and The generated extended sequence embedding is used as a negative sample pair during training. τ represents a hyperparameter and cosine represents a similarity calculation function.
[0017] Step 6. Embed the trained interaction sequence into h u Embed with items i Using formula (4) to perform inner product, we can get the predicted score p of user u for item i. u,i :
[0018] p u,i =h u ·e i (4)
[0019] Select the N items with the highest predicted scores as recommendation results and provide them to user u.
[0020] The present invention makes full use of the rich entity information in the knowledge graph, uses the associated entities of items to construct an extended sequence of the long-tail sequence, provides a self-supervisory signal for the long-tail sequence, and uses a self-supervised learning model to train the long-tail sequence and obtain high-quality embedding, thereby generating high-precision recommendation results, effectively solving the problem that high-precision recommendations cannot be obtained due to the short length of the long-tail sequence. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is a flow chart of an embodiment of the present invention.
[0022] Figure 2 Schematic diagram of the corresponding results of the algorithm of the present invention and the existing algorithm in the Recall index as the number of recommendations changes DETAILED DESCRIPTION
[0023] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific examples described herein are only used to explain the present invention and are not intended to limit the present invention. Figure 1 As shown, the long-tail sequence recommendation algorithm based on knowledge-aware self-supervised learning of the present invention is performed in the following steps:
[0024] Step 1. Transform the user-item interaction bipartite graph G 1={(u, i)|u∈U, i∈I} and knowledge graph G 2 ={(h, r, t)|h, t∈ε, r∈R} to form a collaborative knowledge graph G 3 ={(h, r, t)|h, t∈ε′, r∈R′};
[0025] The user-item interaction bipartite graph G 1 In the example, (u, i) indicates that user u has interacted with item i, and U = {u 1 ,u 2 , ..., u |U|} represents the user set, I={i 1 ,i 2 , ..., i |I|} represents a collection of items;
[0026] The knowledge graph G 2 In the above example, (h, r, t) represents a triple consisting of the head entity h and the tail entity t through the relation r, and ε = {e 1 , e 2 , ..., e |ε|} represents an entity set, R = {r 1 , r 2 , ..., r |R|} represents a relationship set;
[0027] In the collaborative knowledge graph, ε′=ε∪U, R′=R∪{interaction}, |U| represents the total number of users, |I| represents the total number of items, |ε| represents the total number of entities, and |R| represents the total number of relationships;
[0028] If the user set U = {u 1 ,u 2 ,u 3}, item set I = {i 1 ,i 2 ,i 3 ,i 4 ,i 5 ,i 6}, entity set ε = {e 1 , e 2 , e 3 , e 4}, relation set R = {r 1 , r 2 , r 3}, then the user-item interaction bipartite graph G 1 As shown in Table 1, 0 indicates no interaction and 1 indicates interaction:
[0029] Table 1
[0030] <![CDATA[i 1 ]]> <![CDATA[i 2 ]]> <![CDATA[i 3 ]]> <![CDATA[i 4 ]]> <![CDATA[i 5 ]]> <![CDATA[i 6 ]]> <![CDATA[u 1 ]]> 0 1 0 1 0 1 <![CDATA[u 2 ]]> 1 0 1 1 1 0 <![CDATA[u 3 ]]> 1 1 1 1 0 1
[0031] The triple information obtained through the relationship between items and entities in the knowledge graph is shown in Table 2:
[0032] Table 2
[0033] Head Entity relation Tail Entity <![CDATA[i 1 ]]> <![CDATA[r 1 ]]> <![CDATA[e 1 ]]> <![CDATA[i 2 ]]> <![CDATA[r 2 ]]> <![CDATA[e 2 ]]> <![CDATA[i 3 ]]> <![CDATA[r 3 ]]> <![CDATA[e 3 ]]> <![CDATA[i 4 ]]> <![CDATA[r 2 ]]> <![CDATA[e 1 ]]> <![CDATA[i 4 ]]> <![CDATA[r 1 ]]> <![CDATA[e 2 ]]> <![CDATA[i 5 ]]> <![CDATA[r 2 ]]> <![CDATA[e 4 ]]> <![CDATA[i 6 ]]> <![CDATA[r 3 ]]> <![CDATA[e 2 ]]>
[0034] Step 2. Order represents the interaction sequence of user u, |s u | represents the number of items contained in the interaction sequence of user u∈U. All users’ interaction sequences are sorted in descending order according to the number of items in the sequence. The number of items in the sequence is accumulated starting from the shortest interaction sequence. When the accumulated number reaches 20% of the total number of items, this part of the sequence is defined as a long-tail sequence, and a long-tail sequence set is obtained. Starting from the shortest interaction sequence in the long-tail sequence set, the interaction sequence s of user u is used as the starting point. u As input, it is encoded by the Transformer encoder to obtain the original interaction sequence embedding h of user u as shown in formula (1): u :
[0035] h u =TransEnc(s u ) (1)
[0036] If the long-tail sequence set is user u 1 、User u 2 and user u 3 Interaction sequence, due to user u 1 The interaction sequence of user u 2 and u 3 In short, here we first use user u 1 The sequence is a long-tail sequence As input, it is encoded by the Transformer encoder to obtain user u 1 Interaction sequence embedding
[0037]
[0038] Step 3. Based on the interaction sequence s of user u in the long-tail sequence set u Construct K extended sequences from the items in where e m and e n is the connection interaction sequence s in the collaborative knowledge graph u Entity nodes between items, where K is the connection interaction sequence s in the knowledge graph u The number of paths between items;
[0039] Since in the knowledge graph 2 Through the relationship 1 With entity e 1 Connect, 4 Through the relationship 2 With entity e 1 Connect and i 4 Through the relationship 1 With entity e 2 Connect, 6 Through the relationship 3 With entity e 2 Therefore, the interaction sequence can be Expand to get Based on the above ideas, we can obtain multiple extended sequences of the original long-tail sequences;
[0040] Step 4. Each extended sequence obtained As input, it is encoded by the Transformer encoder to obtain the extended sequence embedding of user u as shown in formula (2):
[0041]
[0042] Step 5. Use self-supervised learning to train the original interaction sequence embedding h u and its extended sequence embedding Use formula (3) as the self-supervised loss function
[0043]
[0044] in, and Represents the embedding of different extended sequences generated by the same user u, which are used as positive sample pairs during training; and Represents different users u and The generated extended sequence embedding is used as a negative sample pair during training. τ represents a hyperparameter and cosine represents a similarity calculation function.
[0045] Step 6. Embed the trained interaction sequence into h u Embed with items i Using formula (4) to perform inner product, we can get the predicted score p of user u for item i. u,i :
[0046] p u,i =h u ·e i (4)
[0047] Select the N items with the highest predicted scores as recommendation results and provide them to user u.
[0048] Specifically, we get the trained interaction sequence embedding User 1 Uninteracted Items 1 ,i 3 and i 5 Embed and Embed interaction sequences with uninteracted items i 1 ,i 3 and i 5 The inner product of the embedding is used to obtain user u 1 For item i 1 ,i 3 and i 5 Prediction score of Select item i with the highest predicted score 3 Provided to user u as recommendation results 1 .
[0049] The method of the embodiment of the present invention is experimentally demonstrated, which specifically includes:
[0050] (1) Using public datasets
[0051] The user-item interaction dataset used in this paper is LastFM, and the knowledge graph used is Freebase. The LastFM dataset contains 23,566 users, 48,123 songs, and 3,034,796 user-song interactions. The Freebase database contains 58,266 entities, 9 relationships, and 464,567 triples.
[0052] (2) Evaluation indicators
[0053] The present invention uses Recall as the evaluation index of this experiment. Recall can measure the recommendation performance of long-tail sequences by showing the recommendation accuracy. The higher the value, the higher the recommendation accuracy. The calculation formula is as follows:
[0054]
[0055] Among them, N r Represents the total number of items that the target user likes, N rs Indicates the number of items that the target user likes among the recommended items.
[0056] The algorithm proposed in this paper is compared with the classic recommendation algorithm based on knowledge graph KGAT (Knowledge Graph Attention Network for Recommendation). The results are as follows: Figure 2 As shown in the figure, the experimental data show that as the number of recommendations increases, the Recall result of the algorithm of the present invention is significantly higher than that of the KGAT algorithm. Since the larger the Recall value, the higher the recommendation accuracy, the algorithm of the present invention can recommend items with higher accuracy for long-tail sequences than KGAT.
Claims
1. A long-tail sequence recommendation algorithm based on knowledge-aware self-supervised learning, characterized by Proceed as follows: Step 1. Fuse the user-item interaction bipartite graph G1 = {(u,i)|u∈U,i∈I} with the knowledge graph G2 = {(h,r,t)|h,t∈ε,r∈R} to form a collaborative knowledge graph G3 = {(h,r,t)|h,t∈ε′,r∈R′}; In the user-item interaction bipartite graph G1, (u,i) indicates that user u has interacted with item i, U = {u1,u2,…,u |U| } represents the user set, I={i1,i2,…,i |I| } represents a collection of items; In the knowledge graph G2, (h, r, t) represents a triple consisting of a head entity h and a tail entity t through a relationship r, ε = {e1, e2, …, e |ε| } represents an entity set, R = {r1, r2, ..., r |R| } represents a relationship set; In the collaborative knowledge graph, ε′=ε∪U, R′=R∪{interaction}, |U| represents the total number of users, |I| represents the total number of items, |ε| represents the total number of entities, and |R| represents the total number of relationships; Step 2. Order represents the interaction sequence of user u, |s u | represents the number of items contained in the interaction sequence of user u∈U. All users’ interaction sequences are sorted in descending order according to the number of items in the sequence. The number of items in the sequence is accumulated starting from the shortest interaction sequence. When the accumulated number reaches 20% of the total number of items, this part of the sequence is defined as a long-tail sequence, and a long-tail sequence set is obtained. Starting from the shortest interaction sequence in the long-tail sequence set, the interaction sequence s of user u is used as the starting point. u As input, it is encoded by the Transformer encoder to obtain the original interaction sequence embedding h of user u as shown in formula (1): u : h u =TransEnc(s u ) (1) Step 3. Based on the interaction sequence s of user u in the long-tail sequence set u Construct K extended sequences from the items in where e m and e n is the connection interaction sequence s in the collaborative knowledge graph u Entity nodes between items, where K is the connection interaction sequence s in the collaborative knowledge graph u The number of paths between items; Step 4. Each extended sequence obtained As input, it is encoded by the Transformer encoder to obtain the extended sequence embedding of user u as shown in formula (2): Step 5. Use self-supervised learning to train the original interaction sequence embedding h u and extended sequence embedding Use formula (3) as the self-supervised loss function in, and Represents the embedding of different extended sequences generated by the same user u, which are used as positive sample pairs during training; and Represents different users u and The generated extended sequence embedding is used as a negative sample pair during training. τ represents a hyperparameter and cosine represents a similarity calculation function. Step 6. Embed the trained interaction sequence into h u Embed with items i Using formula (4) to perform inner product, we can get the predicted score p of user u for item i. u,i : p u,i =h u ·e i (4) Select the N items with the highest predicted scores as recommendation results and provide them to user u.
