Hypergraph recommendation method for data enhancement based on large language model

Abstract

The invention discloses a hypergraph recommendation method for performing data enhancement based on a large language model. The hypergraph recommendation method comprises the following steps of S1, obtaining interaction data D0 of a user and a commodity in a data log mode; s2, performing preprocessing operation on the original interaction data D0 to obtain preprocessed interaction data D1; the invention provides a hypergraph recommendation method for performing data enhancement based on a big language model, which aims at the problems of sparse data and noise related to edge information of an existing recommendation system, performs simulation interaction enhancement and interaction noise data elimination on original interaction data by means of the strong understanding capability of the big language model, and improves the recommendation efficiency. The concept of a hypergraph in a complex network is introduced, the hypergraph convolution and joint updating technology is applied, the high-order relation between the user and the commodity is fully extracted, more accurate user and commodity embedding representation is obtained, and therefore the recommendation result is more accurate.

Description

Technical Field

[0001] The invention relates to a hypergraph recommendation method for data enhancement based on a large language model, and belongs to the field of digital information technology. Background Art

[0002] At present, in the current digital information age, when users choose to buy products, they will feel overwhelmed by the large number of online products, and it takes time and effort to choose the products that suit them. How to guide users through the hustle and bustle of information and provide personalized suggestions based on their preferences and behaviors has become an urgent problem to be solved. In order to meet this challenge, the recommendation system has become a key solution.

[0003] Recommendation systems are an effective way to solve the problem of information overload. They can prevent users from wasting too much time when choosing products, thereby improving retrieval efficiency. In recent years, graph neural networks have developed rapidly due to their powerful representation capabilities on graph structured data. However, an important limitation of this recommendation system using interactive edge information is the data noise problem, where attributes or features may lack direct correlation with user preferences due to incorrect clicks. In addition, it is difficult for many recommendation scenarios to learn high-quality user and item representations from limited interactive data. In the hypergraph, some nodes should be included in certain hyperedges but are not, resulting in the inability to obtain high-quality embedded representations, which greatly affects the performance of recommendations. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a hypergraph recommendation method based on a large language model for data enhancement. Aiming at the data sparsity and noise problems related to edge information in the existing recommendation system, the powerful understanding ability of the large language model is used to simulate interaction enhancement and eliminate interaction noise data on the original interaction data. The concept of hypergraph in complex networks is introduced, and hypergraph convolution and joint update technology are used to fully extract high-order relationships between users and products, so as to obtain more accurate user and product embedding representations, thereby making the recommendation results more accurate.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] A hypergraph recommendation method for data enhancement based on a large language model includes the following steps:

[0007] Step S1, obtaining the interaction data D0 between the user and the product in the form of data log;

[0008] Step S2, preprocessing the original interaction data D0 to obtain preprocessed interaction data D1;

[0009] Step S3: reprocess the interaction data D1, summarize user preferences through a large model, filter out products that users are really interested in in historical interactions, reduce noise in the interaction data D1 caused by behaviors such as misclicks, and obtain interaction data D2 with reduced noise;

[0010] Step S4: Use the large model to coarsely predict the user's future implicit interactions, and use the model as simulation data to enhance the interaction data D2, thereby obtaining enhanced interaction data D3;

[0011] Step S5: Model the interaction data D3 as a user-product bipartite graph, then define the high-order connectivity of users and products according to the k-order reachability rule, and generate a user hypergraph and a product hypergraph using the high-order connectivity in the user-product bipartite graph;

[0012] Step S6: After constructing the user and product hypergraphs, use hypergraph convolution to learn the information of high-order adjacent nodes from the user and product hypergraphs.

[0013] Step S7: Use the user embedding vector X u and product embedding vector X i To calculate the relevance score and construct a loss function to optimize the training process;

[0014] Step S8, using the BPR loss function and gradient descent algorithm to train the hypergraph convolutional neural network;

[0015] Step S9: Obtain user and product embedding vectors through a hypergraph convolutional neural network, then obtain the user's predicted score for each product through a vector dot product operation, and select the top K products as actual recommended products in descending order of the predicted scores and push them to relevant users.

[0016] Furthermore, the interaction data D0 in step S1 includes user id, product id, product type and interaction record.

[0017] Furthermore, in step S2, the original interaction data D0 is preprocessed, specifically including the following steps:

[0018] Remove duplicate data, filter users and products with interaction records greater than T, and map user IDs and product IDs to natural numbers respectively.

[0019] Furthermore, in step S3, the interaction data D1 is reprocessed, user preferences are summarized through a large model, and products that users are really interested in in historical interactions are screened out, and noise caused by behaviors such as misclicks in the interaction data D1 is reduced to obtain interaction data D2 with reduced noise, which specifically includes the following steps:

[0020] The llama-7B model was used to fine-tune the instructions, and the thought chain technique was used to construct the prompts, adding intermediate reasoning steps to the prompts;

[0021] Tips for the first step: Given the products that the user has interacted with historically and supplemented with product category information, the large model summarizes the user's preferences for products based on the frequency of product categories in the user's historical interactions;

[0022] Tips for the second step: Given the user's historical interaction sequence and user preferences, the large model selects products that meet the user's preferences from the historical interaction sequence to obtain interaction data D2 with reduced noise.

[0023] Furthermore, in step S4, the user's future implicit interactions are predicted in a coarse-grained manner using a large model as simulation data to enhance the interaction data D2, thereby obtaining enhanced interaction data D3, which specifically includes the following steps:

[0024] First, a task description is constructed, and the large model determines whether the user is willing to interact with the target product based on the user's historical interactions;

[0025] Next, we construct the task input and divide the user's historical interaction items into two groups based on the scores. The two groups of items include items that the user is willing to interact with and items that the user is unwilling to interact with. We also ask the big model about its interaction intention for items that the user has not interacted with.

[0026] Finally, the task description and task input are merged to create the instruction input, and finally the simulation-enhanced interaction data D3 can be obtained.

[0027] Furthermore, in step S5, the interaction data D3 is modeled as a user-product bipartite graph, and then the high-order connectivity of users and products is defined according to the k-order reachability rule, and the user hypergraph and product hypergraph are generated by using the high-order connectivity in the user-product bipartite graph, which specifically includes the following steps:

[0028] The interaction data D3 is modeled as a user-item bipartite graph G = (U, I, E). The high-order connectivity between users and items is defined according to the k-order reachability rule. The set of k-order reachable users (items) for item i (user i) can be marked as Aggregating multiple sets can get user hyperedge groups And commodity super edge group

[0029]

[0030] Construct a hypergraph to represent mixed high-order connections between users, connect a user hyperedge sets and b product hyperedge sets, and get the hypergraph association matrix as follows:

[0031]

[0032] Among them, H U is the user hypergraph association matrix, H I is the product hypergraph association matrix, is the a-th user hyperedge, is the b-th commodity hyperedge.

[0033] Furthermore, in step S6, after constructing the user and product hypergraphs, hypergraph convolution is used to learn information of high-order adjacent nodes from the user and product hypergraphs, which specifically includes the following steps:

[0034] First, construct the user embedding table Eu = [E u1 ,...,E uN ] and the commodity embedding table Ei=[E i1 ,...,E iM ], the normalized hypergraph incidence matrix The hypergraph convolution is input into the embedding table E, adjacent messages are aggregated, and high-order message passing is performed as follows:

[0035]

[0036] in, is the user hypergraph association matrix, is the commodity hypergraph association matrix, E U For user embedding, E I For product embedding, HGNNConv represents the hypergraph convolutional layer;

[0037] Joint message updates for users and products:

[0038]

[0039] Among them, Θ (l) is a trainable parameter, σ is the fully connected layer, is the user embedding of layer l+1, It is the product embedding of level l+1;

[0040] Then, the embeddings obtained at each layer are averaged to obtain the final user embedding and product embedding.

[0041] Furthermore, in step S7, the association score The calculation formula is as follows:

[0042]

[0043] Among them, X u Embedding vectors for users and products X i .

[0044] Further, in step S8, the calculation formula of the BPR loss function is as follows:

[0045]

[0046] Among them, u represents the user, i represents the product that the user has interacted with, and j represents the product that the user has not interacted with. represents the positive sample score, represents the score of negative samples, and the Bayesian personalized ranking loss L BPR The goal is to make the scores of interacted items higher than those of uninteracted items.

[0047] The above technical solution is adopted. The present invention introduces a large model into the hypergraph-based product recommendation model, and uses a hypergraph convolutional network to characterize the high-order associations between users and products, which can well mine non-paired high-order relationships in complex graphs to generate more efficient embedding vectors. At the same time, considering the coarse-grained prediction of implicit interactions through a large model, the original observation data is enhanced to solve the problem of inaccurate user and product embedding representation caused by data sparsity; the llama-7B large model after fine-tuning the instructions summarizes the user's preferences through the thinking chain based on the user's historical interaction data (product category), and screens out the products that the user is interested in from the products that the user interacts with, reducing the noise in the original user interaction caused by behaviors such as misclicks. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flowchart of a hypergraph recommendation method for data enhancement based on a large language model of the present invention.

[0049] Figure 2 It is a flow chart of the large model data enhancement method of the present invention;

[0050] Figure 3 It is a process framework diagram of the hypergraph convolutional network of the present invention. DETAILED DESCRIPTION

[0051] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments in conjunction with the accompanying drawings.

[0052] like Figure 1 As shown, this example provides a hypergraph recommendation method for data enhancement based on a large language model, which includes the following steps:

[0053] Step S1: Obtain interaction data D0 between a user and a product in the form of a data log. The interaction data D0 includes a user ID, a product ID, a product type, and an interaction record.

[0054] Step S2: preprocessing the original interaction data D0 to obtain preprocessed interaction data D1, where the data format of the interaction data D1 is {"user id":["item1 id", "item2 id"...]}. Preprocessing the original interaction data D0 specifically includes the following steps:

[0055] Remove duplicate data, filter users and products with interaction records greater than T, and map user IDs and product IDs to simple natural numbers.

[0056] Step S3, reprocess the interaction data D1, summarize user preferences through a large model, filter out the products that users are really interested in in historical interactions, reduce the noise caused by behaviors such as misclicks in the interaction data D1, and obtain the interaction data D2 with reduced noise. Specifically, it includes the following steps:

[0057] This embodiment uses the llama-7B model to fine-tune instructions, uses the thought chaining technique to construct prompts, and adds intermediate reasoning steps to the prompts to improve the performance of the llma in complex reasoning tasks;

[0058] Tips for the first step: Given the products that the user has interacted with historically and supplemented with product category information, the large model summarizes the user's preferences for products based on the frequency of product categories in the user's historical interactions;

[0059] Tips for the second step: Given the user's historical interaction sequence ["item1 id", "item2 id"...] and the user's preferences, the large model selects the products that meet the user's preferences from the historical interaction sequence. This can reduce the noise caused by the user's accidental clicks and obtain the interaction data D2 with reduced noise.

[0060] Step S4: Use a large model to coarsely predict the user's future implicit interactions as simulation data to enhance the interaction data D2, solve the problem of inaccurate user and product representation caused by data sparsity, and obtain enhanced interaction data D3. Specifically, the following steps are included:

[0061] like Figure 2 As shown, first, a task description is constructed, and the large model determines whether the user is willing to interact with the target product based on the user's historical interactions;

[0062] Next, we construct the task input and divide the user's historical interaction items into two groups based on the scores. The two groups of items include items that the user is willing to interact with and items that the user is unwilling to interact with. We also ask the big model about its interaction intention for items that the user has not interacted with.

[0063] Finally, the task description and task input are merged to create the instruction input, and the expected instruction output is set to yes or no, and finally the simulation-enhanced interaction data D3 can be obtained.

[0064] Step S5: Model the interaction data D3 as a user-product bipartite graph, then define the high-order connectivity of users and products according to the k-order reachability rule, and use the high-order connectivity in the user-product bipartite graph to generate a user hypergraph and a product hypergraph. Specifically, the following steps are included:

[0065] like Figure 3 As shown, the interaction data D3 is modeled as a user-item bipartite graph G = (U, I, E). The high-order connectivity of users and items is defined according to the k-order reachability rule. The set of k-order reachable users (items) for item i (user i) can be marked as Aggregating multiple sets can get user hyperedge groups And commodity super edge group

[0066]

[0067] Construct a hypergraph to represent mixed high-order connections between users, connect a user hyperedge sets and b product hyperedge sets, and get the hypergraph association matrix as follows:

[0068]

[0069] Among them, H U is the user hypergraph association matrix, H I is the product hypergraph association matrix, is the a-th user hyperedge, is the b-th commodity hyperedge.

[0070] Step S6: After constructing the user and product hypergraphs, use hypergraph convolution to learn complex information of high-order adjacent nodes from the user and product hypergraphs. Specifically, the following steps are included:

[0071] First, construct the user embedding table Eu = [E u1 ,...,E uN ] and the commodity embedding table Ei=[E i1 ,...,E iM ], the normalized hypergraph incidence matrix The hypergraph convolution is input into the embedding table E, adjacent messages are aggregated, and high-order message passing is performed as follows:

[0072]

[0073] in, is the user hypergraph association matrix, is the commodity hypergraph association matrix, E U For user embedding, E I For product embedding, HGNNConv represents the hypergraph convolutional layer;

[0074] In order to extract discriminative information, a joint message update is performed on users and products:

[0075]

[0076] Among them, Θ (l) is a trainable parameter, σ is the fully connected layer, is the user embedding of layer l+1, It is the product embedding of level l+1;

[0077] Then, the embeddings obtained at each layer are averaged to obtain the final user embedding and product embedding.

[0078] Step S7: Use the user embedding vector X u and product embedding vector X i To calculate the association score, and construct a loss function to optimize the training process, the association score The calculation formula is as follows:

[0079]

[0080] Among them, X u Embedding vectors for users and products X i .

[0081] Step S8: This embodiment selects pairwise optimization, assuming that a given user u prefers interacted products to non-interacted products. Therefore, the scores of interacted products should be higher than the scores of non-interacted products.

[0082] The hypergraph convolutional neural network is trained using the BPR loss function and gradient descent algorithm. The calculation formula of the BPR loss function is as follows:

[0083]

[0084] Among them, u represents the user, i represents the product that the user has interacted with, and j represents the product that the user has not interacted with. represents the positive sample score, represents the score of negative samples, and the Bayesian personalized ranking loss L BPR The goal is to make the scores of interacted items higher than those of uninteracted items.

[0085] Step S9: Obtain user and product embedding vectors through a hypergraph convolutional neural network, then obtain the user's predicted score for each product through a vector dot product operation, and select the top K products in descending order of predicted scores as actual recommended products and push them to relevant users, where K can be 5, 10, 20, etc.

[0086] The specific embodiments described above further illustrate the technical problems, technical solutions and beneficial effects solved by the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A hypergraph recommendation method for data enhancement based on a large language model, characterized in that: It includes the following steps: Step S1, obtaining the interaction data D0 between the user and the product in the form of data log; Step S2, preprocessing the original interaction data D0 to obtain preprocessed interaction data D1; Step S3: reprocess the interaction data D1, summarize user preferences through a large model, filter out products that users are really interested in in historical interactions, reduce noise in the interaction data D1 caused by behaviors such as misclicks, and obtain interaction data D2 with reduced noise; Step S4: Use the large model to coarsely predict the user's future implicit interactions, and use the model as simulation data to enhance the interaction data D2, thereby obtaining enhanced interaction data D3; Step S5: Model the interaction data D3 as a user-product bipartite graph, then define the high-order connectivity of users and products according to the k-order reachability rule, and generate a user hypergraph and a product hypergraph using the high-order connectivity in the user-product bipartite graph; Step S6: After constructing the user and product hypergraphs, use hypergraph convolution to learn the information of high-order adjacent nodes from the user and product hypergraphs. Step S7: Use the user embedding vector X u and product embedding vector X i To calculate the relevance score and construct a loss function to optimize the training process; Step S8, using the BPR loss function and gradient descent algorithm to train the hypergraph convolutional neural network; Step S9: Obtain user and product embedding vectors through a hypergraph convolutional neural network, then obtain the user's predicted score for each product through a vector dot product operation, and select the top K products as actual recommended products in descending order of the predicted scores and push them to relevant users.

2. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: The interaction data D0 in step S1 includes user ID, product ID, product type and interaction record.

3. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: In step S2, the original interaction data D0 is preprocessed, specifically including the following steps: Remove duplicate data, filter users and products with interaction records greater than T, and map user IDs and product IDs to natural numbers respectively.

4. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: In the step S3, the interaction data D1 is reprocessed, user preferences are summarized through a large model, and products that users are really interested in in historical interactions are screened out, and noise caused by behaviors such as misclicks in the interaction data D1 is reduced to obtain the interaction data D2 with reduced noise, which specifically includes the following steps: The llama-7B model was used to fine-tune the instructions, and the thought chain technique was used to construct the prompts, adding intermediate reasoning steps to the prompts; Tips for the first step: Given the products that the user has interacted with historically and supplemented with product category information, the large model summarizes the user's preferences for products based on the frequency of product categories in the user's historical interactions; Tips for the second step: Given the user's historical interaction sequence and user preferences, the large model selects products that meet the user's preferences from the historical interaction sequence to obtain interaction data D2 with reduced noise.

5. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: In step S4, a large model is used to coarsely predict the user's future implicit interaction as simulation data to enhance the interaction data D2, thereby obtaining enhanced interaction data D3, which specifically includes the following steps: First, a task description is constructed, and the large model determines whether the user is willing to interact with the target product based on the user's historical interactions; Next, we construct the task input and divide the user's historical interaction items into two groups based on the scores. The two groups of items include items that the user is willing to interact with and items that the user is unwilling to interact with. We also ask the big model about its interaction intention for items that the user has not interacted with. Finally, the task description and task input are merged to create the instruction input, and finally the simulation-enhanced interaction data D3 can be obtained.

6. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: In step S5, the interaction data D3 is modeled as a user-product bipartite graph, and then the high-order connectivity of users and products is defined according to the k-order reachability rule, and the user hypergraph and product hypergraph are generated using the high-order connectivity in the user-product bipartite graph, which specifically includes the following steps: The interaction data D3 is modeled as a user-item bipartite graph G = (U, I, E). The high-order connectivity between users and items is defined according to the k-order reachability rule. The set of k-order reachable users of item i is marked as The set of k-order reachable products for user i is marked as Aggregating multiple sets can get user hyperedge groups And commodity super edge group Construct a hypergraph to represent mixed high-order connections between users, connect a user hyperedge sets and b product hyperedge sets, and get the hypergraph association matrix as follows: Among them, H U is the user hypergraph association matrix, H I is the product hypergraph association matrix, is the a-th user hyperedge, is the b-th commodity hyperedge.

7. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: In step S6, after constructing the user and product hypergraphs, hypergraph convolution is used to learn information of high-order adjacent nodes from the user and product hypergraphs, which specifically includes the following steps: First, construct the user embedding table Eu = [E u1 ,...,E uN ] and the commodity embedding table Ei=[E i1 ,...,E iM ], the normalized hypergraph association matrix The hypergraph convolution is input into the embedding table E, adjacent messages are aggregated, and high-order message passing is performed as follows: in, is the user hypergraph association matrix, is the commodity hypergraph association matrix, E U For user embedding, E I For product embedding, HGNNConv represents the hypergraph convolutional layer; Joint message updates for users and products: Among them, Θ (l) is a trainable parameter, σ is the fully connected layer, is the user embedding of layer l+1, It is the product embedding of level l+1; Then, the embeddings obtained at each layer are averaged to obtain the final user embedding and product embedding.

8. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: In step S7, the association score The calculation formula is as follows: Among them, X u Embedding vectors for users and products X i .

9. The hypergraph recommendation method for data enhancement based on a large language model according to claim 1, characterized in that: In step S8, the calculation formula of the BPR loss function is as follows: Among them, u represents the user, i represents the product that the user has interacted with, and j represents the product that the user has not interacted with. represents the positive sample score, Represents the score of negative samples, Bayesian personalized ranking loss L BPR The goal is to make the scores of interacted items higher than those of uninteracted items.

Images