Graph enhancement method based on LLM low-cost interaction

Abstract

This invention discloses a graph enhancement method based on low-cost LLM interaction, belonging to the fields of recommendation systems and deep learning technology. The method includes the following steps: S1: obtaining basic user and item data, constructing a user-item interaction matrix, a user-item rating matrix, and a multi-label item type matrix; S2: calculating the user's type preference vector; S3: calculating cosine similarity and the comprehensive rating of the items; S4: constructing a directed weighted hypergraph, calculating the importance scores of the nodes, and generating predicted connections using the LLM model; S5-S6: calculating the original loss and new loss of the recommendation system, and updating the user-item bipartite graph. This invention overcomes the problems of high computational interaction cost, poor scalability, and high system noise introduced when using LLM to enhance the user-item interaction graph of the recommendation system. By using low-cost LLM interaction, the method effectively improves the integrity and richness of the user-item interaction bipartite graph, ensuring the scalability and real-time performance of the system, while also improving the accuracy of recommendations.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a graph enhancement method based on LLM low-cost interaction. Background Art

[0002] With the rapid development of e-commerce, streaming services, and social media platforms, recommender systems have become a key component for delivering personalized content and enhancing user experience. Traditional recommendation algorithms are primarily based on collaborative filtering, content filtering, or a hybrid of the two. However, these methods have limitations in handling complex user-item interactions and capturing high-order connectivity patterns.

[0003] Graph convolutional neural networks, as a deep learning model capable of processing graph-structured data, have been widely used in recommendation systems in recent years. By modeling the interaction graph between users and items, graph convolutional neural networks can capture high-order associations and implicit features between nodes, thereby improving the accuracy of recommendations. Chinese Patent Publication No. CN113961820A, published on January 21, 2022, entitled "A Social Recommender System Based on Lightweight Graph Convolutional Networks," discloses a recommendation system comprising an information acquisition module, an information propagation module, and an information recommendation module. This patent utilizes a graph convolutional network to propagate user representations across two graphs and item representations across a user-item interaction graph, leveraging user social data to enhance user and item representations. A fusion model ensures that both user representations are fully utilized, ensuring that the information learned from the two graphs does not conflict with each other, leading to a decrease in the representation of users and items. However, the performance of graph convolutional neural networks is highly dependent on the integrity and richness of the graph structure, and its recommendation effectiveness may be limited for users with sparse interaction data. Large language models (LLMs) have powerful natural language understanding and generation capabilities, and can learn complex semantic relationships from large amounts of text data. Therefore, introducing LLMs into recommendation systems can leverage their deep understanding of user preferences and item features to complement the shortcomings of traditional models. However, traditional methods that directly interact with LLMs often perform calculations for each user and item to ensure accuracy, requiring the processing of large amounts of text data. This results in high computational costs and hinders efficient application in large-scale systems. Existing methods lack effective methods that can leverage LLMs to enhance the integrity and richness of graph structures to ensure the high performance of graph convolutional neural networks, while also controlling the cost of interacting with LLMs to ensure system scalability and real-time performance, while also improving the understanding of user preferences and the accuracy of recommendations. Summary of the Invention

[0004] The present invention overcomes the problems of high computational interaction cost, poor scalability and large system noise introduced when using LLM to enhance the user-item interaction graph of the recommendation system in the prior art. It provides a graph enhancement method based on low-cost interaction of LLM. Before using LLM to interact with user-item data, items and users are screened, and representative items and users with less interaction data are screened. These screened items and users are analyzed using LLM to establish new predictive connections. The integrity and richness of the user-item interaction graph are effectively improved at a computational cost far lower than that of conventional LLM usage, and high-quality predictive connections are further screened to reduce the introduced system noise. The scalability and real-time performance of the system are ensured by the low-cost LLM interaction method, while the system's understanding of user preferences and the accuracy of recommendations are improved.

[0005] In order to achieve the above object, the present invention adopts the following scheme:

[0006] The graph enhancement method based on LLM low-cost interaction includes the following steps:

[0007] S1: Obtain basic data of users and items in the target database, and construct a user-item interaction matrix, a user-item rating matrix, and a multi-label item type matrix based on the basic data;

[0008] S2: Calculate the user's item type preference vector based on the user-item rating matrix and the multi-label item type matrix;

[0009] S3: Calculate the cosine similarity between the type vectors of items rated by the user and the type preference vector of the user. Use the number of ratings for each item in the database as the LLM model's understanding of it. Calculate the comprehensive rating of the item based on the cosine similarity and understanding. Select several items from high to low in order of comprehensive rating to form a rating history set.

[0010] S4: Construct a directed weighted hypergraph with users as nodes and items as hyperedges. Use the PageRank algorithm to calculate the importance score of each node. Select several nodes in ascending order of importance. Use the LLM model to filter out items that users may like or dislike from the item candidate set based on the descriptive prompts of the node users. The descriptive prompts include task instructions, user attributes, evaluation history set, and item candidate set. Build new predictive connections between the nodes and the filtered items.

[0011] S5: constructing a user-item bipartite graph based on the user-item interaction matrix, and constructing a graph neural network model based on the user-item bipartite graph to calculate the original loss of the recommendation system;

[0012] S6: Add each new prediction connection to the user-item bipartite graph and calculate the new loss of the recommendation system. If the new loss after adding a new prediction connection is less than the original loss, then retain this prediction connection, otherwise delete it.

[0013] Preferably, the basic data includes user basic information, item basic information and user rating data for items, the user basic information includes the user's age, gender, occupation and place of origin, and the item basic information includes the item's ID, type, name and release time.

[0014] Preferably, an element in the user-item interaction matrix indicates whether a user has interacted with an item. If a user has interacted with an item, the corresponding element value is 1, otherwise the corresponding element value is 0.

[0015] The elements in the user-item rating matrix are the user's ratings for each item, and if there is no rating, the value is 0;

[0016] The elements in the multi-label item type matrix represent the type labels to which the items belong. If an item belongs to a type label, the corresponding element value is 1, otherwise the corresponding element value is 0.

[0017] Preferably, the type preference vector is calculated using a scoring deviation method, comprising the following steps:

[0018] To calculate the user rating, use the following formula:

[0019]

[0020] in, For users The average rating of For users The number of items reviewed, is a user For items 's rating;

[0021] Calculate the deviation of users' ratings for each item using the following formula:

[0022]

[0023] in, For users For items Rating bias;

[0024] The rating bias is used to weight the type vector of each item that the user has interacted with using the following formula:

[0025]

[0026] Among them, For users The type preference vector of For items in the multi-label item type matrix The corresponding type vector;

[0027] The type preference vector is normalized:

[0028]

[0029] in, is the normalized type preference vector.

[0030] As a preference, the weighted hypergraph constructed in step S4 , where the node set , is the user set, the hyperedge set , For item sets, Represents a slave node Pointing to the hyperedge The directed weights on Indicates that from the hyperedge Pointing to a node A directed weight, the weight value is equal to the node The user represented by the hyperedge The rating of the represented item, super edge Connects all users who have rated the item represented by this hyperedge.

[0031] Preferably, the cosine similarity is calculated using the following formula:

[0032]

[0033] in, For users Type preference vector and items The absolute value of the cosine similarity of the type vector, is the normalized type preference vector, is the normalized type vector.

[0034] Preferably, calculating the comprehensive score of an item includes the following steps:

[0035] Normalize the cosine similarity:

[0036]

[0037] in, is the normalized cosine similarity, and are the upper and lower limits of the absolute value of cosine similarity respectively;

[0038] Normalize the LLM model's understanding of the item:

[0039]

[0040] in, For LLM model items level of understanding, is the normalized understanding level, and are the maximum and minimum values ​​of the LLM model’s understanding of each item;

[0041] Calculate the overall score:

[0042]

[0043] in, For users Items obtained Comprehensive score, weight coefficient The value range is 0 to 1.

[0044] Preferably, calculating the importance score comprises the following steps:

[0045] Compute transition probabilities between nodes in a directed weighted hypergraph:

[0046]

[0047]

[0048] in For slave nodes Through the super edge Arrival Node The transition probability, For nodes The neighbor hyperedge set of For super edge The set of neighbor nodes of For slave nodes Hyperedge pointing to its neighbor The directed weight of From the super edge Point to its neighbor node The directed weight of It is a slave node The sum of the directed weights of hyperedges pointing to all its neighbors, From the super edge The sum of the directed weights pointing to all its neighbor nodes, For slave nodes Passing through all connected nodes and Hyperedge set Arrival Node The transition probability of

[0049] The transfer matrix is ​​obtained by calculating the transfer probability between all nodes , the elements in the transfer matrix Represents a slave node To Node The transition probability, is the number of nodes;

[0050] The importance score is calculated iteratively using the PageRank algorithm:

[0051]

[0052] in, is the damping coefficient, the initial importance score of the node , iterate until , is the convergence threshold.

[0053] As a preferred method, the user-item bipartite graph constructed , where the node set , is the user node set, is the set of item nodes and the set of edges ,in , , is the user-item interaction matrix;

[0054] The constructed graph neural network model adopts the following propagation rule formula:

[0055]

[0056]

[0057] in, is the activation function, and denote the normalized adjacency matrices from user to item and from item to user, respectively. is the node representation matrix of the k-1th layer, represents the learnable weight matrix of the kth layer;

[0058] The features of users and items output by the graph neural network model are represented as follows:

[0059]

[0060]

[0061] in, is the user’s feature representation, is the characteristic representation of the item, is the number of propagation layers, For the Importance parameter for layer graph convolution operations.

[0062] Preferably, the original loss and the new loss are calculated as follows:

[0063] For the original bipartite graph without adding new prediction connections , calculate the original loss :

[0064]

[0065] in, is the prediction output of the original model, is the true label, is the BPR loss function;

[0066] For the new bipartite graph with the new prediction connection , calculate the new loss :

[0067]

[0068] in, is the predicted output of the new model.

[0069] The present invention has at least the following beneficial effects: (1) By introducing LLM into the recommendation system, its deep understanding of user preferences and item features can be utilized to supplement the deficiency of traditional recommendation models in not being able to fully recognize user preferences, thereby improving the accuracy of recommendations; (2) by selecting representative items to form evaluation history sets and item candidate sets, and combining necessary information such as user attributes into descriptive prompts, the number of tokens interacting with LLM is reduced, while ensuring that LLM can fully understand user preferences, only users with less connection interaction data are selected to interact with LLM, the number of interactions with LLM is reduced, the cost requirements of computing overhead are reduced, and the scalability of the system is improved; (3) by verifying the loss rate of new connections, only connections that have a positive impact on the performance of the recommendation system are retained to ensure the stability of system performance and avoid introducing low-quality system noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1A schematic diagram of the LLM low-cost interactive graph enhancement method provided by the present invention;

[0071] Figure 2 A flowchart of the steps of the LLM low-cost interactive graph enhancement method provided by the present invention;

[0072] Figure 3 This is an example of a user-item bipartite graph of the present invention. DETAILED DESCRIPTION

[0073] The present invention will be described in further detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0074] like Figure 1-3 As shown, the graph enhancement method based on LLM low-cost interaction provided by the present invention includes the following steps:

[0075] S1: Obtain basic data on users and items in the target database and construct a user-item interaction matrix, a user-item rating matrix, and a multi-label item type matrix based on this basic data. The target database is the data source and service foundation of the recommendation system, containing all user and item data. Item content can include various types of items in various fields, such as movies, music, novels, short videos, and goods. As an example, the target database is a movie database, a structured relational database, where the items are movies. The elements of the user-item interaction matrix include whether each user has an interaction relationship with each movie. Interaction relationships can optionally include various movie-related interactions, such as whether the user has searched for or opened the movie, how long the user has watched the movie, whether the user has rated the movie, or communicated with other users about the movie. The user-item rating matrix records users' positive and negative reviews of all movies, as well as their specific rating scores. The database categorizes, stores, and displays movies according to their tags, such as genre, release date, region, and style. The type vector formed by each row or column in the multi-label item type matrix records the tag type assigned to the corresponding movie.

[0076] S2: Calculate the user's item type preference vector based on the user-item rating matrix and the multi-label item type matrix. A user's item rating directly reflects their interest in the item. By weighting and accumulating the label types of the items rated by the user using rating bias, we can accurately determine the user's preference for items of each label type, thereby obtaining the user's item type preference vector. For example, if a user rates a movie, the rating value is weighted to the movie's type vector. The sum of all weighted type vectors represents the user's preference for all movie types, i.e., the user's movie type preference vector.

[0077] S3: Calculate the cosine similarity between the type vector of the items evaluated by the user and the type preference vector of the user, use the number of ratings of each item in the database as the degree of understanding of the LLM model, calculate the comprehensive score of the item based on the cosine similarity and the degree of understanding, and select several items from high to low according to the comprehensive score to form an evaluation history set; cosine similarity is a commonly used indicator to reflect the similarity between two vectors. During the calculation process, the two vectors can be normalized to facilitate calculation and comparison. The higher the cosine similarity, the closer the directions of the two non-empty vectors are; if the cosine similarity between the type vector of the item and the type preference vector of the user is positive, the larger the absolute value of the cosine similarity, the more the user likes the item; conversely, if the cosine similarity is negative, the larger the absolute value of the cosine similarity, the more the user dislikes the item. In the recommendation system, items that users particularly like or particularly dislike have a strong guiding role in the accuracy of the recommendation. Since different items have different degrees of popularity and user attention, their rating information will also have different ability to reflect customer preferences. This is reflected in the LLM model's analysis process as the model's understanding of different items. The higher the understanding, the more referenceable the LLM model's analysis results will be. The comprehensive score obtained by setting weight coefficients based on importance to balance the influence of cosine similarity and understanding can more accurately represent the reference value of this item for user preference analysis. Taking movie items as an example, the movies that users have reviewed are sorted from high to low according to the comprehensive score, and the top k movies are selected to form a selected evaluation history set. , used to build user Descriptive hints In order to reduce the number of tokens, You can take smaller values, such as Taking 4 means selecting the 4 movies with the highest comprehensive scores as representative members of the evaluation history set.

[0078] S4: Construct a directed weighted hypergraph with users as nodes and items as hyperedges. Use the PageRank algorithm to calculate the importance score of each node. Select several nodes in order of importance from low to high. Use the LLM model to filter out items that users may like or dislike from the item candidate set based on the descriptive prompts of the node users. Construct new predictive connections between the nodes and the filtered items. The descriptive prompts include task instructions, user attributes, evaluation history set, and item candidate set.

[0079] For example, in the descriptive prompt for a movie item:

[0080] Task Instructions: You are a movie recommendation system and required torecommend user with movies based on user profiles and their ratinghistory. Consider movie titles, years, ratings, and genres, alongside user attributes such as age, gender, occupation, and nationality for personalized suggestions.

[0081] The above task instructions are used as a simple example in the movie item scenario. In actual use, the content and organization of task instructions will vary depending on the item type, data source, etc. of the recommendation system.

[0082] User attributes, which consist of the obtained user information, generate a text description of the user profile based on the pre-processed user basic information.

[0083] Evaluation history set: the selected evaluation history set obtained in step S3 .

[0084] Movie Candidates: Movie Candidates Generated by the LightGCN recommendation model without graph enhancement. This is because LLM cannot rank all the large number of items. These candidate samples Samples with high prediction scores can be valuable positive samples or difficult-to-distinguish negative samples. By screening and distinguishing potential, valuable positive and negative samples, graph neural networks can learn higher-quality interaction information after graph augmentation.

[0085] Hypergraph is an extension of traditional graph, consisting of nodes and hyperedges. Unlike ordinary graphs, a hyperedge in a hypergraph can connect any number of nodes. Directed weighted hypergraph has different weights on the connection paths of different nodes. , where the node set , hyperedge set , Represents a slave node Pointing to the hyperedge A directed weight on Indicates that from the hyperedge Pointing to a node A directed weight, the weight value is equal to the node The user represented by the hyperedge The rating of the item represented by it. There are multiple directed weights on a hyperedge. Connects all users who have rated the item represented by this hyperedge.

[0086] because A directed weighting mechanism is used, so the weights are different when propagating forward and backward between two nodes. The size of the directed weight indicates the possibility of transferring from one node to another along this hyperedge. As an example, for the hypergraph structure constructed above, in the random walk process of the PageRank algorithm, the walk path can reflect the user's preference for the items they interact with together. Specifically, there is an item and two users ,user For items The rating is 1 point, user For items The rating is 5 points, then in the graph structure, the random walker starts from the user Passing items Reach users , and from the user Passing items Reach users The process has different wandering effects.

[0087] The easier it is for a node to be connected to by other nodes, the more important and central this node is in the hypergraph. The PageRank algorithm uses the node and hyperedge connection structure in the hypergraph to score nodes. Each connection can be regarded as a score for the target node. At each node position, the PageRank algorithm calculates the probability of transferring from this node to another node, and a damping factor can be set to make it have a smaller probability of not transferring with the connection but randomly transferring to any node. The PageRank algorithm assigns an initial score to all nodes, and then updates the score of each node through continuous iterative calculation until the score stabilizes and reaches a balanced state. The importance score of each node in the hypergraph can be obtained. The users corresponding to the nodes with the lowest importance scores are relatively isolated from other users. Therefore, it is necessary to use the LLM model to filter out items that these users may like or dislike from the candidate items, and to establish new connection relationships for these users. As an example, the descriptive prompts of k users are added. Use OpenAI's API to upload to the Large Language Model (LLM) with descriptive prompts It contains information such as the user's user attributes, evaluation history set, and item candidate set. The item candidate set is a representative set of items selected from the database, which can ensure that LLM can fully understand user preferences. The representative items and necessary user information are constructed into concise descriptive prompts, which can reduce the number of tokens passed to the LLM model and reduce the interaction cost with LLM. In an actual operation example, OpenAI's API is used to send the constructed descriptive prompt Prompt to the large language model LLM; the response message of the large language model LLM is checked by polling; the script traverses the response message of the model, extracts the predicted connection data generated by LLM and saves it locally; during the request process, if an HTTP error, response parsing error or other abnormal situation occurs, the error message is captured and printed, and appropriate processing is performed, such as re-requesting or waiting for a period of time before requesting again; the predicted connection data generated by LLM is the ID of the item selected by LLM from the item candidate set. This process is equivalent to using LLM to enrich the user's historical interaction record. A new predicted connection is constructed between the node and the filtered item. This predicted connection is not a hyperedge connection relationship added to the hypergraph, but a predictive interaction relationship is established between the user and the corresponding item. Figure 3 The connection relationship in the user-item bipartite graph shown. The predicted connections generated by LLM form a connection set , which means using the LLM model to process low-importance user sets Descriptive hint for users All prediction connections generated.

[0088] S5: Construct a user-item bipartite graph based on the user-item interaction matrix, and construct a graph neural network model based on the user-item bipartite graph to calculate the original loss of the recommendation system; a bipartite graph is a special type of graph whose vertices can be divided into two disjoint sets, namely, a user node set and an item node set. Each edge in the bipartite graph is used to connect vertices in the two sets, and there is no edge connecting vertices in the same set, that is, the connection relationship only exists between users and items, and there is no connection relationship between users and items. Figure 3 The figure shows an example of a user-item bipartite graph, showing a portion of an actual bipartite graph. Blank nodes represent users, shaded nodes represent items, solid lines connecting the nodes are original connections, and dashed lines are new predicted connections generated by the LLM. These new predicted connections create new interaction relationships between users who were previously relatively isolated and had little interaction information. As an example, the graph neural network described uses the Lightweight Graph Convolutional Neural Network (LightGCN) model. LightGCN is used to perform forward propagation on the user-item bipartite graph constructed from the user-item interaction matrix to obtain the predicted output of the original graph. A loss function suitable for recommendation systems, such as the Bayesian personalized ranking loss function, is used to calculate the original loss between the LightGCN model's predicted output and the true labels.

[0089] S6: Add each new prediction connection to the user-item bipartite graph and calculate the new loss of the recommendation system. If the new loss after adding a new prediction connection is less than the original loss, retain this prediction connection, otherwise delete it. After all the prediction connections are added to the user-item bipartite graph for verification, the low-quality connections are removed to obtain a new predicted user-item bipartite graph with richer interactive relationships. It is used as the model parameter of the graph neural network of the recommendation system. The user-item interaction matrix is ​​updated according to the user-item bipartite graph as the recommendation parameter. Compared with the original user-item bipartite graph that has not been processed using the method provided in this application, the new user-item bipartite graph is used as the input of the graph neural convolutional network, so that the recommendation system can recommend items that are more likely to suit the user's preferences. This improvement in recommendation performance is particularly obvious for users who originally had fewer interactive relationships. The quality of the new prediction connection is evaluated by calculating the difference in loss before and after adding the new prediction connection. If the new loss is , the original loss is , then the loss difference :

[0090]

[0091] if , then the new predicted connection The model loss is reduced, and it is considered to be a high-quality connection and retained; if , then the new predicted connection If the model loss cannot be reduced, it is considered a low-quality connection and removed to avoid introducing new system noise. The recommendation system corrects the item recommendation information for the user based on the updated user-item interaction matrix.

[0092] According to Figure 1 The principle of the method shown here provides an example implementation flow: First, preprocess the database data, including preliminary data processing steps such as data integration, classification, and screening. Then, representative items are selected to reduce the number of tokens passed to the LLM model. Furthermore, users with low interactivity (i.e., low importance scores) are screened out to reduce the number of users who need to interact with the LLM. This two-pronged approach enhances the user-item interaction graph while significantly reducing the interaction cost with the LLM. When selecting representative items, a similarity score between the item and the user's type preference is calculated, while a score indicating the LLM's understanding of the item is calculated. Based on this combined similarity and understanding scores, the most representative item queue is selected. When screening users with poor interactivity, a directed weighted hypergraph is constructed from the user's perspective, and the PageRank algorithm is used to calculate the importance score of each user node in the directed weighted hypergraph. Then, user nodes with low importance scores, that is, users with poor interactivity, are selected, and the information of these users is interacted with LLM. Before interacting with LLM, descriptive prompts are first constructed for users to simplify the interaction process and reduce the amount of interaction data. Based on the powerful understanding and analysis capabilities of LLM, LLM generates predictive interaction connections between interacting users and items, and then adds the interaction connections to the user-item bipartite graph and deletes the low-quality connections according to the changes in the before and after losses of the graph neural network. The new user-item bipartite graph is obtained for the graph neural network model of the recommendation system to improve the recommendation accuracy of the recommendation system.

[0093] Introducing LLM into the recommendation system can leverage its deep understanding of user preferences and item features, supplementing the shortcomings of traditional recommendation models in fully recognizing user preferences and improving the accuracy of recommendations. By selecting representative items to form evaluation history sets and item candidate sets, and combining necessary information such as user attributes into descriptive prompts, the number of tokens interacting with the LLM is reduced, while ensuring that the LLM can fully understand user preferences. Only users with less connection interaction data are selected to interact with the LLM, reducing the number of interactions with the LLM. Since communication with high-performance large language models such as ChatGPT requires charging based on the number of interactions, the method provided in this application reduces the interaction with the LLM model and can undoubtedly directly reduce the overall cost requirements of computing overhead. Moreover, since only a portion of users and items are screened, even when facing recommendation systems with large user and item sets, the completeness and richness of the user-item bipartite graph connection relationship can be improved without increasing the LLM interaction cost significantly, thereby improving the scalability of the system and making it applicable to recommendation systems of various types and data sizes. By verifying the loss rate of new connections, only connections with positive benefits to the performance of the recommendation system are retained to ensure system performance stability and avoid introducing low-quality system noise.

[0094] In another technical solution, the basic data includes basic user information, basic item information, and user ratings of the items. The basic user information includes the user's age, gender, occupation, and place of origin, and the basic item information includes the item's ID, type, name, and release date. Taking movie items as an example, the target movie database is Movielens. The basic movie information includes the movie's unique identifier in the target movie database, the movie title, the movie's release date, and the movie genre. Movie genre tags include: Unknown, Action, Adventure, Animation, Children's, Comedy, Crime, Documentary, Drama, Fantasy, Film-Noir, Horror, Musical, Mystery, Romance, Sci-Fi, Thriller, War, and Western.

[0095] The elements in the user-item interaction matrix represent whether a user has interacted with an item. If a user has interacted with an item, the corresponding element value is 1, otherwise the corresponding element value is 0;

[0096] The elements in the user-item rating matrix are the user's ratings for each item, and if there is no rating, the value is 0;

[0097] The elements in the multi-label item type matrix represent the type labels to which the items belong. If an item belongs to a type label, the corresponding element value is 1, otherwise the corresponding element value is 0.

[0098] In another technical solution, the type preference vector is calculated using a scoring bias method, including the following steps:

[0099] To calculate the user rating, use the following formula:

[0100]

[0101] Among them, For users The average rating of For users The number of items reviewed, is a user For items 's rating;

[0102] Calculate the deviation of users' ratings for each item using the following formula:

[0103]

[0104] in, For users For items The rating bias can reflect the user's actual preference for the item. By considering positive and negative biases, we can capture both the types that users like and the types that they dislike. ): indicates that the user likes the item more than the average; negative deviation ( ): indicates that the user dislikes the item more than the average.

[0105] The rating bias is used to weight the type vector of each item that the user has interacted with using the following formula:

[0106]

[0107] in, For users The type preference vector of For items in the multi-label item type matrix The corresponding type vector. Type vector Derived from the multi-label item type matrix, , item type collection , if p=19, then the number of all possible item types is 19. Multi-label item type matrix , in the multi-label item type matrix In the elements Indicates items Is it of type ,Right now: Using Rating Bias To weight the type vector of each item browsed by the user, the type that the user likes will get a positive weight, while the type that the user dislikes will get a negative weight.

[0108] The type preference vector is normalized:

[0109]

[0110] in, This is the normalized type preference vector. Different users may have different rating habits; for example, some may generally rate high or low. Normalization can reduce the impact of these differences in rating scales. The normalized vector length is 1, eliminating the influence of the number of ratings and the magnitude of the ratings. This allows similarity calculations to be based solely on directional similarity, making calculations fast and easy to compare.

[0111] In another technical solution, the cosine similarity is calculated using the following formula:

[0112]

[0113] in, For users Type preference vector and items The absolute value of the cosine similarity of the type vector, is the normalized type preference vector, is the normalized type vector, that is: Items may have multiple types. Normalization can make the length of the type vectors consistent, making them easier to compare. Similarity without an absolute value can be positive or negative. A positive value indicates that the item type matches the user's preferred type, while a negative value indicates that the item type matches the user's disliked type. Larger values ​​better represent the user's preference, so we use cosine similarity with an absolute value.

[0114] Calculating an item's overall rating involves the following steps:

[0115] Normalize the cosine similarity:

[0116]

[0117] in, is the normalized cosine similarity, and are the upper and lower limits of the absolute value of cosine similarity respectively;

[0118] Normalize the LLM model's understanding of the item:

[0119]

[0120] in, For LLM model items level of understanding, is the normalized understanding level, and are the maximum and minimum values ​​of the LLM model’s understanding of each item;

[0121] Calculate the overall score:

[0122]

[0123] in, For users Items obtained Comprehensive score, weight coefficient The value range is from 0 to 1. Weight coefficient Adjustments are made based on actual conditions to balance the importance of the cosine similarity of the user's type preference vector and the LLM's level of familiarity. Constructing a comprehensive score ensures that items are selected that are both representative of the user's preferences and familiar to the LLM.

[0124] In another technical solution, calculating the importance score includes the following steps:

[0125] Compute transition probabilities between nodes in a directed weighted hypergraph:

[0126]

[0127]

[0128] in For slave nodes Through the super edge Arrival Node The transition probability, For nodes The neighbor hyperedge set of For super edge The set of neighbor nodes of For slave nodes Hyperedge pointing to its neighbor The directed weight of From the super edge Point to its neighbor node The directed weight of It is a slave node The sum of the directed weights of hyperedges pointing to all its neighbors, From the super edge The sum of the directed weights pointing to all its neighbor nodes, For slave nodes Passing through all connected nodes and Hyperedge set Arrival Node The transition probability of and are used to normalize the transition probability. Any two nodes , there exists a hyperedge subset , including all connected nodes and nodes The super edge, ,condition Represents a hyperedge Connecting nodes simultaneously and nodes .

[0129] The transfer matrix is ​​obtained by calculating the transfer probability between all nodes , the elements in the transfer matrix Represents a slave node To Node The transition probability, is the number of nodes;

[0130] The importance score is calculated iteratively using the PageRank algorithm:

[0131]

[0132] in, is the damping coefficient, the initial importance score of the node , iterate until , is the convergence threshold. The value of will affect the convergence speed and results of the algorithm, and is usually set to 0.85.

[0133] In another technical solution, the user-item bipartite graph is constructed , where the node set , is the user node set, is the set of item nodes and the set of edges ,in , , is the user-item interaction matrix;

[0134] The constructed graph neural network model adopts the following propagation rule formula:

[0135]

[0136]

[0137] in, is the activation function, and denote the normalized adjacency matrices from user to item and from item to user, respectively. The nodes in the k-1th layer represent matrices, represents the learnable weight matrix of the kth layer;

[0138] As an example, the LightGCN model is constructed, which adopts the following propagation rule formula:

[0139]

[0140]

[0141] in, Represents user A collection of item nodes that have been interacted with, Representation and items The set of user nodes that have interacted; is a symmetric normalization term that prevents graph convolution operations from increasing the size of embeddings.

[0142] The features of users and items output by the LightGCN model are represented as follows:

[0143]

[0144]

[0145] in, is the user’s feature representation, is the characteristic representation of the item, is the number of layers of graph convolution operation, For the Importance parameter for layer graph convolution operations. , which can be used as a hyperparameter that needs to be manually tuned, or as a model parameter (e.g., the output of the attention network) that can be automatically optimized. As an example, Evenly set to .

[0146] The original loss and the new loss are calculated as follows:

[0147] For the original bipartite graph without adding new prediction connections , calculate the original loss :

[0148]

[0149] in, is the prediction output of the original model, is the true label, is the BPR loss function; the model prediction is defined as the inner product of the final representation of the user and the item:

[0150]

[0151] Bayesian personalized ranking (BPR) loss is used as the model loss function:

[0152]

[0153] σ(·) is the Sigmoid function, 𝜆 is the regularization coefficient, and Θ is the set of model parameters.

[0154] For the new bipartite graph with the new prediction connection , calculate the new loss :

[0155]

[0156] in, is the predicted output of the new model.

[0157] As an example, construct the original user-item bipartite graph structure , where the node set ; Edge set: ,in , , is the user-item interaction matrix.

[0158] In the original picture Above, use the LightGCN model for forward propagation to get the predicted output:

[0159]

[0160] in, is a randomly initialized node feature matrix.

[0161] Calculate the original loss .

[0162] Second, for each new prediction connection , construct a new edge set , construct a new graph In the new map On the , use LightGCN to perform forward propagation and get the new prediction output:

[0163]

[0164] Calculate new loss .

[0165] Finally, calculate the loss difference And evaluate the quality of predicted connections to make trade-offs.

[0166] It should be noted that although the steps are described above in a specific order, this does not necessarily mean that the steps must be performed in this specific order. In fact, some of these steps can be performed concurrently or even in a different order, as long as the required functions can be achieved. The number of devices and processing scales described here are intended to simplify the description of the present invention. Applications, modifications, and variations of the present invention will be apparent to those skilled in the art.

[0167] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A graph enhancement method based on LLM low-cost interaction, characterized by: The following steps are involved: S1: Obtain basic data of users and items in the target database, and construct a user-item interaction matrix, a user-item rating matrix, and a multi-label item type matrix based on the basic data; S2: Calculate the user's item type preference vector based on the user-item rating matrix and the multi-label item type matrix; S3: Calculate the cosine similarity between the type vectors of items rated by the user and the type preference vector of the user. Use the number of ratings for each item in the database as the LLM model's understanding of it. Calculate the comprehensive rating of the item based on the cosine similarity and understanding. Select several items from high to low in order of comprehensive rating to form a rating history set. S4: Construct a directed weighted hypergraph with users as nodes and items as hyperedges. Use the PageRank algorithm to calculate the importance score of each node. Select several nodes in ascending order of importance. Use the LLM model to filter out items that users may like or dislike from the item candidate set based on the descriptive prompts of the node users. The descriptive prompts include task instructions, user attributes, evaluation history set, and item candidate set. Build new predictive connections between the nodes and the filtered items. S5: constructing a user-item bipartite graph based on the user-item interaction matrix, and constructing a graph neural network model based on the user-item bipartite graph to calculate the original loss of the recommendation system; S6: Add each new prediction connection to the user-item bipartite graph and calculate the new loss of the recommendation system. If the new loss after adding a new prediction connection is less than the original loss, then retain this prediction connection, otherwise delete it.

2. The LLM-based low-cost interactive graph enhancement method according to claim 1, characterized in that: The basic data includes user basic information, item basic information and user rating data for the item. The user basic information includes the user's age, gender, occupation and place of origin. The item basic information includes the item's ID, type, name and release time.

3. The graph enhancement method based on LLM low-cost interaction according to claim 1 is characterized in that: The elements in the user-item interaction matrix represent whether a user has interacted with an item. If a user has interacted with an item, the corresponding element value is 1, otherwise the corresponding element value is 0; The elements in the user-item rating matrix are the user's ratings for each item, and if there is no rating, the value is 0; The elements in the multi-label item type matrix represent the type labels to which the items belong. If an item belongs to a type label, the corresponding element value is 1, otherwise the corresponding element value is 0.

4. The graph enhancement method based on LLM low-cost interaction according to claim 1, characterized in that: The type preference vector is calculated using a scoring bias method, including the following steps: To calculate the user rating, use the following formula: Among them, For users The average rating of For users The number of items reviewed, is a user For items 's rating; Calculate the deviation of users' ratings for each item using the following formula: in, For users For items Rating bias; The rating bias is used to weight the type vector of each item that the user has interacted with using the following formula: in, For users The type preference vector of For items in the multi-label item type matrix The corresponding type vector; The type preference vector is normalized: in, is the normalized type preference vector.

5. The LLM-based low-cost interactive graph enhancement method according to claim 4, characterized in that: The directed weighted hypergraph constructed in step S4 , where the node set , is the user set, the hyperedge set , For item sets, Represents a slave node Pointing to the hyperedge The directed weights on Indicates that from the hyperedge Pointing to a node A directed weight, the weight value is equal to the node The user represented by the hyperedge The rating of the represented item, super edge Connects all users who have rated the item represented by this hyperedge.

6. The LLM-based low-cost interactive graph enhancement method according to claim 1, characterized in that: The cosine similarity is calculated using the following formula: in, For users Type preference vector and items The absolute value of the cosine similarity of the type vector, is the normalized type preference vector, is the normalized type vector.

7. The LLM-based low-cost interactive graph enhancement method according to claim 6, characterized in that: Calculating an item's overall rating involves the following steps: Normalize the cosine similarity: in, is the normalized cosine similarity, and are the upper and lower limits of the absolute value of cosine similarity respectively; Normalize the LLM model's understanding of the item: in, For LLM model items level of understanding, is the normalized understanding level, and are the maximum and minimum values ​​of the LLM model’s understanding of each item; Calculate the overall score: in, For users Items obtained Comprehensive score, weight coefficient The value range is 0 to 1.

8. The graph enhancement method based on LLM low-cost interaction according to claim 1 is characterized in that: Calculating the importance score involves the following steps: Compute transition probabilities between nodes in a directed weighted hypergraph: in For slave nodes Through the super edge Arrival Node The transition probability, For nodes The neighbor hyperedge set of For super edge The set of neighbor nodes of For slave nodes Hyperedge pointing to its neighbor The directed weight of From the super edge Points to its neighbor nodes The directed weight of It is a slave node The sum of the directed weights of hyperedges pointing to all its neighbors, From the super edge The sum of the directed weights pointing to all its neighbor nodes is the slave node Passing through all connected nodes and Hyperedge set Arrival Node The transition probability of The transfer matrix is ​​obtained by calculating the transfer probability between all nodes , the elements in the transfer matrix Represents a slave node To Node The transition probability, is the number of nodes; The importance score is calculated iteratively using the PageRank algorithm: in, is the damping coefficient, the initial importance score of the node , iterate until , is the convergence threshold.

9. The LLM-based low-cost interactive graph enhancement method according to claim 1, characterized in that: Constructed user-item bipartite graph , where the node set , is the user node set, is the set of item nodes and the set of edges ,in , , is the user-item interaction matrix; The constructed graph neural network model adopts the following propagation rule formula: in, is the activation function, and denote the normalized adjacency matrices from user to item and from item to user, respectively. The nodes in the k-1th layer represent matrices, represents the learnable weight matrix of the kth layer; The features of users and items output by the graph neural network model are represented as follows: in, is the user’s feature representation, is the characteristic representation of the item, is the number of propagation layers, For the Importance parameter for layer graph convolution operations.

10. The LLM-based low-cost interactive graph enhancement method according to claim 9, characterized in that: The original loss and the new loss are calculated as follows: For the original bipartite graph without adding new prediction connections , calculate the original loss : in, is the prediction output of the original model, is the true label, is the BPR loss function; For the new bipartite graph with the new prediction connection , calculate the new loss : in, is the predicted output of the new model.

Images