Hyperbolic hypergraph neural network and attention mechanism based conversational recommendation system and method

Abstract

A conversation recommendation system and method based on hyperbolic hypergraph neural network and attention mechanism, the system comprises a hyperbolic hypergraph neural network module, a conversation learning module SLM, an interest extraction module IEM and a prediction module;The method comprises the following operation steps: (1) the hyperbolic hypergraph neural network module generates high-quality initialization representation for all items contained in all conversations;(2) the conversation learning module SLM calculates the new representation of all items containing data space structural properties of each conversation;(3) the interest extraction module IEM calculates the new representation of each conversation containing the long-distance dependence relationship between each item in the conversation and the relative position information of the item;(4) the prediction module generates the final Top-K prediction for each conversation.

Description

Technical Field

[0001] This invention relates to a conversation recommendation system and method based on hyperbolic hypergraph neural networks and attention mechanisms, belonging to the field of information technology, and particularly to the field of conversation recommendation technology. Background Technology

[0002] Conversational recommendation refers to predicting the most likely product or item a user will click on in the next moment, based solely on the user's previous historical behavior sequence without requiring any other user information. Currently, commonly used conversational recommendation technologies mainly fall into two categories: sequence models based on the Transformer architecture and sequence models based on graph neural network architectures.

[0003] Existing technologies in conversation recommendation research mainly focus on using graph neural networks or self-attention mechanisms alone to capture the features of items and sequences, without using both in parallel. However, these two techniques have different focuses. Graph neural networks mainly capture the spatial structure of data, while ignoring the sequential order of conversation data; self-attention mechanisms, as a sequence model, usually focus on the existing order of conversation data, but lack the ability to capture the spatial structure of the data.

[0004] Since graph neural networks (GNNs) lack the ability to learn the positional representations of items within a conversation, existing GNN-based techniques append positional representations to item representations only after information propagation is complete. However, in this case, the positional representations do not undergo information propagation and fusion, making it difficult to enrich their representations and thus reducing the model's ability to learn item positions. Sequence models, such as self-attention mechanisms, are designed to focus on relative positional relationships within a sequence rather than handling arbitrary spatial structures. This property is obviously superior for ordinary sequence data, but in the context of conversational recommendation, extensive work has demonstrated the interrelationships between different items, and introducing a constructed global item graph into the model is beneficial for overall prediction. Therefore, existing sequence-based techniques, lacking the ability to learn the spatial structure of the data, also reduce the model's predictive ability to some extent.

[0005] How to combine the aforementioned technical features of graph neural network models and self-attention models in parallel to learn rich location representations and spatial structure properties of items in a conversation, thereby improving the accuracy of conversation recommendations, has become a pressing technical challenge in the field of conversation recommendation technology. Summary of the Invention

[0006] In view of this, the purpose of this invention is to develop a conversation recommendation technology that can fully combine and utilize graph neural network models and self-attention models to improve the accuracy of conversation recommendations.

[0007] To achieve the above objectives, this invention proposes a conversation recommendation system based on hyperbolic hypergraph neural networks and attention mechanisms, the system comprising the following modules:

[0008] Hyperbolic Hypergraph Neural Network Module: This module's function is to use a hyperbolic hypergraph neural network to generate a high-quality initial representation H for all items included in the entire session;

[0009] The Session Learning Module (SLM) uses a graph neural network to capture the local spatial structure of data within a session, based on the output H of the hyperbolic hypergraph neural network module, and calculates a new representation of all items containing the spatial structure properties of the data for each session.

[0010] Interest Extraction Module (IEM): This module's function is to calculate, based on the output H of the hyperbolic hypergraph neural network module, a new representation for each session using a sequence model based on a self-attention mechanism. This representation contains long-distance dependencies between items in the session and information on the relative positions of the items.

[0011] Prediction Module: This module's function is to predict the new representations output by the Session Learning Module (SLM). and the new representation output by the interest extraction module IEM. The final representation of the session is calculated. Then based on this final representation Generate the final Top-K predictions for each session, which are the K items corresponding to the K highest predicted values.

[0012] The hyperbolic hypergraph neural network module generates a high-quality initial representation H for all items included in the entire session, and the specific steps include the following:

[0013] (21) Encode and calculate each item contained in each session S to obtain the basic representation E = {e1, e2, e3, ..., e} of all items in all sessions. i ,…,e n}, where e i Let n represent the basic representation of the i-th item among all items contained in all sessions, and n represent the total number of items contained in all sessions.

[0014] (22) Project the basic representation E of all items in all the conversations onto hyperbolic space according to the following formula to obtain the hyperbolic space representation of each item:

[0015]

[0016] In the above formula, Represent the hyperbolic space representation of the i-th item. This indicates that L2 regularization is performed, k = 1 / c, where c represents the curvature of the hyperboloid;

[0017] (23) Build a global item relationship hypergraph based on all sessions. In this global item relationship hypergraph, model each session as a hyperedge, and each hyperedge contains all items in that session; for any item in any session S It will establish undirected connections with all other items in S; define the indicator matrix Z∈R. N×M Let Z represent the adjacency matrix of a hypergraph, i.e., the hypergraph contains M hyperedges and N item nodes, and let Z be the element in the matrix Z. i∈ =1 indicates that the hyperedge in the hypergraph contains item v. i If the value is 0, then it does not include item v. i Define the degree matrix of the nodes. Among them W ∈∈ The weight of the hyperedge ∈ is represented by the degree matrix of the hyperedge.

[0018] (24) Perform L-layer hypergraph convolution calculation on the hyperbolic space representation of each item obtained in step (22), as follows:

[0019] For the aforementioned global item relationship hypergraph, the matrix calculation formula for hypergraph convolution is:

[0020] E hg(l+1) =D -1 ZWB -1 Z T E hg(l)

[0021] In the above formula, E hg(l+1) Let W represent the output of the hypergraph convolution computation at layer l, where W is a learnable parameter matrix representing the weights of all hyperedges, and B is the degree of each hyperedge, defined as... E hg(l) This represents the input for the hypergraph convolution calculation of the l-th layer; the input for the hypergraph convolution calculation of the 1st layer is the hyperbolic space representation of each item obtained in step (22);

[0022] (25) The initial representation H of all items in the entire session is calculated according to the following formula:

[0023]

[0024] In the above formula, L is a hyperparameter representing the number of layers in the hypergraph convolution.

[0025] The SLM (Session Learning Module) module calculates a new representation of all items in each session, containing the spatial structure properties of the data space. The specific content includes the following operational steps:

[0026] (31) Given any session S = {v1, v2, v3, ..., v...} m}, for any one of the items v i (1≤i≤m) all make it similar to the previous item v i-1 and subsequent items v i+1 Create an edge to obtain a session-level graph;

[0027] (32) Based on the importance of different item node pairs in the graph obtained above, and based on the different order in which the items are clicked in the session, the edges established between the aforementioned items are divided into four types: namely, inbound edges, outbound edges, inbound-outbound edges, and self-loops.

[0028] (33) Based on the type of the edge, the weights between different node pairs are calculated using the attention mechanism according to the following formula:

[0029]

[0030] In the above formula, α i,j Represents item v i and items v j The weights between the node pairs formed, h i and h j These represent the item v output by the hyperbolic hypergraph neural network module. i and items v j The initialization representation, r i,j Represents item v i and items v j The type of the edge between them, er i,j The type is r i,j The learnable weight parameters of the edges, ⊙ represents the element-wise dot product operation, and LeakyReLU() is the activation function; Let represent the set of all neighboring items of item i in session S, and exp represent an exponential function with the natural constant e as the base.

[0031] (34) Calculate the new representation of each item in session S containing the data space structure properties according to the following formula:

[0032]

[0033] In the above formula, h iSLMLet i represent the new representation of the i-th item in session S containing data space structure properties. Then, concatenate the new representations of all items in session S to obtain the new representations of all items in session S.

[0034] The Interest Extraction (IEM) module calculates a new representation for each session containing long-distance dependencies between items in the session and the relative position information of the items. The specific content includes the following operational steps:

[0035] (41) The position representation of the item is concatenated into the initial representation of the item output by the hyperbolic hypergraph neural network module according to the following formula:

[0036]

[0037] In the above formula, h i and p i Let these represent the initialization representation and position representation of the i-th item in session S, respectively; Let H represent the item representation of the i-th item in session S after concatenating the position representations; concatenate the above item representations of all items in session S to obtain the session representation H of session S after concatenating the position representations. s ;

[0038] (42) Using the self-attention mechanism, calculate the dependencies between items within session S according to the following formula:

[0039]

[0040] In the above formula, A represents the matrix of dependencies between items within session S, and d is the session representation H. s The dimensions of each item's representation vector are shown below. The subscript T indicates the matrix transpose, and Q, K, and V represent the query matrix, key matrix, and value matrix, respectively.

[0041] Q = LeakyReLU(W Q ×H s )

[0042] K = leakyReLU(W K ×H s )

[0043] V = leakyReLU(W) V ×H s )

[0044] The W here Q W K W VThese represent the learnable weight parameters for different types of matrices;

[0045] (43) The dependency matrix A obtained in step (42) is further nonlinearly calculated using an L-level feedforward neural network FFN, as shown in the following formula:

[0046] A l+1 =LeakyReLU(W l A l +b l )

[0047] In the above formula, W l and b l Let A represent the learnable weight parameters and bias parameters of the l-th layer feedforward neural network FFN, respectively. l+1 A represents the output of the l-th layer of the feedforward neural network. l This represents the input of the l-th layer feedforward neural network; the input of the first layer feedforward neural network is the dependency matrix A obtained in step (42);

[0048] (44) The output of the last layer, i.e., the Lth layer of the feedforward neural network FFN, is the new representation of each session calculated by the interest extraction module (IEM), which contains the long-distance dependencies between items in the session and the relative position information of the items.

[0049]

[0050] The prediction module generates the final Top-K prediction for each session, including the following steps:

[0051] (51) Using a self-attention mechanism, calculate the session S according to the following formula. Characterization and Similarity matrix γ between representations:

[0052]

[0053] In the above formula, W γ ∈R d W IEM ∈R m×1 W SLM ∈R m×m There are three learnable weight matrices, b γ ∈R d It is a learnable bias parameter matrix, σ is the activation function; γ∈R m This is a similarity matrix, where each value represents the degree of correlation between the item at the corresponding position and the next predicted item;

[0054] (52) Based on the similarity matrix γ, the final representation of the session S is calculated according to the following formula.

[0055]

[0056] In the above formula, γ i It is the i-th value in the similarity matrix γ. Represents the characterization of the i-th item in session S;

[0057] (53) Calculate the predicted value of the i-th item according to the following formula:

[0058]

[0059] In the above formula, h i This represents the item v output by the hyperbolic hypergraph neural network module. i Initialization representation;

[0060] (54) Select the K highest predicted values The corresponding K items are used as the final Top-K prediction.

[0061] The system is trained using the following loss function:

[0062] Loss = L pred +βL diff

[0063] In the above formula, β is a control L diff The hyperparameters of the weights;

[0064]

[0065] Among them, y i This represents the actual label of the i-th item;

[0066]

[0067] In the above formula, KL(·) represents the KL divergence;

[0068]

[0069]

[0070]

[0071]

[0072] In the above formula, H I and H SThis represents the new representation of all items in all sessions obtained from the Interest Extraction module (IEM) and the Session Learning module (SLM); that is, the new representation of all sessions. and The new representations of all the items in the text are pieced together separately;

[0073] In the above formula, and δ represents the values ​​calculated by the Interest Extraction Module (IEM) and the Conversation Learning Module (SLM) according to the following formula. adv :

[0074]

[0075] In the above formula, ∈ is a variable used to control δ adv The hyperparameters need to be adjusted according to different data; adv () represents the loss function for perturbed samples; x represents the true sample, and δ represents the perturbation applied to x;

[0076] calculate hour, This represents the new representation output by the Interest Extraction Module (IEM). As the final representation of the session The best prediction made by the prediction module for the session;

[0077] calculate hour, This represents the new representation output by the Session Learning Module (SLM). As the final representation of the session The best prediction made by the prediction module for the session;

[0078] This invention also proposes a session recommendation method based on hyperbolic hypergraph neural networks and attention mechanisms, the method comprising the following steps:

[0079] (1) The hyperbolic hypergraph neural network module uses the hyperbolic hypergraph neural network to generate a high-quality initial representation H for all items included in the entire session;

[0080] (2) The Session Learning Module (SLM) uses a graph neural network to capture the local spatial structure of the data within a session, based on the output H of the hyperbolic hypergraph neural network module, and calculates a new representation of all items containing the spatial structure properties of the data in each session.

[0081] (3) The Interest Extraction Module (IEM) calculates a new representation for each session based on the output H of the hyperbolic hypergraph neural network module and a sequence model based on a self-attention mechanism. This representation contains the long-distance dependencies between items in the session and the relative position information of the items.

[0082] (4) The prediction module uses the representation output by the conversation learning module SLM. and the representation output by the interest extraction module IEM. The final representation of the session is calculated. Then based on this final representation Generate the final Top-K predictions for each session, which are the K items corresponding to the K highest predicted values.

[0083] The beneficial effects of this invention are as follows: This invention explicitly combines the characteristics of graph neural network models and self-attention models, enabling parallel learning of the rich positional representations of each item in the session data, as well as the spatial structural relationships between items; at the same time, this invention uses hyperbolic graph embedding as the method of item embedding, which can more naturally represent the relationships between nodes, has stronger expressive power, and effectively improves the session recommendation accuracy of this invention. Attached Figure Description

[0084] Figure 1 This is a schematic diagram of the structure of the conversation recommendation system based on hyperbolic hypergraph neural network and attention mechanism proposed in this invention.

[0085] Figure 2 This is a flowchart illustrating the conversation recommendation method based on hyperbolic hypergraph neural network and attention mechanism proposed in this invention. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings.

[0087] See Figure 1 This invention introduces a conversation recommendation system based on hyperbolic hypergraph neural network and attention mechanism, which includes the following modules:

[0088] Hyperbolic Hypergraph Neural Network Module: This module's function is to use a hyperbolic hypergraph neural network to generate high-quality initial representations H for all items included in the entire session; the items can be music, movies, or goods from shopping websites, etc.

[0089] The Session Learning Module (SLM) uses a graph neural network to capture the local spatial structure of data within a session, based on the output H of the hyperbolic hypergraph neural network module, and calculates a new representation of all items containing the spatial structure properties of the data for each session.

[0090] Interest Extraction Module (IEM): This module's function is to calculate, based on the output H of the hyperbolic hypergraph neural network module, a new representation for each session using a sequence model based on a self-attention mechanism. This representation contains long-distance dependencies between items in the session and information on the relative positions of the items.

[0091] Prediction Module: This module's function is to predict the new representations output by the Session Learning Module (SLM). and the new representation output by the interest extraction module IEM. The final representation of the session is calculated. Then based on this final representation Generate the final Top-K predictions for each session, which are the K items corresponding to the K highest predicted values.

[0092] The hyperbolic hypergraph neural network module generates a high-quality initial representation H for all items included in the entire session, and the specific steps include the following:

[0093] (21) Encode and calculate each item contained in each session S to obtain the basic representation E = {e1, e2, e3, ..., e} of all items in all sessions. i ,…,e n}, where e i Let n represent the basic representation of the i-th item among all items contained in all sessions, and n represent the total number of items contained in all sessions.

[0094] (22) Project the basic representation E of all items in all the conversations onto hyperbolic space according to the following formula to obtain the hyperbolic space representation of each item:

[0095]

[0096] In the above formula, Represent the hyperbolic space representation of the i-th item. This indicates that L2 regularization is performed, k = 1 / c, where c represents the curvature of the hyperboloid;

[0097] (24) Build a global item relationship hypergraph based on all sessions. In this global item relationship hypergraph, model each session as a hyperedge, and each hyperedge contains all items in that session; for any item in any session S It will establish undirected connections with all other items in S; define the indicator matrix Z∈R. N×M Let Z represent the adjacency matrix of a hypergraph, i.e., the hypergraph contains M hyperedges and N item nodes, and let Z be the element in the matrix Z. i∈ =1 indicates that the hyperedge in the hypergraph contains item v. i If the value is 0, then it does not include item v. i Define the degree matrix of the nodes. Among them W ∈∈ The weight of the hyperedge ∈ is represented by the degree matrix of the hyperedge.

[0098] (24) Perform L-layer hypergraph convolution calculation on the hyperbolic space representation of each item obtained in step (22), as follows:

[0099] For the aforementioned global item relationship hypergraph, the matrix calculation formula for hypergraph convolution is:

[0100] E hg(l+1) =D -1 ZWB -1 Z T E hg(l)

[0101] In the above formula, E hg(l+1) Let W represent the output of the hypergraph convolution computation at layer l, where W is a learnable parameter matrix representing the weights of all hyperedges, and B is the degree of each hyperedge, defined as... E hg(l) This represents the input for the hypergraph convolution calculation of the l-th layer; the input for the hypergraph convolution calculation of the 1st layer is the hyperbolic space representation of each item obtained in step (22);

[0102] (25) The initial representation H of all items in the entire session is calculated according to the following formula:

[0103]

[0104] In the above formula, L is a hyperparameter representing the number of layers in the hypergraph convolution. In this embodiment, L is set to 3.

[0105] The SLM (Session Learning Module) module calculates a new representation of all items in each session, containing the spatial structure properties of the data space. The specific content includes the following operational steps:

[0106] (31) Given any session S = {v1, v2, v3, ..., v...} m}, for any one of the items v i (1≤i≤m) all make it similar to the previous item v i-1and subsequent items v i+1 Create an edge to obtain a session-level graph;

[0107] (32) Based on the importance of different item node pairs in the graph obtained above, and based on the different order in which the items are clicked in the session, the edges established between the aforementioned items are divided into four types: namely, inbound edges, outbound edges, inbound-outbound edges, and self-loops.

[0108] (33) Based on the type of the edge, the weights between different node pairs are calculated using the attention mechanism according to the following formula:

[0109]

[0110] In the above formula, α i,j Represents item v i and items v j The weights between the node pairs formed, h i and h j These represent the item v output by the hyperbolic hypergraph neural network module. i and items v j The initialization representation, r i,j Represents item v i and items v j The type of the edges between them The type is r i,j The learnable weight parameters of the edges, ⊙ represents the element-wise dot product operation, and LeakyReLU() is the activation function; Let represent the set of all neighboring items of item i in session S, and exp represent an exponential function with the natural constant e as the base.

[0111] (34) Calculate the new representation of each item in session S containing the data space structure properties according to the following formula:

[0112]

[0113] In the above formula, h iSLM Let i represent the new representation of the i-th item in session S containing data space structure properties. Then, concatenate the new representations of all items in session S to obtain the new representations of all items in session S.

[0114] The Interest Extraction (IEM) module calculates a new representation for each session containing long-distance dependencies between items in the session and the relative position information of the items. The specific content includes the following operational steps:

[0115] (41) The position representation of the item is concatenated into the initial representation of the item output by the hyperbolic hypergraph neural network module according to the following formula:

[0116]

[0117] In the above formula, h i and p i Let these represent the initialization representation and position representation of the i-th item in session S, respectively; Let H represent the item representation of the i-th item in session S after concatenating the position representations; concatenate the above item representations of all items in session S to obtain the session representation H of session S after concatenating the position representations. s ;

[0118] (42) Using the self-attention mechanism, calculate the dependencies between items within session S according to the following formula:

[0119]

[0120] In the above formula, A represents the matrix of dependencies between items within session S, and d is the session representation H. s The dimensions of each item's representation vector are shown below. The subscript T indicates the matrix transpose, and Q, K, and V represent the query matrix, key matrix, and value matrix, respectively.

[0121] Q = LeakyReLU(W Q ×H s )

[0122] K = LeakyReLU(W K ×H s )

[0123] V = LeakyReLU(W V ×H s )

[0124] The W here Q W K W V These represent the learnable weight parameters for different types of matrices;

[0125] (43) The dependency matrix A obtained in step (42) is further nonlinearly calculated using an L-level feedforward neural network FFN, as shown in the following formula:

[0126] A l+1 =LeakyReLU(W l A l +b l )

[0127] In the above formula, Wl and b l Let A represent the learnable weight parameters and bias parameters of the l-th layer feedforward neural network FFN, respectively. l+1 A represents the output of the l-th layer of the feedforward neural network. l This represents the input of the l-th layer feedforward neural network; the input of the first layer feedforward neural network is the dependency matrix A obtained in step (42); in the embodiment, L is 3;

[0128] (44) The output of the last layer, i.e., the Lth layer of the feedforward neural network FFN, is the new representation of each session calculated by the interest extraction module (IEM), which contains the long-distance dependencies between items in the session and the relative position information of the items.

[0129]

[0130] The prediction module generates the final Top-K prediction for each session, including the following steps:

[0131] (51) Using a self-attention mechanism, calculate the session S according to the following formula. Characterization and Similarity matrix γ between representations:

[0132]

[0133] In the above formula, W γ ∈R d W IEM ∈R m×1 W SLM ∈R m×m There are three learnable weight matrices, b γ ∈R d It is a learnable bias parameter matrix, σ is the activation function; γ∈R m This is a similarity matrix, where each value represents the degree of correlation between the item at the corresponding position and the next predicted item;

[0134] (52) Based on the similarity matrix γ, the final representation of the session S is calculated according to the following formula.

[0135]

[0136] In the above formula, γ i It is the i-th value in the similarity matrix γ. Represents the characterization of the i-th item in session S;

[0137] (53) Calculate the predicted value of the i-th item according to the following formula:

[0138]

[0139] In the above formula, h i This represents the item v output by the hyperbolic hypergraph neural network module. i Initialization representation;

[0140] (54) Select the K highest predicted values The corresponding K items are used as the final Top-K prediction. In this example, K is set to 20.

[0141] The system is trained using the following loss function:

[0142] Loss = L pred +βL diff

[0143] In the above formula, β is a control L diff The hyperparameter of the weights; in this example, β is 0.2.

[0144]

[0145] Among them, y i This represents the actual label of the i-th item;

[0146]

[0147] In the above formula, KL(·) represents the KL divergence;

[0148]

[0149]

[0150]

[0151]

[0152] In the above formula, H I and H S This represents the new representation of all items in all sessions obtained from the Interest Extraction module (IEM) and the Session Learning module (SLM); that is, the new representation of all sessions. and The new representations of all the items in the text are pieced together separately;

[0153] In the above formula, and δ represents the values ​​calculated by the Interest Extraction Module (IEM) and the Conversation Learning Module (SLM) according to the following formula. adv :

[0154]

[0155]

[0156] In the above formula, ∈ is a variable used to control δ adv The hyperparameters need to be adjusted according to different data; adv () represents the loss function for the perturbed sample. In this embodiment, the cross-entropy loss function is used; x represents the real sample, and δ represents the perturbation applied to x.

[0157] calculate hour, This represents the new representation output by the Interest Extraction Module (IEM). As the final representation of the session The best prediction made by the prediction module for the session;

[0158] calculate hour, This represents the new representation output by the Session Learning Module (SLM). As the final representation of the session The best prediction made by the prediction module for the session;

[0159] See Figure 2 This invention introduces a conversation recommendation method based on hyperbolic hypergraph neural network and attention mechanism, which includes the following steps:

[0160] (1) The hyperbolic hypergraph neural network module uses the hyperbolic hypergraph neural network to generate a high-quality initial representation H for all items included in the entire session;

[0161] (2) The Session Learning Module (SLM) uses a graph neural network to capture the local spatial structure of the data within a session, based on the output H of the hyperbolic hypergraph neural network module, and calculates a new representation of all items containing the spatial structure properties of the data in each session.

[0162] (3) The Interest Extraction Module (IEM) calculates a new representation for each session based on the output H of the hyperbolic hypergraph neural network module and a sequence model based on a self-attention mechanism. This representation contains the long-distance dependencies between items in the session and the relative position information of the items.

[0163] (4) The prediction module uses the representation output by the conversation learning module SLM. and the representation output by the interest extraction module IEM. The final representation of the session is calculated. Then based on this final representation Generate the final Top-K predictions for each session, which are the K items corresponding to the K highest predicted values.

[0164] The inventors conducted extensive experiments on the system and method proposed in this invention, using the Tmall and Diginetica datasets, two commonly used datasets in the field of conversational recommendation. The Tmall dataset comes from the IJCAI-15 competition and includes purchase records of users on the Tmall platform. The Diginetica dataset comes from the CIKM 2016 competition, and the inventors used all the transaction data from it.

[0165] The evaluation metrics used in the experiment were Recall and MRR. Recall assesses the model's ability to cover all relevant instances, measuring its success in finding all relevant instances. Its value ranges from 0 to 1, with values ​​closer to 1 indicating more relevant instances found. MRR assesses the quality of the ranking results returned by the model, focusing on the position of relevant instances in the ranking. Its value also ranges from 0 to 1, with values ​​closer to 1 indicating the model better prioritizes relevant instances in the ranking.

[0166] The comparative schemes used in the experiment were SRGNN (for details on SRGNN, see: Shu, W., Yuyuan, T., Yanqiao, Z., Liang, W., Xing, X., & Tieniu, T. (2019) Session-Based Recommendation with Graph Neural Networks, AAAI Conference on Artificial Intelligence, 33.01:346-353.), STAMP (for details on STAMP, see: Liu, Qiao, et al. "STAMP: short-term attention / memory priority model for session-based recommendation." Proceedings of the 24th ACM SIGKDD), and GRU4Rec model (for details on GRU4Rec, see: Balázs, H., Alexandros, K., Linas, B., & Domonkos, T. (2015) Session-based Recommendations with Recurrent Neural Networks., International Conference on Learning Representations, abs / 1511.06939.). SR-GNN uses a graph neural network to encode items and conversations, followed by a self-attention layer. STAMP uses self-attention instead of recurrent neural network layers to enhance conversation-based recommendations. GRU4REC uses GRU layers to encode conversations and employs a ranking-based loss for model training.

[0167] The experimental comparison results are shown in Table 1. The Recall and MRR values ​​corresponding to the Top-20 items recommended in this invention are as follows:

[0168] Table 1

[0169]

[0170] The above experimental results prove that the system and method proposed in this invention are effective.

Claims

1. A conversation recommendation system based on hyperbolic hypergraph neural network and attention mechanism, characterized in that: The system includes the following modules: Hyperbolic Hypergraph Neural Network Module: This module's function is to generate high-quality initial representations H for all items included in the entire session using a hyperbolic hypergraph neural network. The specific tasks of this module include the following steps: (21) Each session Each item contained therein is encoded and calculated to obtain the basic representation of all items in all sessions. ,in Let n represent the basic representation of the i-th item among all items contained in all sessions, and n represent the total number of items contained in all sessions. (22) Represent the basic characteristics of all items in all the conversations according to the following formula. Projecting onto hyperbolic space yields a hyperbolic space representation of each object: In the above formula, Represent the hyperbolic space representation of the i-th item. This indicates that L2 regularization is being performed. , where c represents the curvature of the hyperboloid; (23) Build a global item relationship hypergraph based on all sessions. In this global item relationship hypergraph, model each session as a hyperedge, and each hyperedge contains all items in that session; for any session Any item in It will be with All other items in the array are undirected; define an indicator matrix. Let the adjacency matrix of a hypergraph represent the hypergraph containing M hyperedges and N item nodes, and let the index matrix be the adjacency matrix. elements in Represents the hyperedge in the hypergraph Contains items If the value is 0, then no items are included. Define the degree matrix of the nodes. ,in Indicates the superedge The weights are used to define the degree matrix of the hyperedges. ; (24) Perform L-layer hypergraph convolution calculation on the hyperbolic space representation of each item obtained in step (22), as follows: For the aforementioned global item relationship hypergraph, the matrix calculation formula for hypergraph convolution is: In the above formula, Let W represent the output of the hypergraph convolution computation at layer l, where W is a learnable parameter matrix representing the weights of all hyperedges, and B is the degree of each hyperedge, defined as follows: , This represents the input for the hypergraph convolution calculation of the l-th layer; the input for the hypergraph convolution calculation of the 1st layer is the hyperbolic space representation of each item obtained in step (22); (25) Calculate the initial representation H of all items in the entire session according to the following formula: In the above formula, L is a hyperparameter representing the number of layers in the hypergraph convolution; The Session Learning Module (SLM) uses a graph neural network to capture the local spatial structure of data within a session, based on the output H of the hyperbolic hypergraph neural network module, and calculates a new representation of all items containing the spatial structure properties of the data for each session. ; Interest Extraction Module (IEM): This module's function is to calculate, based on the output H of the hyperbolic hypergraph neural network module, a new representation for each session using a sequence model based on a self-attention mechanism. This representation contains long-distance dependencies between items in the session and information on the relative positions of the items. ; Prediction Module: This module's function is to predict the new representations output by the Session Learning Module (SLM). and the new representation output by the interest extraction module IEM. The final representation of the session is calculated. Then based on this final representation Generate the final Top-K predictions for each session, which are the K items corresponding to the K highest predicted values.

2. The conversation recommendation system based on hyperbolic hypergraph neural network and attention mechanism according to claim 1, characterized in that: The SLM (Session Learning Module) module calculates a new representation of all items in each session, containing the data space structure properties. The specific content includes the following operational steps: (31) Given any session For any one of these items All of them are the same as the previous items. and subsequent items Create an edge to obtain a session-level graph; (32) Based on the importance of different item node pairs in the graph obtained above, and based on the different order in which the items are clicked in the session, the edges established between the aforementioned items are divided into four types: namely, incoming edges, outgoing edges, in-outgoing edges and self-loops. (33) Based on the type of the edge, the weights between different node pairs are calculated using the attention mechanism according to the following formula: In the above formula, Represents items and items The weights between the node pairs formed and These represent the items output by the hyperbolic hypergraph neural network module. and items Initialization representation, Represents items and items The type of the edges between them The type is The learnable weight parameters of the edges, This represents the element-wise dot product operation. It is an activation function; Represents items in session S The collection of all neighboring items, Represented by natural constant An exponential function with base 0; (34) The new representation of each item in session S containing the data space structure properties is calculated according to the following formula: In the above formula, Let i represent the new representation of the i-th item in session S containing data space structure properties. Then, concatenate the new representations of all items in session S to obtain the new representations of all items in session S. .

3. The conversation recommendation system based on hyperbolic hypergraph neural network and attention mechanism according to claim 1, characterized in that: The Interest Extraction (IEM) module calculates a new representation for each session containing long-distance dependencies between items in the session and the relative position information of the items. The specific content includes the following operational steps: (41) The position representation of the item is concatenated into the initial representation of the item output by the hyperbolic hypergraph neural network module according to the following formula: In the above formula, and Let these represent the initialization representation and position representation of the i-th item in session S, respectively; Let represent the item representation of the i-th item in session S after concatenating the positional representations; concatenate the above item representations of all items in session S to obtain the session representation of session S after concatenating the positional representations. ; (42) Using the self-attention mechanism, calculate the dependencies between items within session S according to the following formula: In the above formula, A matrix representing the dependencies between items within session S. It is a conversation representation The dimension and index of each item's representation vector. Indicates matrix transpose. These represent the query matrix, key matrix, and value matrix, respectively, and are calculated as follows: Here These represent the learnable weight parameters for different types of matrices; (43) Using an L-level feedforward neural network Further nonlinear calculations are performed on the dependency matrix A obtained in step (42), as shown in the following formula: In the above formula, and They represent the first Layered feedforward neural network Learnable weight parameters and bias parameters, This represents the output of the l-th layer of the feedforward neural network. This represents the input of the l-th layer feedforward neural network; the input of the first layer feedforward neural network is the dependency matrix A obtained in step (42); (44) The last layer, i.e., the Lth layer, is the feedforward neural network The output is the new representation of each session calculated by the Interest Extraction Module (IEM), which contains long-distance dependencies between items in the session and the relative position information of the items. : 。 4. The conversation recommendation system based on hyperbolic hypergraph neural network and attention mechanism according to claim 1, characterized in that: The prediction module generates the final Top-K prediction for each session, including the following steps: (51) Using a self-attention mechanism, calculate the session S according to the following formula. Characterization and Similarity matrix between representations : In the above formula, , , There are three learnable weight matrices. It is a learnable bias parameter matrix. It is an activation function; This is a similarity matrix, where each value represents the degree of correlation between the item at the corresponding position and the next predicted item; (52) Based on similarity matrix The final representation of session S is calculated according to the following formula. : In the above formula, It is a similarity matrix The i-th value in Represents the characterization of the i-th item in session S; (53) Calculate the predicted value of the i-th item according to the following formula: In the above formula, This represents the items output by the hyperbolic hypergraph neural network module. Initialization representation; (54) Select the K highest predicted values The corresponding K items are used as the final Top-K prediction.

5. The conversation recommendation system based on hyperbolic hypergraph neural network and attention mechanism according to claim 1 or 4, characterized in that: The system is trained using the following loss function: In the above formula, It is a control The hyperparameters of the weights; in, This represents the actual label of the i-th item; In the above formula, express Divergence; In the above formula, and This represents the new representation of all items in all sessions obtained from the Interest Extraction module (IEM) and the Session Learning module (SLM); that is, the new representation of all sessions. and The new representations of all the items in the text are pieced together separately; In the above formula, and This represents the values ​​calculated by the Interest Extraction Module (IEM) and the Conversation Learning Module (SLM) according to the following formula. : In the above formula, It is a control The hyperparameters need to be adjusted according to different data. The loss function representing the perturbed samples; Represents a real sample. Indicates the application of Disturbances on the surface; calculate hour, This represents the new representation output by the Interest Extraction Module (IEM). As the final representation of the session The best prediction made by the prediction module for the session; calculate hour, This represents the new representation output by the Session Learning Module (SLM). As the final representation of the session The best prediction made for the session by the prediction module.

6. The session recommendation method based on the system of claim 1, characterized in that: The method includes the following steps: (1) Hyperbolic Hypergraph Neural Network Module All items included in the entire session generate a high-quality initialization representation H; (2) The Session Learning Module (SLM) uses a graph neural network to capture the local spatial structure of the data within a session, based on the output H of the hyperbolic hypergraph neural network module, and calculates a new representation of all items containing the spatial structure properties of the data in each session. ; (3) The Interest Extraction Module (IEM) calculates a new representation of each session based on the output H of the hyperbolic hypergraph neural network module and a sequence model based on a self-attention mechanism. This representation contains the long-distance dependencies between items in the session and the relative position information of the items. ; (4) The prediction module uses the representation output by the conversation learning module SLM. and the representation output by the interest extraction module IEM. The final representation of the session is calculated. Then based on this final representation Generate the final Top-K predictions for each session, which are the K items corresponding to the K highest predicted values.

Images