Dialogue recommendation method based on user multi-interest modeling and related device

Abstract

The application belongs to the field of artificial intelligence, and discloses a dialogue recommendation method based on user multi-interest modeling and related devices, wherein dialogue state updates and dialogue history state updates are obtained according to the last round of dialogue feedback of a user; a symbol dialogue dynamic graph is modeled according to the dialogue state updates, embedding operation is performed, graph representation learning is performed to obtain a user graph representation vector, then an interest state representation vector of the user is obtained by combining the dialogue state updates and a preset multi-interest constructor; a decision space is constructed according to the user graph representation vector and the dialogue history state updates; a dialogue state representation vector of the user is modeled according to the dialogue history state updates; a pre-trained hierarchical strategy network is called according to the decision space, the interest state representation vector and the dialogue state representation vector of the user, to obtain dialogue recommendation of the current round. The user interest state representation is modeled based on user multi-attribute preference, and the multi-agent reinforcement learning technology is used, so that the recommendation accuracy and quality are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and relates to a dialogue recommendation method and related apparatus based on user multi-interest modeling. Background Technology

[0002] With the development of information technology, internet content has exploded, leading to information overload. To help users sift through massive amounts of information and identify truly useful content, recommendation systems have emerged. These systems model user preferences based on their historical interactions with items, recommending items that users might be interested in. Today, they have become an indispensable component in e-commerce, lifestyle, and multimedia platform business scenarios.

[0003] However, traditional recommendation algorithms, such as collaborative filtering and matrix factorization, typically model users and items based on static historical interaction records to generate recommendations. However, user preferences change over time, and statically modeled recommendation algorithms cannot capture real-time user preferences, resulting in poor recommendation performance. Dialogue recommendation algorithms, on the other hand, acquire user preferences through multi-turn conversations and make recommendations based on this. Compared to traditional recommendation algorithms, dialogue recommendation algorithms can accurately model dynamic user preferences and update the model in real time based on feedback, further refining the recommendation results.

[0004] Question-driven dialogue recommendation algorithms employ a multi-turn dialogue model of "system asking questions - user answering," where the system can ask users for attribute information or recommend items. Existing methods typically assume that a user's preference for attribute values ​​within a single attribute type is unique, failing to consider that a user might like multiple attribute values ​​within the same attribute type. Furthermore, the various combinations of attribute values ​​across different attribute types constitute a user's multiple interests and preferences. Therefore, a user's preference for a particular attribute value may stem from different combinations of attribute values. If only a single user interest is modeled, the resulting attribute value combinations may contradict reality, thus affecting recommendation performance. Moreover, existing methods do not effectively utilize reinforcement learning capabilities, often employing a single agent to learn complex policies in a large decision space, failing to obtain high-quality dialogue recommendation strategies and thus impacting dialogue effectiveness. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a dialogue recommendation method and related apparatus based on user multi-interest modeling.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] In a first aspect, the present invention provides a dialogue recommendation method based on user multi-interest modeling, comprising:

[0008] Based on the user's feedback from the previous round of dialogue, we obtain dialogue status updates and dialogue history status updates;

[0009] The symbolic dialogue dynamic graph is modeled based on the dialogue state update, and then the symbolic dialogue dynamic graph is input into a preset embedding layer for embedding operation and graph representation learning to obtain the user graph representation vector; based on the user graph representation vector, the dialogue state update and the preset multi-interest builder, the user's interest state representation vector is obtained.

[0010] The decision space is constructed based on the user graph representation vector and the dialogue history state update.

[0011] Update the user's dialogue state representation vector based on the dialogue history state;

[0012] Based on the decision space and the user's interest state representation vector and dialogue state representation vector, a pre-trained hierarchical policy network is invoked to obtain the dialogue recommendation for the current round.

[0013] The weight parameters of the preset embedding layer are represented by a knowledge graph, which is obtained by constructing a knowledge graph based on the dataset and learning the knowledge graph representation. The multi-interest builder is used to obtain the user's updated multi-interest sequence set based on the user's historical multi-interest sequence set and the dialogue state update, and to obtain the user's interest state representation vector based on the similarity between the user's updated multi-interest sequence set and the user's graph representation vector. The hierarchical policy network is constructed based on a multi-agent reinforcement learning neural network.

[0014] Optionally, obtaining the dialogue status update based on the user's feedback from the previous round of dialogue includes:

[0015] Based on the user's feedback from the previous round of dialogue, determine the set of attributes the user has accepted. The user has rejected the attribute set. The user has rejected the item collection. Candidate Item Set and candidate attribute set Update:

[0016]

[0017] in, This represents the set of attributes that user u received in the previous round. This represents the set of attributes that user u rejected in the previous round. This represents the set of items that user u rejected in the previous round. Indicates having A collection of items with neutral attributes. express The set of attributes possessed by items in the game;

[0018] The updated and This is an update to the conversation status.

[0019] Optionally, the step of updating the modeled symbolic dialogue dynamic graph according to the dialogue state, and then inputting the symbolic dialogue dynamic graph into a preset embedding layer for embedding operations and performing graph representation learning to obtain the user graph representation vector includes:

[0020] Using symbolic dialogue dynamic graph G (t) =(N (t) A (t) Modeling dialogue state updates:

[0021]

[0022] Where, N (t) Represents a set of nodes;

[0023] Using adjacency matrix A (t) Representing the edges in a symbolic dialogue animation graph:

[0024]

[0025] Representing knowledge graph F kg Loaded into the preset embedding layer as weight parameters, the dialogue dynamic graph G (t) After the input embedding layer performs the embedding operation, the Symbolic Graph Convolutional Network (SGCN) is used for graph representation learning to obtain G. (t) Positive and negative representation vectors of each node:

[0026]

[0027] in, Represents node n i The positive representation vector, Represents node n i The negative representation vector, Represents node n i The set of positive neighbor nodes, Represents node n i The set of negative neighbor nodes, W P(l) and W N(l) These are the linear transformation matrices for the positive and negative edges, respectively, and σ represents the ReLU activation function. n is the positive neighbor node j The positive representation vector of the (l-1)th layer, n is the negative neighbor node k The negative representation vector of the (l-1)th layer, For node n i The positive representation vector of the (l-1)th layer; n is the positive neighbor node jThe negative representation vector of the (l-1)th layer, n is the negative neighbor node k The positive representation vector of the (l-1)th layer; For node n i The negative representation vector of the (l-1)th layer; subtract the positive and negative representation vectors of each node to obtain the user graph representation vector.

[0028]

[0029] Among them, z i User graph representation vector Middle node n i User graph representation vector;

[0030] The knowledge graph representation is obtained through the following method:

[0031] The dataset is used to extract user-interaction-item and item-containment-attribute type triples (h,r,t) to construct a knowledge graph. Then, the TransE algorithm is used to randomly replace one of the head entity h, tail entity t, and relation r of the correct triples in the knowledge graph with other entities or relations to obtain incorrect triples for negative sampling. The hinge loss function L is then minimized. kg Learning, obtaining the knowledge graph representation F kg =(h,r,t):

[0032]

[0033] Where Δ represents the set of correct triples, Δ′ represents the set of incorrect triples, γ represents the distance between positive and negative samples, and d represents the L2 norm.

[0034] Optionally, the multi-interest builder includes a first Transformer layer, an attention mechanism layer, a second Transformer layer, and a representation fusion layer; obtaining the user's interest state representation vector based on the user graph representation vector, dialogue state update, and the preset multi-interest builder includes:

[0035] Obtain the user's historical multi-interest sequence set M (t-1) M (t-1) Composed of multiple interest sequences Each interest sequence represents a user's interest and is a combination of multiple user-accepted attributes:

[0036]

[0037] in, User u has accepted the attribute set. A subset, K I Indicates the number of multiple interests;

[0038] M (t-1) The interest sequence is input into the first Transformer layer to obtain the corresponding sequence representation vector:

[0039]

[0040] in, Representing interest sequences The representation matrix is ​​obtained by concatenating the user graph representation vectors of each node. MultiHead represents the multi-head self-attention operation, LayerNorm represents the layer normalization operation, FFN represents the feedforward neural network, MeanPooling represents the average pooling operation, and X′... i This represents the intermediate output obtained after multi-head attention calculation and Add&Norm processing. This represents the output matrix obtained after processing by the Transformer layer. It is a sequence The sequence representation vectors; each sequence representation vector constitutes a historical multi-interest sequence representation set:

[0041]

[0042] Based on the new set of attributes the user accepted in the previous round of dialogue Based on the attention mechanism layer, calculate respectively Each attribute p in j The corresponding representation z j With L (t-1) Chinese characterization The weights are used as combination coefficients, and the sequence with the largest combination coefficient is selected to combine attribute p. j By adding these sequences, we obtain a multi-interest sequence set M. (t) :

[0043]

[0044] Where W1 and W2 are weight matrices, || denotes vector concatenation operation, and μ j,i Representation of sequence characterization For attribute p j The combination coefficients, For sequence The sequence representation vector, For sequence The sequence representation vector; M (t) Inputting into the second Transformer layer yields the multi-interest sequence representation set L. (t) Based on the representation fusion layer, calculate L (t) Characterization of interest sequences With user graph representation vector The similarity between them, for the multi-interest sequence representation set L (t) By performing weighted summation, we obtain the user's interest state representation vector.

[0045]

[0046] Where, α u,i express and Cosine similarity between them.

[0047] Optionally, the step of constructing the decision space based on the user graph representation vector and the dialogue history state update includes:

[0048] Based on user graph representation vector Item representation vector z v and attribute representation vector z p Calculate preference scores for items and attributes:

[0049]

[0050] in, and These represent the preference scores for candidate items and candidate attributes, respectively. For the set of attributes that the user has already accepted; The user has rejected the attribute set;

[0051] Next, calculate the information entropy weights for each item and attribute:

[0052]

[0053] in, P represents the information entropy weight of candidate item v. v Let v represent the set of attributes possessed by candidate item v, and freq(p) represent the frequency of attribute p when counting the attributes possessed by each candidate item. V represents the information entropy weight of candidate attribute p. p This represents a set of items that possess the attribute p. For the candidate item set, For candidate attribute set;

[0054] By combining preference scores and information entropy weights, candidate scores for candidate items are obtained. Candidate scores for candidate attributes

[0055]

[0056] Based on the candidate scores of the candidate items Candidate scores for candidate attributes Size, select top-N v Items and top-N p The attributes constitute the decision space D. (t) ;

[0057] The user's dialogue state representation vector, which is updated based on the dialogue history, includes:

[0058] Based on dialogue history his (t) and candidate set length len (t) The dialogue state representation vector is obtained.

[0059] his (t) ={res i |i=0,1,…,t-1}

[0060] h (t) =GRU(his) (t) )

[0061] l (t) =EMB(len) (t) )

[0062]

[0063] Among them, his (t) It is the sequence of dialogue actions in each round, res i =0 indicates that the dialogue action is a query attribute, res i =1 indicates that the dialogue action is to recommend an item, and the candidate set length is len. (t) Represents the candidate item set The size, GRU represents gated recurrent unit computation operation, EMB represents embedding operation, h (t) It is a historical sequence representation vector, l (t) It is the embedding vector of the candidate set length. This indicates a vector concatenation operation.

[0064] Optionally, the step of invoking a pre-trained hierarchical policy network to obtain the dialogue recommendation for the current round based on the decision space, the user's interest state representation vector, and the dialogue state representation vector includes:

[0065] User interest state representation vector User's dialogue state representation vector and decision space D (t) Input into a pre-trained hierarchical policy network;

[0066] The first-level agent in the pre-trained hierarchical policy network is based on In the dialogue target decision space O(t) ={g rec ,g ask Select the dialogue target in}; where g rec For recommendation, g ask To inquire; after determining the dialogue target, the secondary agent then... In decision space D (t) Item dialogue action decision space Or attribute dialogue action decision space Select a dialogue action;

[0067] Using Dueling DQN as a concrete implementation of the hierarchical policy network, the value function of the first-level agent in the hierarchical policy network... Value functions of second-level agents Defined as:

[0068]

[0069] in, Indicates the dialogue status. It is the dialogue state representation vector, g (t) ∈O (t) To recommend an item or inquire about its attributes, g (t) It is g (t) The corresponding representation vector, Representing state Next action g (t) Q value, Indicates the user's interest status. It is the user's interest state representation vector, a (t) ∈D (t) To indicate a specific item or attribute, a (t) It is its corresponding representation vector. Representing state Next action a (t) Q value, and These represent the state value function and the advantage function in the Dueling structure, respectively.

[0070] When making hierarchical action decisions, the first-level agent in O (t) The first-level agent calculates the Q-value of each action and selects the action with the largest Q-value as the output; the second-level agent then outputs the Q-value based on the output of the first-level agent in D. (t) Calculate the Q-value for each action. If the action with the highest Q-value is an item, then select the top-K action. v Generate an item list from the items; if the action with the highest Q value is an attribute, then select the top-K items under the corresponding attribute type. pGenerate an attribute list based on each attribute; use the item list or attribute list as a dialogue recommendation for the current round.

[0071] Optionally, the hierarchical policy network is pre-trained using a value factorization-based multi-agent reinforcement learning technique, VDN:

[0072] The following global value function is used for training:

[0073]

[0074] Among them, s (t) Represents the global state, u (t) Q represents the joint action of the first-level agent and the second-level agent in a hierarchical policy network. tot (s (t) ,u (t) () represents a global value function;

[0075] The value function of the first-level agent is corrected using a hierarchical decision-making debiasing method:

[0076]

[0077]

[0078] Where Q′1 represents the modified value function of the first-level agent. and These represent the item decision space and the attribute decision space, respectively. and The probability distribution output of Gumbel-Softmax is represented by ∈=-log(-log(i)), where i~Uniform(0,1) are independent parameters, and τ is the temperature parameter that controls the output of Softmax;

[0079] Synchronize Q′1 into the global value function formula to obtain the updated Q. tot (s (t) ,u (t) By minimizing the loss function L(θ), the optimal global value function is learned, resulting in the optimal policies of the first-level agent and the second-level agent.

[0080]

[0081] Where, r (t) The hierarchical policy network outputs a joint action u. (t) The environmental reward obtained later, γ is the discount factor, and B represents the stored experience. The experience replay pool, where θ represents the parameters of all network layers in the hierarchical policy network.

[0082] In a second aspect, the present invention provides a dialogue recommendation system based on user multi-interest modeling, comprising:

[0083] The dialogue update module is used to obtain dialogue status updates and dialogue history status updates based on the user's feedback from the previous round of dialogue.

[0084] The interest state module is used to update the modeled symbolic dialogue dynamic graph according to the dialogue state, and input the symbolic dialogue dynamic graph into the preset embedding layer for embedding operation and then perform graph representation learning to obtain the user graph representation vector; based on the user graph representation vector, the dialogue state update and the preset multi-interest builder, the user's interest state representation vector is obtained.

[0085] The decision space construction module is used to construct the decision space based on the user graph representation vector and the updated dialogue history state.

[0086] The dialogue state module is used to update the dialogue state representation vector of the modeling user based on the dialogue history.

[0087] The recommendation module is used to call a pre-trained hierarchical policy network to obtain dialogue recommendations for the current round based on the decision space, the user's interest state representation vector, and the dialogue state representation vector.

[0088] The weight parameters of the preset embedding layer are represented by a knowledge graph, which is obtained by constructing a knowledge graph based on the dataset and learning the knowledge graph representation. The multi-interest builder is used to obtain the user's updated multi-interest sequence set based on the user's historical multi-interest sequence set and the dialogue state update, and to obtain the user's interest state representation vector based on the similarity between the user's updated multi-interest sequence set and the user's graph representation vector. The hierarchical policy network is constructed based on a multi-agent reinforcement learning neural network.

[0089] In a third aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the above-described dialogue recommendation method based on user multi-interest modeling.

[0090] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described dialogue recommendation method based on user multi-interest modeling.

[0091] Compared with the prior art, the present invention has the following beneficial effects:

[0092] This invention presents a dialogue recommendation method based on user multi-interest modeling. It obtains dialogue state updates and dialogue history state updates based on the user's feedback from the previous round of dialogue. Then, based on the dialogue state updates, a preset embedding layer, and a preset multi-interest builder, it obtains the user's interest state representation vector. Next, it constructs a decision space based on the user graph representation vector and the dialogue history state update. Finally, it models the user's dialogue state representation vector based on the dialogue history state update. Finally, based on the decision space, the user's interest state representation vector, and the dialogue state representation vector, it calls a pre-trained hierarchical policy network to obtain the dialogue recommendation for the current round. This dialogue recommendation method uses user multi-attribute preference modeling to represent user interest states and employs a hierarchical policy network to decide dialogue actions based on the interest state representation and dialogue state representation. Compared to previous dialogue recommendation methods, it considers the role of user multi-attribute preferences in interest state representation. By designing a hierarchical policy network using multi-agent reinforcement learning technology for dialogue recommendation policy learning, it improves the accuracy and quality of dialogue recommendations, increases the accuracy of dialogue actions, and reduces the average number of dialogue rounds, demonstrating significant industrial application value in the search and recommendation business field. Attached Figure Description

[0093] Figure 1 This is a flowchart of a dialogue recommendation method based on user multi-interest modeling, according to an embodiment of the present invention.

[0094] Figure 2 This is a schematic diagram of the dialogue recommendation method based on user multi-interest modeling in an embodiment of the present invention.

[0095] Figure 3 This is a block diagram of the multi-interest builder structure according to an embodiment of the present invention.

[0096] Figure 4 This is a block diagram of the hierarchical strategy network structure according to an embodiment of the present invention.

[0097] Figure 5 This is a block diagram of the dialogue recommendation system based on user multi-interest modeling according to an embodiment of the present invention. Detailed Implementation

[0098] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0099] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0100] The present invention will now be described in further detail with reference to the accompanying drawings:

[0101] See Figure 1 In one embodiment of the present invention, a dialogue recommendation method based on user multi-interest modeling is provided, which improves the accuracy and quality of dialogue recommendation and reduces the average number of dialogue rounds.

[0102] Specifically, the dialogue recommendation method based on user multi-interest modeling includes the following steps:

[0103] S1: Based on the user's feedback from the previous round of dialogue, obtain dialogue status updates and dialogue history status updates.

[0104] S2: Update the modeling symbolic dialogue dynamic graph according to the dialogue state, and input the symbolic dialogue dynamic graph into the preset embedding layer for embedding operation and then perform graph representation learning to obtain the user graph representation vector; and obtain the user's interest state representation vector according to the user graph representation vector, the dialogue state update and the preset multi-interest builder.

[0105] S3: Construct the decision space based on the user graph representation vector and the dialogue history state update.

[0106] S4: Update the user's dialogue state representation vector based on the dialogue history.

[0107] S5: Based on the decision space and the user's interest state representation vector and dialogue state representation vector, call the pre-trained hierarchical policy network to obtain the dialogue recommendation for the current round.

[0108] The weight parameters of the preset embedding layer are represented by a knowledge graph, which is obtained by constructing a knowledge graph based on the dataset and learning the knowledge graph representation. The multi-interest builder is used to obtain the user's updated multi-interest sequence set based on the user's historical multi-interest sequence set and the dialogue state update, and to obtain the user's interest state representation vector based on the similarity between the user's updated multi-interest sequence set and the user's graph representation vector. The hierarchical policy network is constructed based on a multi-agent reinforcement learning neural network.

[0109] This invention presents a dialogue recommendation method based on user multi-interest modeling. It obtains dialogue state updates and dialogue history state updates based on the user's feedback from the previous round of dialogue. Then, based on the dialogue state updates, a preset embedding layer, and a preset multi-interest builder, it obtains the user's interest state representation vector. Next, it constructs a decision space based on the user graph representation vector and the dialogue history state update. Finally, it models the user's dialogue state representation vector based on the dialogue history state update. Finally, based on the decision space, the user's interest state representation vector, and the dialogue state representation vector, it calls a pre-trained hierarchical policy network to obtain the dialogue recommendation for the current round. This dialogue recommendation method uses user multi-attribute preference modeling to represent user interest states and employs a hierarchical policy network to decide dialogue actions based on the interest state representation and dialogue state representation. Compared to previous dialogue recommendation methods, it considers the role of user multi-attribute preferences in interest state representation. By designing a hierarchical policy network using multi-agent reinforcement learning technology for dialogue recommendation policy learning, it improves the accuracy and quality of dialogue recommendations, increases the accuracy of dialogue actions, and reduces the average number of dialogue rounds, demonstrating significant industrial application value in the search and recommendation business field.

[0110] See Figure 2 In one possible implementation, a knowledge graph is constructed by extracting triples (h, r, t) of the types "user-interaction-item" and "item-containment-attribute" from the dataset. Using the TransE algorithm, one of the head entity h, tail entity t, and relation r of the correct triples in the knowledge graph is randomly replaced with another entity or relation to perform negative sampling. This is achieved by minimizing the hinge loss function L. kg The embedding vector (h,r,t) of the knowledge graph triple (h,r,t) is learned, which is the knowledge graph representation F. kg And save it as a static file:

[0111]

[0112] Each sample in the dataset represents a user-item interaction record. Items in an interaction record are considered ground-truth items (i.e., target items) in a dialogue, and their attributes are called ground-truth attributes. A user simulator (US) is constructed to store this ground-truth information. US provides feedback based on the match between dialogue actions (questioned attributes / recommended items) and the ground-truth information, thus maintaining the dialogue interaction. In multi-interest scenarios, users have multiple target items. In this case, overlapping attribute sampling is used to select items with overlapping attributes from each user's interaction record set and combine them, so that the user simulator US stores multiple ground-truth items and their corresponding ground-truth attributes.

[0113] In one possible implementation, based on the user's feedback from the previous round of dialogue, whether the user accepted / rejected attributes or items, a set of attributes the user has accepted is generated. The user has rejected the attribute set. The user has rejected the item collection. Candidate Item Set and candidate attribute set Update:

[0114]

[0115] in, This represents the set of attributes that user u received in the previous round. This represents the set of attributes that user u rejected in the previous round. This represents the set of items that user u rejected in the previous round. Indicates having A collection of items with neutral attributes. express The set of attributes possessed by items in the game.

[0116] The updated and This is an update to the conversation status.

[0117] In one possible implementation, a symbolic dialogue dynamic graph G is used. (t) =(N (t) A (t) Model the current dialogue state, where N (t) Representing a set of nodes:

[0118]

[0119] Using adjacency matrix A (t) Representing the edges in a symbolic dialogue animation graph:

[0120]

[0121] The weight of a positive edge is 1, and the weight of a negative edge is -1.

[0122] In one possible implementation, the knowledge graph representation F obtained during the pre-training phase is... kg Loaded into the embedding layer as its weight parameters, the dialogue dynamic graph G (t) After embedding operations are performed on the input embedding layer, a symbolic graph convolutional network (SGCN) is used for graph representation learning to obtain G. (t) Positive and negative representation vectors of each node:

[0123]

[0124] in, Represents node n i The positive representation vector, Represents node n i The negative representation vector, Represents node n i The set of positive neighbor nodes, Represents node n i The set of negative neighbor nodes, W P(l) and W N(l) These are the linear transformation matrices for the positive and negative edges, respectively, and σ represents the ReLU activation function. n is the positive neighbor node j The positive representation vector of the (l-1)th layer, n is the negative neighbor node k The negative representation vector of the (l-1)th layer, For node n i The positive representation vector of the (l-1)th layer; n is the positive neighbor node j The negative representation vector of the (l-1)th layer, n is the negative neighbor node k The positive representation vector of the (l-1)th layer; For node n i The negative representation vector of the (l-1)th layer is obtained; then the positive and negative representation vectors of each node are subtracted to obtain the user graph representation vector.

[0125]

[0126] Among them, z i User graph representation vector Middle node n i User graph representation vector.

[0127] In one possible implementation, see Figure 3 The multi-interest builder includes a first Transformer layer, an attention mechanism layer, a second Transformer layer, and a representation fusion layer.

[0128] Pre-maintain the user's historical multi-interest sequence set M (t-1) It consists of multiple interest sequences Composed of multiple interest sequences, each representing a user's interest. This includes a collection of the user's historical multiple interest sequences. The sequence representations corresponding to each interest sequence are obtained by feeding them into the first Transformer layer of the multi-interest builder:

[0129]

[0130] Among them, K I Indicates the number of multiple interests. Representing interest sequences The representation matrix is ​​obtained by concatenating the user graph representation vectors of each node. MultiHead represents the multi-head self-attention operation, LayerNorm represents the layer normalization operation, FFN represents the feedforward neural network, MeanPooling represents the average pooling operation, and X′... i This represents the intermediate output obtained after multi-head attention calculation and Add&Norm processing. This represents the output matrix obtained after processing by the Transformer layer. It is a sequence The sequence representation vectors. These sequence representation vectors constitute the historical multi-interest sequence representation set:

[0131]

[0132] At the attention mechanism layer of the multi-interest builder, based on the new set of attributes newly received by the user in the previous round of dialogue... Based on the attention mechanism, calculate respectively Each attribute p in j The corresponding representation z j With L (t -1) Chinese characterization The weights are used as the combination coefficients μ, and the sequence with the largest combination coefficient μ is selected to assign attribute p. j Join and complete the multi-interest sequence set M (t) The update yields the multi-interest sequence set M. (t) :

[0133]

[0134] Where W1 and W2 are weight matrices, || denotes vector concatenation operation, and μ j,i Representation of sequence characterization For attribute p j The combination coefficients; For sequence The sequence representation vector, For sequence The sequence representation vector.

[0135] The updated M (t) Inputting the second Transformer layer yields the updated multi-interest sequence representation set L. (t) Then calculate the representation of each interest sequence. With user graph representation vector The similarity between them is used to characterize multi-interest sequences L (t) By performing weighted summation, we obtain the user's interest state representation vector.

[0136]

[0137] Where, α u,i User graph representation vector and interest sequence representation vector Cosine similarity between them.

[0138] In one possible implementation, based on user graph representation vectors Item representation z v and attribute representation z p Calculate preference scores for items and attributes:

[0139]

[0140] in, and These represent the preference scores for candidate items and candidate attributes, respectively.

[0141] Next, calculate the information entropy weights for each item and attribute:

[0142]

[0143]

[0144] in, P represents the information entropy weight of candidate item v. v Let v represent the set of attributes possessed by candidate item v, and freq(p) represent the frequency of attribute p when counting the attributes possessed by each candidate item. V represents the information entropy weight of candidate attribute p. pThis represents a set of items that possess the attribute p.

[0145] By combining the preference score and the information entropy weight, candidate scores for candidate items and candidate attributes are obtained:

[0146]

[0147] Then according to and Size, select top-N v and top-N p The decision space D consists of items and attributes. (t) This narrows down the range of candidate actions.

[0148] In one possible implementation, based on the dialogue history his (t) and candidate set length len (t) Perform dialogue state representation to obtain dialogue state representation vector.

[0149] his (t) ={res i |i=0,1,…,t-1} (27)

[0150] h (t) =GRU(his) (t) (28)

[0151] l (t) =EMB(len) (t) (29)

[0152]

[0153] Among them, his (t) It is the sequence of dialogue actions in each round, res i =0 indicates that the dialogue action is a query attribute, res i =1 indicates that the dialogue action is to recommend an item, and the candidate set length is len. (t) Indicates the current set of candidate items The size, GRU represents gated recurrent unit computation operation, EMB represents embedding operation, h (t) It is a historical sequence representation vector, l (t) It is the embedding vector of the candidate set length. This indicates a vector concatenation operation.

[0154] In one possible implementation, the interest state representation vector Dialogue state representation vector and decision space D (t)In the hierarchical policy network (HPN), the first-level agents in the HPN are based on... Dialogue Target Decision Space O (t) ={g rec ,g ask In the next step, a coarse-grained dialogue target (recommendation or inquiry) is selected. After the dialogue target is determined, the secondary agent then... In the dialogue action decision space or Select fine-grained dialogue actions (items or attributes).

[0155] See Figure 4 Using Dueling DQN as the specific implementation of the hierarchical policy network, the value function of the first-level agent... Value functions of second-level agents They are defined as follows:

[0156]

[0157] in, Indicates the dialogue status. It is its corresponding representation vector, g (t) ∈O (t) To recommend an item or inquire about its attributes, g (t) It is its corresponding representation vector. Representing state Next action g (t) The Q value. Indicates the user's interest status. It is its corresponding representation vector, a (t) ∈D (t) To indicate a specific item or attribute, a (t) It is its corresponding representation vector. Representing state Next action a (t) The Q value. and These represent the state value function and the advantage function in the Dueling structure, respectively.

[0158] When making hierarchical action decisions, the first-level agent in O (t) The Q-value of each action is calculated, and the action with the highest Q-value is selected as the output. The second-level agent then calculates the Q-value based on the output of the first-level agent in D. (t) Calculate the Q-value for each action. If the action with the highest Q-value is an item, then select the top-K action. v Generate an item list from the items; if the action with the highest Q value is an attribute, then select the top-K items under the corresponding attribute type. pEach attribute generates an attribute list, which serves as the final output of the policy network, i.e., the dialogue action for the current round.

[0159] In one possible implementation, the hierarchical policy network is pre-trained using a value decomposition-based multi-agent reinforcement learning technique (VDN), i.e., the entire model is trained.

[0160]

[0161] Among them, s (t) Represents the global state, u (t) Q represents the joint action of two agents in a hierarchical policy network. tot (s (t) ,u (t) ) represents a global value function.

[0162] To mitigate the negative impact of upper-level agents on lower-level agents during hierarchical decision-making, a hierarchical decision-making debiasing method is used to correct the value function of the upper-level agent:

[0163]

[0164] Where Q′1 represents the modified value function of the first-level agent. and These represent the item decision space and the attribute decision space, respectively. and Let denot represent the probability distribution output of Gumbel-Softmax, ∈=-log(-log(i)), i~Uniform(0,1) are independent parameters, and τ is the temperature parameter that controls the output of Softmax.

[0165] Synchronize Q′1 to the global value function to obtain the updated Q. tot (s (t) ,u (t) Then, by minimizing the loss function L(θ), the optimal global value function is learned, thereby obtaining the optimal policies for the first-level and second-level agents, i.e., the dialogue recommendation policy:

[0166]

[0167] Where, r (t) The hierarchical policy network outputs a joint action u. (t) The environmental reward obtained later, γ is the discount factor, and B represents the stored experience. The experience replay pool, where θ represents the parameters of all network layers in the hierarchical policy network.

[0168] In this implementation, the embedding vector size during pre-training is set to 64; the representation vector size during training is set to 100; the GCN convolutional layer size is set to 2; the number of training iterations is set to 10,000; the experience replay pool size is set to 50,000; the min-batch size is set to 128; the reward function is set as follows: successful recommendation reward is 1, failed recommendation reward is -0.1, successful query reward is 0.01, failed query reward is -0.1, exceeding the round limit reward is -0.3; the discount factor γ is set to 0.99; the learning rate of the Adam optimizer is set to 0.0001; the L2 regularization coefficient is set to 0.000001; and the item selection threshold N is set to... v and attribute filtering threshold N p All are set to 10; the length K of the candidate item list generated by the hierarchical strategy network is... v Set to 10, and the length of the attribute candidate list is K. p Set it to 3.

[0169] In one possible implementation, the original dataset is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5. The training set is used to extract triples to construct a knowledge graph and obtain a knowledge graph representation, while the validation set is used for pre-training the hierarchical policy network.

[0170] The system interacts with the user (US) on the test set to implement a dialogue recommendation process. In each round of dialogue, the trained hierarchical policy network is used, based on the interest state representation vector. Dialogue state representation vector and decision space D (t) It selects and outputs dialogue actions, and provides dialogue recommendations to users through multi-turn dialogue interactions.

[0171] In one possible implementation, the dialogue recommendation method based on user multi-interest modeling of the present invention and some conventional methods were validated on the LastFM and Yelp datasets to obtain Tables 1 and 2.

[0172] Table 1 shows the recall (Recall@10) and normalized depreciation cumulative gain (NDCG@10) metrics for different methods on the LastFM and Yelp datasets.

[0173] Table 1

[0174]

[0175] Table 2 shows the dialogue success rate (SR@15) and average number of dialogue turns (AT@15) for different methods on the LastFM and Yelp datasets.

[0176] Table 2

[0177]

[0178] As can be seen, the dialogue recommendation method based on user multi-interest modeling in this invention improves upon existing methods in terms of recall (Recall@10), normalized depreciation cumulative gain (NDCG@10), and dialogue success rate (SR@15), while reducing the average number of dialogue turns (AT@15).

[0179] The following are embodiments of the apparatus of the present invention, which can be used to execute embodiments of the method of the present invention. For details not disclosed in the apparatus embodiments, please refer to the embodiments of the method of the present invention.

[0180] See Figure 5 In another embodiment of the present invention, a dialogue recommendation system based on user multi-interest modeling is provided, which can be used to implement the above-mentioned dialogue recommendation method based on user multi-interest modeling. Specifically, the dialogue recommendation system based on user multi-interest modeling includes a dialogue update module, an interest state module, a decision space construction module, a dialogue state module, and a recommendation module.

[0181] The system comprises the following modules: a dialogue update module to obtain dialogue state updates and dialogue history state updates based on the user's feedback from the previous round of dialogue; an interest state module to model a symbolic dialogue dynamic graph based on the updated dialogue state, and input the symbolic dialogue dynamic graph into a preset embedding layer for embedding operations followed by graph representation learning to obtain the user's graph representation vector; and an interest state representation vector for the user based on the user's graph representation vector, the updated dialogue state, and a preset multi-interest builder. A decision space construction module to construct a decision space based on the user's graph representation vector and the updated dialogue history state. A dialogue state module to model the user's dialogue state representation vector based on the updated dialogue history state. A recommendation module to call a pre-trained hierarchical policy network based on the decision space, the user's interest state representation vector, and the dialogue state representation vector to obtain dialogue recommendations for the current round. The weight parameters of the preset embedding layer are represented using knowledge graphs, which are constructed from a dataset and learned through knowledge graph representation learning. The multi-interest builder obtains the user's updated multi-interest sequence set based on the user's historical multi-interest sequence set and the updated dialogue state, and obtains the user's interest state representation vector based on the similarity between the updated multi-interest sequence set and the user's graph representation vector. The hierarchical policy network is constructed based on a multi-agent reinforcement learning neural network.

[0182] All relevant content of each step involved in the aforementioned embodiments of the dialogue recommendation method based on user multi-interest modeling can be referenced to the functional description of the corresponding functional module of the dialogue recommendation system based on user multi-interest modeling in the embodiments of the present invention, and will not be repeated here.

[0183] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0184] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a dialogue recommendation method based on user multi-interest modeling.

[0185] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the dialogue recommendation method based on user multi-interest modeling in the above embodiments.

[0186] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A dialogue recommendation method based on user multi-interest modeling, characterized in that, include: Based on the user's feedback from the previous round of dialogue, we obtain dialogue status updates and dialogue history status updates; The symbolic dialogue dynamic graph is modeled based on the dialogue state update, and then the symbolic dialogue dynamic graph is input into a preset embedding layer for embedding operation and graph representation learning to obtain the user graph representation vector; based on the user graph representation vector, the dialogue state update and the preset multi-interest builder, the user's interest state representation vector is obtained. The decision space is constructed based on the user graph representation vector and the dialogue history state update. Update the user's dialogue state representation vector based on the dialogue history state; Based on the decision space and the user's interest state representation vector and dialogue state representation vector, a pre-trained hierarchical policy network is invoked to obtain the dialogue recommendation for the current round. Among them, the weight parameters of the preset embedding layer are represented by knowledge graphs, which are obtained by constructing a knowledge graph based on the dataset and learning the knowledge graph representation. The multi-interest builder is used to obtain the user's updated multi-interest sequence set based on the user's historical multi-interest sequence set and dialogue state update, and to obtain the user's interest state representation vector based on the similarity between the user's updated multi-interest sequence set and the user's graph representation vector; the hierarchical policy network is built based on a multi-agent reinforcement learning neural network. The multi-interest builder comprises a first Transformer layer, an attention mechanism layer, a second Transformer layer, and a representation fusion layer connected in sequence.

2. The dialogue recommendation method based on user multi-interest modeling according to claim 1, characterized in that, The step of updating the dialogue status based on the user's feedback from the previous round of dialogue includes: Based on the user's feedback from the previous round of dialogue, determine the set of attributes the user has accepted. The user has rejected the attribute set. The user has rejected the item collection. Candidate Items Set and candidate attribute set Update: in, Indicates user The set of attributes received in the previous round, Indicates user In the set of attributes rejected in the previous round, Indicates user In the set of items rejected in the previous round, Indicates having A collection of items with neutral attributes. express The set of attributes possessed by items in the game; The updated , , , and This is an update to the conversation status.

3. The dialogue recommendation method based on user multi-interest modeling according to claim 1, characterized in that, The step of updating the modeled symbolic dialogue dynamic graph according to the dialogue state, and then inputting the symbolic dialogue dynamic graph into a preset embedding layer for embedding operation, followed by graph representation learning to obtain the user graph representation vector includes: Using symbolic dialogue animation Modeling dialogue state updates: in, Represents a set of nodes; Use adjacency matrix Representing the edges in a symbolic dialogue animation graph: Representing knowledge graphs Loaded into a preset embedding layer as weight parameters, the dialogue animation graph is... After the input embedding layer performs the embedding operation, the Symbolic Graph Convolutional Network (SGCN) is used for graph representation learning to obtain... Positive and negative representation vectors of each node: in, Represents a node The positive representation vector, Represents a node The negative representation vector, Represents a node The set of positive neighbor nodes, Represents a node The set of negative neighbor nodes, and These are the linear transformation matrices for the positive and negative sides, respectively. Represents the ReLU activation function. Positive neighbor node The Layer positive representation vector, Negative neighbor node The Layer negative representation vector, For nodes The Layer positive representation vector; Positive neighbor node The Layer negative representation vector, Negative neighbor node The Layer positive representation vector; For nodes The Layer negative representation vector; subtract the positive and negative representation vectors of each node to obtain the user graph representation vector. : in, User graph representation vector Middle node User graph representation vector; The knowledge graph representation is obtained through the following method: Extract the triplet of user-interaction-item and item-containment-attribute type from the dataset. A knowledge graph is constructed, and then the TransE algorithm is used to identify the head entities of the correct triples in the knowledge graph. h Tail entity t and relationships r One of the three elements is randomly replaced with another entity or relation to obtain incorrect triples for negative sampling, and the hinge loss function is minimized. Learning, obtaining knowledge graph representation ( ): in, Represents the set of correct triples. Represents the set of incorrect triples. Indicates the distance between positive and negative samples. This represents the L2 norm.

4. The dialogue recommendation method based on user multi-interest modeling according to claim 1, characterized in that, The process of obtaining the user's interest state representation vector based on the user graph representation vector, dialogue state update, and a preset multi-interest builder includes: Obtain the user's historical multi-interest sequence set , Composed of multiple interest sequences Each interest sequence represents a user's interest and is a combination of multiple user-accepted attributes: in, User u Accepted attribute set A subset, Indicates the number of multiple interests; Will The interest sequence is input into the first Transformer layer to obtain the corresponding sequence representation vector: in, Representing interest sequences The representation matrix is ​​obtained by concatenating the user graph representation vectors of each node. This indicates a multi-head self-attention computation operation. Presentation layer normalization processing operation, This represents a feedforward neural network. This indicates the average pooling operation. This represents the intermediate output obtained after multi-head attention calculation and Add&Norm processing. This represents the output matrix obtained after processing by the Transformer layer. It is a sequence The sequence representation vectors; each sequence representation vector constitutes a historical multi-interest sequence representation set: Based on the new set of attributes the user accepted in the previous round of dialogue Based on the attention mechanism layer, calculate respectively Each attribute Corresponding representation and Chinese characterization The weights are used as combination coefficients, and the sequence with the largest combination coefficient is selected to combine the attributes. Add to obtain a multi-interest sequence set : in, and It is a weight matrix. This represents a vector concatenation operation. Representation of sequence characterization For attributes The combination coefficients, For sequence The sequence representation vector, For sequence The sequence representation vector; Inputting into the second Transformer layer yields a multi-interest sequence representation set. Based on the representation fusion layer, the calculation Characterization of each interest sequence With user graph representation vector The similarity between them, for multi-interest sequence representation sets By performing weighted summation, we obtain the user's interest state representation vector. : in, express and Cosine similarity between them.

5. The dialogue recommendation method based on user multi-interest modeling according to claim 1, characterized in that, The process of constructing the decision space based on the user graph representation vector and the dialogue history state update includes: Based on user graph representation vector Item representation vector and attribute representation Calculate preference scores for items and attributes: in, and These represent the preference scores for candidate items and candidate attributes, respectively. For the set of attributes that the user has already accepted; The user has rejected the attribute set; Next, calculate the information entropy weights for each item and attribute: in, Indicates candidate items Information entropy weight, Indicates candidate items The set of attributes it possesses Representing attributes The frequency of occurrence of the attributes possessed by each candidate item. Representing candidate attributes Information entropy weight, Indicates that it has attributes A collection of items; For the candidate item set, For candidate attribute set; By combining preference scores and information entropy weights, candidate scores for candidate items are obtained. Candidate scores for candidate attributes : Based on the candidate scores of the candidate items Candidate scores for candidate attributes Size, select and Each attribute constitutes the decision space. ; The user's dialogue state representation vector, which is updated based on the dialogue history, includes: Based on dialogue history and candidate set length The dialogue state representation vector is obtained. : in, It is the sequence of dialogue actions in each round. This indicates that the dialogue action is an inquiry about attributes. This indicates that the dialogue action is to recommend an item, and the candidate set length is... Represents the candidate item set Size, This indicates the calculation operation of the gated loop unit. This indicates an embedding operation. It is a historical sequence representation vector. It is the embedding vector of the candidate set length. This indicates a vector concatenation operation.

6. The dialogue recommendation method based on user multi-interest modeling according to claim 1, characterized in that, The step of calling a pre-trained hierarchical policy network to obtain the dialogue recommendation for the current round based on the decision space, the user's interest state representation vector, and the dialogue state representation vector includes: User interest state representation vector User's dialogue state representation vector and decision space Input into the pre-trained hierarchical policy network; The first-level agent in the pre-trained hierarchical policy network is based on In the decision space of dialogue objectives Select the dialogue target; where, As a recommendation, To inquire; after determining the dialogue target, the secondary agent then... In the decision-making space Item dialogue action decision space Or attribute dialogue action decision space Select a dialogue action; Using Dueling DQN as a concrete implementation of the hierarchical policy network, the value function of the first-level agent in the hierarchical policy network... Value functions of second-level agents Defined as: in, Indicates the dialogue status. It is a dialogue state representation vector. To recommend items or inquire about attributes, yes The corresponding representation vector, Representing state Next action of value, Indicates the user's interest status. It is a vector representing the user's interest state. Indicates a specific item or attribute. It is its corresponding representation vector. Representing state Next action of value, and These represent the state value function and the advantage function in the Dueling structure, respectively. When making hierarchical action decisions, the first-level agent in Calculate each action Value, selection The action with the highest value is taken as the output; the second-level agent is based on the output of the first-level agent. Calculate each action Value, if If the action with the highest value is an item, then select... Generate an item list from each item; if If the action with the highest value is an attribute, then select the attribute type corresponding to that attribute. Generate an attribute list based on each attribute; use the item list or attribute list as a dialogue recommendation for the current round.

7. The dialogue recommendation method based on user multi-interest modeling according to claim 6, characterized in that, The hierarchical policy network is pre-trained using a value factorization-based multi-agent reinforcement learning technique, VDN. The following global value function is used for training: in, Represents the global state. This represents the joint action of the first-level and second-level agents in a hierarchical policy network. Represents a global value function; The value function of the first-level agent is corrected using a hierarchical decision-making debiasing method: in, This represents the value function of the corrected first-level agent. and These represent the item decision space and the attribute decision space, respectively. and This represents the probability distribution output of Gumbel-Softmax. These are independent parameters. It controls the temperature parameter output by Softmax; Will It is synchronized to the global value function formula, resulting in the updated version. By minimizing the loss function The optimal global value function is obtained through learning, leading to the optimal policies of the first-level agent and the second-level agent. in, This indicates the joint action output by the hierarchical policy network. The environmental rewards received later As a discount factor, Indicates storage experience The experience replay pool This represents the parameters of all network layers in a hierarchical strategy network.

8. A dialogue recommendation system based on user multi-interest modeling, characterized in that, include: The dialogue update module is used to obtain dialogue status updates and dialogue history status updates based on the user's feedback from the previous round of dialogue. The interest state module is used to update the modeled symbolic dialogue dynamic graph according to the dialogue state, and input the symbolic dialogue dynamic graph into the preset embedding layer for embedding operation and then perform graph representation learning to obtain the user graph representation vector; based on the user graph representation vector, the dialogue state update and the preset multi-interest builder, the user's interest state representation vector is obtained. The decision space construction module is used to construct the decision space based on the user graph representation vector and the updated dialogue history state. The dialogue state module is used to update the dialogue state representation vector of the modeling user based on the dialogue history. The recommendation module is used to call a pre-trained hierarchical policy network to obtain dialogue recommendations for the current round based on the decision space, the user's interest state representation vector, and the dialogue state representation vector. Among them, the weight parameters of the preset embedding layer are represented by knowledge graphs, which are obtained by constructing a knowledge graph based on the dataset and learning the knowledge graph representation. The multi-interest builder is used to obtain the user's updated multi-interest sequence set based on the user's historical multi-interest sequence set and dialogue state update, and to obtain the user's interest state representation vector based on the similarity between the user's updated multi-interest sequence set and the user's graph representation vector; the hierarchical policy network is built based on a multi-agent reinforcement learning neural network. The multi-interest builder comprises a first Transformer layer, an attention mechanism layer, a second Transformer layer, and a representation fusion layer connected in sequence.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the dialogue recommendation method based on user multi-interest modeling as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the dialogue recommendation method based on user multi-interest modeling as described in any one of claims 1 to 7.

Images