An adaptive dialogue recommendation method and system for new products and new users

Abstract

The present invention discloses an adaptive dialogue recommendation method and system for new products and new users, the method comprising: constructing a user interaction graph based on users, products, product attributes, and dialogues; using an inductive graph algorithm to learn and obtain representations of all nodes in the current graph based on the user interaction graph; obtaining representations of states and actions based on the representations of the nodes; introducing state transfer and rewards to form a Markov decision process based on the state and action representations, and learning the action value function through a deep Q network algorithm; and querying attributes or recommending products in each round of dialogue recommendation based on the modeled action value function. The present invention dynamically obtains representations of newly added products and users through neighbor information, and simultaneously models dialogue strategies and recommendation strategies by introducing a graph model, so that they can be integrated with each other for better interaction.

Description

Technical Field

[0001] The present invention relates to the field of interactive dialogue recommendation, and in particular to an adaptive dialogue recommendation method and system for new products and new users. Background Art

[0002] Recommendation systems are tools that suggest items that target users may be interested in. Recommendation systems have emerged as people often listen to recommendations from others in their daily work and decision-making. With the development of e-commerce websites, the number and types of products have grown rapidly, and users need to spend a lot of time to find the products they want to buy. Therefore, there is an urgent need for recommendation systems that can provide recommendation results from all items based on user preferences.

[0003] However, traditional recommendation systems recommend products to users based on user interaction history and friend interaction history, lack direct user feedback, and cannot adapt well to the problem of user preference migration. At the same time, users may not know what they like before being told some feasible options. Therefore, interactive dialogue recommendation systems are favored. However, recommendations based entirely on prior knowledge such as user interaction history cannot solve problems such as user preference migration. In addition, dialogue recommendation systems also have inherent problems inherited from recommendation systems, such as new products and new users joining online, which makes the model unavailable. On the other hand, if prior knowledge such as user interaction history is completely abandoned, meaningless exploration will be greatly increased. Therefore, how to ensure that the recommendation system can perform adaptive dialogue recommendations when new products and new users join online is a problem worth exploring and urgently needs to be solved. Summary of the invention

[0004] Purpose of the invention: The present invention proposes an adaptive dialogue recommendation method and system for new products and new users, which at least partially solves the problems in the prior art.

[0005] Technical solution: In order to achieve the above purpose, the present invention adopts the following technical solution:

[0006] In a first aspect, an adaptive dialogue recommendation method for new products and new users comprises the following steps:

[0007] The user interaction graph is constructed by taking users, products, product attributes, and conversations as nodes, and taking the connection between products and corresponding product attributes, the connection between conversations and preferred products or preferred attributes involved in the conversations, the connection between users and their historical conversations, and the connection between users and their friends as edges between nodes;

[0008] Initialize the embedding vectors of the nodes on the user interaction graph. For each node, find all the source nodes whose target nodes are the node. According to the type of the edge between the source node and the target node, use the graph embedding algorithm to perform matrix transformation on the representation of the source node. Aggregate the transformed representations of all the source nodes, and use the inductive graph algorithm to learn the aggregation function.

[0009] The current state representation is formed by concatenating the representation corresponding to the user who currently initiates the conversation and the representation corresponding to the current conversation. The current action representation is formed by concatenating the probability that the current candidate node will be selected in the next round given the current conversation and the probability that the current candidate node will be selected in the next round given the node of the current interaction. The current candidate node includes the candidate product nodes that meet all the preferred attributes of the user in the current conversation and exclude the candidate product nodes that have been rejected in the current conversation, and the candidate attribute nodes that have all the attributes of the current candidate product set and exclude the attributes that the user prefers or has rejected in the current conversation.

[0010] Based on the current state representation and current action representation, the state transfer and reward are introduced to form a Markov decision process, and the action value function is learned through the deep Q network algorithm;

[0011] Based on the modeled action-value function, in each round of dialogue recommendation, the representation of action and state is obtained according to the current user interaction graph and input into the action-value function, and the input action with the highest output action value is selected for execution.

[0012] According to certain embodiments of the first aspect, in the user interaction graph, the user's historical conversations with the user and the user and his or her friend users serve as data describing the user's prior characteristics, and the conversations and the preferred products and preferred attributes involved therein serve as a posteriori information.

[0013] According to certain embodiments of the first aspect, the matrix transformation of the representation of the source node using the graph embedding algorithm includes: calling the found source node as the neighbor node of the target node, and using the idea of ​​the relational graph convolutional neural network, for all neighbor nodes, before information aggregation, setting different transformation matrices according to the different relationships between the edges with the target node: θ r X j ,r∈R,j∈N(i),X j To initialize the representation of all source nodes in the graph by random walk or adjacency matrix, θ r Represents the transformation matrix associated with the relationship edge, R is the set of relationship edges, N(i) represents the neighbor node of i, and i is the source node.

[0014] According to some implementations of the first aspect, the representation of the source node obtained after aggregating neighbor nodes is: f aggregate () is the aggregation function. The inductive graph algorithm learns the aggregation function based on known data, so that when a new node is added, the vectorized representation of the newly added node can be obtained by aggregating the representations of neighboring nodes.

[0015] According to certain embodiments of the first aspect, the probability that the current candidate node will be selected in the next round given the current dialogue is obtained by using a connection prediction algorithm between graph nodes; the probability that the current candidate node will be selected in the next round given the current interactive node is obtained by predicting the probability that a certain type of edge exists between the candidate node and the current round dialogue node.

[0016] According to certain implementations of the first aspect, the state transfer is to connect the current dialog node with the attribute or the preferred product of the current round of the user in the user interaction graph after the user performs an action;

[0017] The rewards include five types of rewards: rec_suc : When the user accepts the recommended product, there will be a positive reward of the first amount; r rec_fail : When a user rejects a recommended product, there will be a negative penalty of the first amount; r ask_suc : When the user accepts the attribute of the inquiry, there will be a second amount of positive reward; r ask_fail : When the user refuses to ask for the property, there will be a negative reward of the second amount; r quit : When the conversation exceeds the maximum number of rounds, there will be a negative reward of the first amount; the first amount is greater than the second amount;

[0018] Based on the Markov decision process, the competitive deep Q-network algorithm is used to separate the action-value function into the value function and the advantage function, and two networks are used for calculation to complete the learning of the action-value function.

[0019] According to certain embodiments of the first aspect, the value function is constructed by a fully connected neural network, which inputs the current state, makes a value judgment on the current state, and outputs the value V(S) of the current state; the advantage function is constructed by a fully connected neural network, which inputs the current state and action, makes a value evaluation on an action in the current state, and outputs the value A(S, a) of an action in the current state; the action value function is the sum of the value function and the advantage function: Q(S, a) = V(S) + A(S, a), and the action value function performs parameter learning based on existing data through a policy gradient algorithm.

[0020] In the second aspect, an adaptive dialogue recommendation system for new products and new users includes:

[0021] A graph construction module is used to construct a user interaction graph by taking users, commodities, commodity attributes, and conversations as nodes, and taking the connection between commodities and corresponding commodity attributes, the connection between conversations and preferred commodities or preferred attributes involved in the conversations, the connection between users and their historical conversations, and the connection between users and their friend users as edges between nodes;

[0022] The embedding module is used to initialize the embedding vectors of the nodes on the user interaction graph. For each node, all source nodes whose target nodes are the node are found. According to the type of the edge between the source node and the target node, the representation of the source node is transformed into a matrix using the graph embedding algorithm. The transformed representations of all source nodes are aggregated, and the aggregation function is learned using the inductive graph algorithm.

[0023] A feature representation module is used to construct a current state representation based on the representation corresponding to the user who currently initiates the conversation and the representation corresponding to the current conversation, and to construct a current action representation based on the probability that the current candidate node will be selected in the next round given the current conversation and the probability that the current candidate node will be selected in the next round given the node of the current interaction, wherein the current candidate node includes candidate product nodes that meet all the preferred attributes of the user in the current conversation and exclude the candidate product nodes that have been rejected in the current conversation, and candidate attribute nodes that have all the attributes of the current candidate product set and exclude the attributes that the user prefers or has rejected in the current conversation;

[0024] The strategy learning module is used to introduce state transfer and reward to form a Markov decision process based on the current state representation and the current action representation, and learn the action value function through the deep Q network algorithm;

[0025] The strategy execution module is used to obtain the representation of actions and states based on the modeled action value function in each round of dialogue recommendation according to the current user interaction graph, and input them into the action value function, and select the input action with the highest output action value for execution.

[0026] In a third aspect, a computer device comprises: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the adaptive dialogue recommendation method for new products and new users as described in the first aspect of the present invention are implemented.

[0027] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for adaptive dialogue recommendation of new products and new users as described in the first aspect of the present invention.

[0028] Beneficial effects:

[0029] 1. The present invention provides an adaptive conversation recommendation method and system for new products and new users. The method solves the online joining problem of new nodes through an inductive algorithm. When a new user joins, the initial representation of the new user is provided by dynamically aggregating the representations of its friend user nodes. When a new product joins, the initial representation of the new product node is provided by dynamically aggregating the representations of its corresponding attribute nodes, so that conversation recommendations can be performed adaptively.

[0030] 2. The present invention uses a unified model to model dialogue strategy and recommendation strategy, so that the dialogue strategy and recommendation strategy can be integrated with each other for more effective interaction.

[0031] 3. The present invention simultaneously models the user interaction history and the current user's conversation through the user interaction graph, so that the prior information can directly affect the state and action representation of the current user conversation through the graph aggregation algorithm, thereby providing a meaningful initial representation and reducing ineffective exploration in the early stage of strategy learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of a method for adaptively recommending new products to new users provided by an embodiment of the present invention;

[0033] Figure 2 A user interaction diagram provided for an embodiment of the present invention;

[0034] Figure 3 The inductive graph algorithm provided in the embodiment of the present invention learns and obtains a schematic diagram representing all nodes in the current graph;

[0035] Figure 4 A schematic diagram of the modeling process of actions and states provided in an embodiment of the present invention;

[0036] Figure 5 A schematic diagram of the process of modeling action-value function and performing strategy learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0038] In order to solve the problem of user preference deviation, all current dialogue recommendation strategies are completely implemented based on the user's current feedback, which discards all prior information and often results in meaningless exploration during the strategy learning process. At the same time, the addition of new products will cause the dialogue recommendation strategy to be re-learned, which is very inefficient and time-consuming. The present invention integrates prior information into strategy learning through user interaction graphs, improves the efficiency of strategy learning exploration, reduces the number of meaningless explorations, and thus learns more valuable strategy models. At the same time, by using the inductive graph algorithm, the neighbor nodes of new products such as product attribute nodes can be used to summarize the vector representation of the new product, eliminating the need to re-learn the strategy.

[0039] Figure 1 The flowchart of the method for adaptively recommending new products to new users provided by the embodiment of the present invention is shown in Figure 1. The method can be executed by a server, and the generated model can be applied to a variety of terminal application scenarios. The server can execute the dialogue recommendation strategy model to conduct a dialogue with the user with a certain strategy, and send the system questions or responses to the terminal so that the terminal presents the system questions or responses to the user. Figure 2 This method is introduced in detail. The specific steps of this method are as follows:

[0040] S101. Construct a user interaction diagram.

[0041] Users, products, product attributes, and conversations are taken as nodes. Product attributes are also referred to as attributes below. Figure 2 As shown, user nodes include "Liqing" and "Dake", and dialogue nodes include dialogue 1, dialogue 2, and dialogue 3. The user's preferred products or preferred attributes will be reflected in the dialogue, such as Figure 2 As shown, the commodity nodes include treadmills and mini refrigerators. The commodity attributes follow the preferences reflected in the commodities or conversations. The attributes of the treadmill involved in the embodiment include sports, home use, and running. The attributes of the mini refrigerator include dormitory use, small size, and refrigerated food. In addition, the attributes involved in the conversation include grinding soy milk and cutting meat. By combining commodities with their corresponding attributes, conversations with the preferred commodities and the preferred attributes involved, users with their historical conversations, and users with their friend users as different types of edges, user nodes, commodity attribute nodes, commodity nodes, and conversation nodes are connected to each other to construct a graph such as Figure 2 The user interaction graph shown in FIG. 2 is a graph in which the product node and the attribute node are connected by the attributes of the product as the relationship edge, forming a product attribute graph as shown in FIG. 21. The dialog node and the attribute node or the product node are connected by the attributes or products preferred by the user in the dialog as the relationship edge, and the product attribute node and the attribute node are connected by the attributes or products preferred by the user in the dialog as the relationship edge. Figure 1The user dialogue history graph shown in FIG. 22 is formed. The user node and the dialogue node are connected by the user and the related dialogue as the relationship edge, and the user and the user are connected by the friend relationship edge. Figure 1 Together, a user interaction diagram as shown in FIG. 23 is formed.

[0042] S102, using the inductive graph algorithm to learn and obtain representations of all nodes in the current graph.

[0043] First, the representation of all nodes in the graph is initialized through random walk or adjacency matrix. Then, for each node in the graph, all edges with the node as the target node are found, so as to find all the source nodes corresponding to the node. These found source nodes are called the neighbor nodes of the target node. Then, with the idea of ​​relational graph convolutional neural network, for all neighbor nodes, before information aggregation, different transformation matrices are set according to the different relationships between the edges and the target node: θ r X j ,r∈R,j∈N(i), R is the set of relationship edges, N(i) represents the neighbor node of i, and i is the source node. j To initialize the representation of all source nodes in the graph by random walk or adjacency matrix, θ r Represents the transformation matrix associated with the relationship edge. Different relationship edges, transformation matrix θ r Different. The relationship graph convolutional neural network means that according to the different relationship edges between the source node and the target node, different transformation matrices will be generated, and the source node representation will be multiplied to obtain the representation of the source node mapped to the target node.

[0044] After aggregating neighbor nodes, the source node is represented as: 1,…,…,n represents n different relationship edges, where j 1 ,…j m Indicates that the edge type between the source node and the target node in the relationship graph is all the source nodes of the first relationship edge. Figure 3 As shown in the figure, the "Dialogue" node is the target node, and the "Refrigerated Food", "Li Qing", "Dormitory", and "Mini Refrigerator" nodes are the corresponding source nodes. The edges between the "Refrigerated Food", "Dormitory", and "Mini Refrigerator" nodes and the "Dialogue" node belong to the same relationship edge, while the edge between the "Li Qing" node and the "Dialogue" node belongs to another relationship edge. Then Refrigerate food, dorm use, mini refrigerator∈r 1 , Liqing∈r 2 .

[0045] The goal of the inductive graph algorithm is to perform an aggregation function f through known data. aggregate() is learned so that when a new node is added, the vectorized representation of the newly added node is obtained by aggregating the representations of the neighboring nodes. That is, the new node is represented by the aggregation of all the neighboring nodes before the new node is added. Therefore, the goal is to learn what kind of function to use to aggregate the information of these neighboring nodes to obtain the most effective representation of the new node. New nodes in the reasoning stage mainly refer to new products and new users, but at the same time, as the conversation proceeds, the vectorized embeddings of the conversation nodes and users are also updated based on the inductive graph algorithm. In the training stage, various types of nodes need to be randomly sampled for training. In the inductive graph algorithm, first set the aggregation depth to K. For a node, sample their neighbor nodes (first-order neighbors). For first-order neighbors, sample their neighbors (second-order neighbors) until the K-order neighbor sampling is completed. By acting on the K-order neighbors with the aggregation function, the representation of the K-1-order neighbors is obtained, until the aggregated representation of itself is finally obtained. Assume that K=2, with Figure 2 For example, when Li Qing's dialogue 2 talks about the preferred product being a mini refrigerator, in order to obtain the representation of dialogue 2, first find the second-order neighbors of dialogue 2, which are dialogue 1 (dialogue 3 will appear after dialogue 2 is completed, so there is no dialogue 3 at present), Dake, dormitory use, small and refrigerated food. Use an aggregation function to aggregate the information of dialogue 1 and Dake (the aggregation function can be a mean aggregator, lstmaggregator or pooling aggregator or other aggregation functions that need to be learned) to obtain Li Qing's neighbor information. Connect Li Qing's own representation and neighbor information, and obtain Li Qing's updated representation through a fully connected neural network. Similarly, update all first-order neighbor node information of dialogue 2. Finally, based on the updated first-order neighbor node information, the updated representation of dialogue 2 is calculated in the same way. The loss source of the learning of the inductive graph algorithm can be either the downstream connection prediction task or the final recommendation task. S103, obtain the representation of the state and action.

[0046] Since the subsequent strategy model is modeled as a Markov decision process, the Markov decision process can be defined in four most important parts: state, action, transfer and reward; among them, the most important thing to clarify is the modeling of state and action, such as Figure 4 As shown, by using the representation of the user of the current conversation and its node, and the representation of the current conversation and its node, a state representation is formed, such as the representation of the current round state shown by reference numeral 43: State =

[0047] concate(E user ,E dialog ).

[0048] The action describes the actions that the system can take in the current round, including: continue to ask the user about the attributes they prefer, and recommend products to the user based on the user preference information obtained from the current conversation. Therefore, the action space contains both product nodes and attribute nodes. Here, all selectable attribute nodes are called candidate attribute nodes, and all selectable product nodes are called candidate product nodes. In the tth round, the candidate product nodes include all products that meet the user's current preference attributes, but do not include products that the user has rejected: in represents the set of user preference attributes until round t, represents all the products in round t that satisfy the user’s current preference attributes, Indicates the set of products rejected by the user until round t. In round t, the candidate attribute nodes include all the attributes contained in all the candidate product nodes so far, but do not include the attributes that the user has rejected in rounds 1 to t-1, nor the attributes that the user has accepted in rounds 1 to t-1: in Represents the set of all attributes contained in the candidate product node. Indicates the attributes that the user has accepted in rounds 1 to t-1. Indicates the attributes that the user has rejected in rounds 1 to t-1. For the connection prediction task, it can be simply considered as a binary classification task, and the classification goal is to determine the probability of an edge between two nodes in the current graph. Specifically, it refers to the probability that there is an edge between the current dialogue node and a certain attribute or a certain product node, that is, to predict the attribute or product that the user currently prefers. The simplest way to do this task is to concatenate the representations of the two nodes, pass them through a fully connected neural network, and finally output the probability of an edge between the two nodes (a number between 0 and 1).

[0049] Each candidate node representation and the probability that the candidate node will be selected in the next round given the current dialogue are concatenated to form the representation of the current round action as shown in the reference numeral 42: Action = concate(E cand ,p(cand)),E cand Represents the embedded vector representation of candidates, which is the candidate node representation obtained after aggregation by the above graph algorithm, where V cand and P cand Constitutes the set of all candidates.

[0050]

[0051] cand=P cand ∪V cand

[0052] S104. Modeling action-value function.

[0053] On the basis of the representation of the state and action in the previous step, the definition of state transition and reward is supplemented to form a complete Markov decision process. Among them, the state transition is to connect the current dialogue node with the attribute or preferred product of the current user in the user interaction graph after the user takes an action. Update the state and action representation through the inductive graph algorithm. Since the state is composed of the representation of the current user node and the current dialogue node, during the dialogue process, since the user will indicate his preferred attributes or products, the dialogue node will connect more product or attribute nodes, and these product and attribute nodes are also the second-order neighbors of the user node. Therefore, the representation of the current user node and the current dialogue node output by the inductive graph algorithm will change. For example:

[0054] The rewards include five types of rewards:

[0055] r rec_suc : When the user accepts the recommended product, he will receive a positive reward of the first amount;

[0056] r rec_fail : When a user rejects a recommended product, there will be a negative penalty of the first amount;

[0057] r ask_suc : When the user accepts the attributes of the inquiry, there will be a second amount of positive rewards;

[0058] r ask_fail : When the user refuses the attribute being asked, there will be a negative reward of the second amount;

[0059] r quit : When the conversation exceeds the maximum number of rounds, there will be a negative reward of the first amount;

[0060] Among them, the first amount is greater than the second amount.

[0061] Based on this Markov decision process, the Dueling DQN algorithm is introduced as a strategy learning module that unifies the dialogue strategy and recommendation strategy. Figure 5 As shown. This value-based algorithm explicitly models the expected reward for each action performed in each state. Considering the continuity and infinity of state representation, the continuity of action representation and the finiteness of action, network oscillation, overvaluation and other factors, the present invention uses the Dueling DQN algorithm as a policy learning module to model the action value function.

[0062] Reference Figure 5, the dueling deep q network (Dueling DQN) separates the action value function into a value function and an advantage function, and uses two networks for calculation. Among them, the value function 52 is constructed by a fully connected neural network, which inputs the current state, makes a value judgment on the current state, and outputs the value V(S) of the current state. V(S) is a neural network, which inputs the current state and outputs a prediction of the value of the current state. The measurement standard of this value is as mentioned above about rewards. First, when the dialogue recommendation task is completed, the reward of the dialogue can be accurately calculated according to the five types of rewards defined above, and the reward is the value of the final entire dialogue. Derived forward, in the round before the end of the dialogue, the value of the state is: the value of the final entire dialogue minus the value of the action taken to lead to the final dialogue (multiple actions can be taken in the round before the end of the dialogue, only one or several actions can lead to the final dialogue, and choosing other actions will become other dialogues), and so on, the value of the initial dialogue sentence is obtained. Through neural network training, it is possible to predict the value of the current state of a new dialogue. The first layer of fully connected network and hidden layer is to further extract the characteristics of state and action, and the second layer of fully connected network is to calculate the value of action (advantage function) and the value of state (value function) through deep features.

[0063] Advantage function 51 is constructed by a fully connected neural network. It evaluates the value of an action in the current state by inputting the current state and action, and outputs the value of an action in the current state A(S, a). A(S, a) refers to the value of taking action a in state S. Figure 2 For example, in dialogue 3, it is known that the user's preferences are grinding soy milk and cutting meat. At this time, if the action selected by the system is to ask the user whether the preferred product is a wall-breaking machine, and the user's preference is indeed a wall-breaking machine, this action will generate a large positive value. Specifically, the value generated by this action is the final value of the dialogue minus the value of the current state.

[0064] The action value function 53 is formed by summing the value function 52 and the advantage function 51: Q(S, a) = V(S) +

[0065] A(S, a). The action value function 53 performs parameter learning based on the existing data through the policy gradient algorithm.

[0066] S105: Generate a knowledge-integrated dialogue for querying attributes or recommending products based on the strategy.

[0067] Based on the modeled action-value function, in each round of dialogue recommendation, the representation of action and state is obtained according to the current user interaction graph and input into the action-value function, and the input action with the highest output action value is selected for execution.

[0068] The present invention uses an inductive graph algorithm to dynamically obtain the representation of newly added goods and users through neighbor information, and obtains a better self-representation by aggregating neighbor node vectors to benefit subsequent tasks such as link prediction and reinforcement learning. By introducing a graph model to simultaneously model dialogue strategies and recommendation strategies, the dialogue strategy module and the recommendation strategy module can be integrated with each other for better interaction. During the dialogue process, the representation of the dialogue user and the representation of the current dialogue node are spliced ​​as the state, the candidate node representation and the connection prediction probability between the candidate node and the current user node, and the candidate node and the previous round of user preference node are spliced ​​as the action representation, and the reinforcement learning algorithm is used to model the action value function. Finally, by maximizing the entire dialogue value, the next node is selected as a product or an attribute, and the recommendation or dialogue inquiry operation is performed according to the different types of the next node selected.

[0069] The present invention also provides an adaptive dialogue recommendation system for new products and new users, comprising:

[0070] A graph construction module is used to construct a user interaction graph by taking users, commodities, commodity attributes, and conversations as nodes, and taking the connection between commodities and corresponding commodity attributes, the connection between conversations and preferred commodities or preferred attributes involved in the conversations, the connection between users and their historical conversations, and the connection between users and their friend users as edges between nodes;

[0071] The embedding module is used to initialize the embedding vectors of the nodes on the user interaction graph. For each node, all source nodes whose target nodes are the node are found. According to the type of the edge between the source node and the target node, the representation of the source node is transformed into a matrix using the graph embedding algorithm. The transformed representations of all source nodes are aggregated, and the aggregation function is learned using the inductive graph algorithm.

[0072] A feature representation module is used to construct a current state representation based on the representation corresponding to the user who currently initiates the conversation and the representation corresponding to the current conversation, and to construct a current action representation based on the probability that the current candidate node will be selected in the next round given the current conversation and the probability that the current candidate node will be selected in the next round given the node of the current interaction, wherein the current candidate node includes candidate product nodes that meet all the preferred attributes of the user in the current conversation and exclude the candidate product nodes that have been rejected in the current conversation, and candidate attribute nodes that have all the attributes of the current candidate product set and exclude the attributes that the user prefers or has rejected in the current conversation;

[0073] The strategy learning module is used to introduce state transfer and reward to form a Markov decision process based on the current state representation and the current action representation, and learn the action value function through the deep Q network algorithm;

[0074] The strategy execution module is used to obtain the representation of actions and states based on the modeled action value function in each round of dialogue recommendation according to the current user interaction graph, and input them into the action value function, and select the input action with the highest output action value for execution.

[0075] It should be understood that the adaptive dialogue recommendation system for new products and new users in the embodiment of the present invention can implement all the technical solutions in the above-mentioned method embodiment, and the functions of its various functional modules can be specifically implemented according to the methods in the above-mentioned method embodiments. The specific implementation process can refer to the relevant description in the above-mentioned embodiments, which will not be repeated here.

[0076] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the adaptive dialogue recommendation method for new products and new users as described above are implemented.

[0077] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the adaptive dialogue recommendation method for new products and new users as described above are implemented.

[0078] It will be appreciated by those skilled in the art that all or part of the steps to implement the above-described embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware to accomplish the steps, and the program may be stored in a computer-readable storage medium. In the context of the present invention, the computer-readable medium may be considered to be tangible and non-temporary. Non-limiting examples of non-temporary tangible computer-readable media include non-volatile memory circuits (e.g., flash memory circuits, erasable programmable read-only memory circuits, or mask read-only memory circuits), volatile memory circuits (e.g., static random access memory circuits or dynamic random access memory circuits), magnetic storage media (e.g., analog or digital tapes or hard disk drives), and optical storage media (e.g., CDs, DVDs, or Blu-ray discs), etc.

[0079] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0080] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present invention. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0081] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. An adaptive dialogue recommendation method for new products and new users, characterized in that: The following steps are involved: The user interaction graph is constructed by taking users, products, product attributes, and conversations as nodes, and taking the connection between products and corresponding product attributes, the connection between conversations and preferred products or preferred attributes involved in the conversations, the connection between users and their historical conversations, and the connection between users and their friends as edges between nodes; Initialize the embedding vectors of the nodes on the user interaction graph. For each node, find all the source nodes whose target nodes are the node. According to the type of the edge between the source node and the target node, use the graph embedding algorithm to perform matrix transformation on the representation of the source node. Aggregate the transformed representations of all the source nodes, and use the inductive graph algorithm to learn the aggregation function. The current state representation is formed by concatenating the representation corresponding to the user who currently initiates the conversation and the representation corresponding to the current conversation. The current action representation is formed by concatenating the probability that the current candidate node will be selected in the next round given the current conversation and the probability that the current candidate node will be selected in the next round given the node of the current interaction. The current candidate node includes the candidate product nodes that meet all the preferred attributes of the user in the current conversation and exclude the candidate product nodes that have been rejected in the current conversation, and the candidate attribute nodes that have all the attributes of the current candidate product set and exclude the attributes that the user prefers or has rejected in the current conversation. Based on the current state representation and current action representation, the state transfer and reward are introduced to form a Markov decision process, and the action value function is learned through the deep Q network algorithm; Based on the modeled action-value function, in each round of dialogue recommendation, the representation of action and state is obtained according to the current user interaction graph and input into the action-value function, and the input action with the highest output action value is selected for execution.

2. The method according to claim 1, characterized in that In the user interaction graph, the user's historical conversations with his or her friends are used as data to describe the user's prior characteristics, and the conversations and the preferred products and preferred attributes involved are used as a posteriori information.

3. The method according to claim 1, characterized in that The matrix transformation of the source node representation using the graph embedding algorithm includes: calling the found source node as the neighbor node of the target node, and using the idea of ​​the relational graph convolutional neural network, for all neighbor nodes, before information aggregation, different transformation matrices are set according to the different relationships between the edges with the target node: θ r X j ,r∈R,j∈N(i),X j To initialize the representation of all source nodes in the graph by random walk or adjacency matrix, θ r Represents the transformation matrix associated with the relationship edge, R is the set of relationship edges, N(i) represents the neighbor node of i, and i is the source node.

4. The method according to claim 3, characterized in that After aggregating neighbor nodes, the source node is represented as: f aggregate () is the aggregation function. The inductive graph algorithm learns the aggregation function based on known data, so that when a new node is added, the vectorized representation of the newly added node can be obtained by aggregating the representations of neighboring nodes.

5. The method according to claim 1, characterized in that By using the connection prediction algorithm between graph nodes, the probability that the current candidate node will be selected in the next round given the current dialogue is obtained; by predicting the probability that a certain type of edge exists between the candidate node and the current round of dialogue node, the probability that the current candidate node will be selected in the next round given the current interaction node is obtained.

6. The method according to claim 5, characterized in that State transfer is the process of connecting the current dialogue node with the attributes or products preferred by the current user in the user interaction graph after the user has taken an action. The rewards include five types of rewards: rec_suc : When the user accepts the recommended product, there will be a positive reward of the first amount; r rec_fail : When a user rejects a recommended product, there will be a negative penalty of the first amount; r ask_suc : When the user accepts the attribute of the inquiry, there will be a second amount of positive reward; r ask_fail : When the user refuses to ask for the property, there will be a negative reward of the second amount; r quit : When the conversation exceeds the maximum number of rounds, there will be a negative reward of the first amount; the first amount is greater than the second amount; Based on the Markov decision process, the competitive deep Q-network algorithm is used to separate the action-value function into the value function and the advantage function, and two networks are used for calculation to complete the learning of the action-value function.

7. The method according to claim 6, characterized in that The value function is constructed by a fully connected neural network. By inputting the current state, it makes a value judgment on the current state and outputs the value V(S) of the current state. The advantage function is constructed by a fully connected neural network. By inputting the current state and action, it makes a value evaluation of an action under the current state and outputs the value A(S, a) of an action under the current state. The action value function is the sum of the value function and the advantage function: Q(S, a) = V(S) + A(S, a). The action value function learns parameters based on existing data through the policy gradient algorithm.

8. An adaptive dialogue recommendation system for new products and new users, characterized in that: include: A graph construction module is used to construct a user interaction graph by taking users, commodities, commodity attributes, and conversations as nodes, and taking the connection between commodities and corresponding commodity attributes, the connection between conversations and preferred commodities or preferred attributes involved in the conversations, the connection between users and their historical conversations, and the connection between users and their friend users as edges between nodes; The embedding module is used to initialize the embedding vectors of the nodes on the user interaction graph. For each node, all source nodes whose target nodes are the node are found. According to the type of the edge between the source node and the target node, the representation of the source node is transformed into a matrix using the graph embedding algorithm. The transformed representations of all source nodes are aggregated, and the aggregation function is learned using the inductive graph algorithm. A feature representation module is used to construct a current state representation based on the representation corresponding to the user who currently initiates the conversation and the representation corresponding to the current conversation, and to construct a current action representation based on the probability that the current candidate node will be selected in the next round given the current conversation and the probability that the current candidate node will be selected in the next round given the node of the current interaction, wherein the current candidate node includes candidate product nodes that meet all the preferred attributes of the user in the current conversation and exclude the candidate product nodes that have been rejected in the current conversation, and candidate attribute nodes that have all the attributes of the current candidate product set and exclude the attributes that the user prefers or has rejected in the current conversation; The strategy learning module is used to introduce state transfer and reward to form a Markov decision process based on the current state representation and the current action representation, and learn the action value function through the deep Q network algorithm; The strategy execution module is used to obtain the representation of actions and states based on the modeled action value function in each round of dialogue recommendation according to the current user interaction graph, and input them into the action value function, and select the input action with the highest output action value for execution.

9. A computer device, characterized in that: include: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the adaptive dialogue recommendation method for new products and new users as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the adaptive dialogue recommendation method for new products and new users as described in any one of claims 1 to 7 are implemented.

Images