Interaction method and model training method and device for recommending items in a conversation

Abstract

Embodiments of the present specification provide an interaction method, a model training method and a device for recommending items in a dialogue. In the interaction method for recommending items in a dialogue, a dialogue history and a candidate information set of a target user are obtained, the dialogue history including historical behaviors in at least one round of dialogue and feedbacks of the target user to the historical behaviors; a current state code of the target user is determined according to the dialogue history and the candidate information set; the current state code is provided to a behavior prediction model to obtain a predicted behavior, the predicted behavior including asking an attribute of the target user or recommending an item; the current state code is provided to a candidate information determination model to obtain a target candidate information matched with the predicted behavior, the target candidate information including candidate attribute information matched with the target user determined from a candidate attribute information set matched with the predicted behavior or candidate item information determined from a candidate item information set.

Description

Technical Field

[0001] The embodiments in this specification generally relate to the field of computer technology, and more particularly to interactive methods, model training methods, and apparatus for recommending items through dialogue. Background Technology

[0002] With the rapid development of artificial intelligence technology, recommendation systems, which aim to provide users with more valuable information in specific scenarios, have been increasingly widely used. In existing technologies, traditional recommendation systems typically push results to users based on their browsing history, user attributes, and product attributes. This leads to risks of infringing on and abusing user privacy. Furthermore, since some user tags and preferences can change dynamically, directly using historical information may result in recommendations that do not meet the user's current needs. Conversational Recommender Systems (CRS), on the other hand, can determine the user's current needs through fluent natural language interaction, even without knowing the user's explicit intent, thus providing more valuable information in specific scenarios. One existing conversational recommendation system typically places all product attributes and actions related to the recommended product into a general action set. This results in an extremely uneven distribution of action types in the action space, often leading to an excessive number of questions about attribute preferences, resulting in a poor user experience. Moreover, the general action set generates a huge candidate space, significantly increasing the difficulty of strategy optimization. Therefore, a conversational recommendation system that can at least partially overcome these shortcomings is needed. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide an interaction method and apparatus for dialogic item recommendation, as well as a model training method and apparatus for dialogic item recommendation. Using this method and apparatus, it is possible to understand the user's preferences during the interaction process and accordingly inquire about attributes or recommend items, without relying too heavily on user profiles. Furthermore, by dividing the interaction process with the user into two stages, the first stage only decides whether to inquire about attributes or recommend items, while the second stage filters the candidate space based on the decision made in the first stage to specifically determine which attribute to inquire about or which item to recommend, thereby improving the effectiveness of item recommendation.

[0004] According to one aspect of an embodiment of this specification, an interactive method for recommending items through dialogue is provided, comprising: acquiring a dialogue history and a candidate information set of a target user, wherein the dialogue history includes historical behaviors in at least one round of dialogue and the target user's feedback on the historical behaviors, the historical behaviors including asking about target attributes or recommending target items, and the candidate information set including a candidate attribute information set and a candidate item information set; determining a current state code of the target user based on the dialogue history and the candidate information set; providing the current state code to a behavior prediction model to obtain a predicted behavior, wherein the predicted behavior includes asking the target user about attributes or recommending items; and providing the current state code to a candidate information determination model to obtain target candidate information matching the predicted behavior, wherein the target candidate information includes candidate attribute information matching the target user determined from the candidate attribute information set matching the predicted behavior or candidate item information determined from the candidate item information set.

[0005] According to another aspect of the embodiments of this specification, a model training method for dialogue-based item recommendation is provided, comprising: determining a sample current state code of a sample user based on a sample user's sample dialogue history and a sample candidate information set; providing the sample current state code to a current first action value determination model to obtain a first action value of the sample current state under each alternative action, wherein the alternative actions include asking the sample user for attributes or recommending items; determining the probability of the sample current state selecting each alternative action based on the first action value of the sample current state under each alternative action; and converting the probability of the sample current state selecting each alternative action into sampling from a classification distribution to obtain a sample. The current state selects the sample transition probability of each candidate behavior; based on the obtained sample current state, the maximum value of the sample transition probability of each candidate behavior is selected, and a sample candidate item information set matching the recommended item or a sample candidate attribute information set matching the query attribute is selected from the sample candidate information set as the sample candidate set; the sample current state encoding is provided to the current second behavior value determination model to obtain the second behavior value of the sample current state under each candidate information in the sample candidate set; and the parameters of the current first behavior value determination model and the current second behavior value determination model are adjusted based on the reinforcement learning algorithm so that the sample user accepts the recommended sample candidate item information.

[0006] According to another aspect of the embodiments of this specification, an interactive device for recommending items through dialogue is provided, comprising: an information acquisition unit configured to acquire a dialogue history and a candidate information set of a target user, wherein the dialogue history includes historical behaviors in at least one round of dialogue and the target user's feedback on the historical behaviors, the historical behaviors including asking about target attributes or recommending target items, and the candidate information set including a candidate attribute information set and a candidate item information set; an encoding determination unit configured to determine a current state encoding of the target user based on the dialogue history and the candidate information set; a behavior prediction unit configured to provide the current state encoding to a behavior prediction model to obtain a predicted behavior, wherein the predicted behavior includes asking the target user about attributes or recommending items; and an information prediction unit configured to provide the current state encoding to a candidate information determination model to obtain target candidate information matching the predicted behavior, wherein the target candidate information includes candidate attribute information matching the target user determined from the candidate attribute information set matching the predicted behavior or candidate item information determined from the candidate item information set.

[0007] According to another aspect of the embodiments of this specification, a model training apparatus for dialogue-based item recommendation is provided, comprising: an encoding unit configured to determine a sample current state encoding of a sample user based on a sample user's sample dialogue history and a sample candidate information set; a first value determination unit configured to provide the sample current state encoding to a current first behavior value determination model to obtain a first behavior value of the sample current state under various candidate behaviors, wherein the candidate behaviors include asking the sample user for attributes or recommending items; and a probability conversion unit configured to determine the probability of the sample current state selecting each candidate behavior based on the first behavior value of the sample current state under each candidate behavior; and to convert the probability of the sample current state selecting each candidate behavior into a probability obtained from a classification distribution. The system comprises: a first behavior value determination model and a second behavior value determination model; a second value determination unit, configured to select the maximum value of the sample transition probabilities of each candidate behavior based on the obtained sample current state, and select a sample candidate item information set that matches the recommended item or a sample candidate attribute information set that matches the query attribute from the sample candidate information set; providing the sample current state encoding to the current second behavior value determination model to obtain the second behavior value of the sample current state under each candidate information in the sample candidate set; and a training unit, configured to adjust the parameters of the current first behavior value determination model and the current second behavior value determination model based on a reinforcement learning algorithm, so that the sample user accepts the recommended sample candidate item information.

[0008] According to another aspect of the embodiments of this specification, an interactive device for recommending items through dialogue is provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored in the memory, wherein the at least one processor executes the computer program to implement the interactive method for recommending items through dialogue as described above.

[0009] According to another aspect of the embodiments of this specification, a model training apparatus for dialogue-based item recommendation is provided, comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored in the memory, wherein the at least one processor executes the computer program to implement the model training method for dialogue-based item recommendation as described above.

[0010] According to another aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the interaction method for recommending items in dialogue and / or the model training method for recommending items in dialogue as described above.

[0011] According to another aspect of the embodiments of this specification, a computer program product is provided, including a computer program that is executed by a processor to implement the interaction method for recommending items in dialogue and / or the model training method for recommending items in dialogue as described above. Attached Figure Description

[0012] A further understanding of the nature and advantages of this specification can be achieved by referring to the following figures. In the figures, similar components or features may have the same reference numerals.

[0013] Figure 1 An exemplary architecture of an interaction method and apparatus for recommending items in dialogue, and a model training method and apparatus for recommending items in dialogue, according to embodiments of this specification, is shown.

[0014] Figure 2 A flowchart illustrating an example of an interactive method for recommending items through dialogue, according to an embodiment of this specification, is shown.

[0015] Figure 3 A schematic diagram illustrating an example of a process for determining the current state code of a target user according to an embodiment of this specification is shown.

[0016] Figure 4 A schematic diagram illustrating an example of the process for determining the values ​​of the elements of an adjacency matrix according to an embodiment of this specification is shown.

[0017] Figure 5A flowchart illustrating yet another example of the process for determining the current state code of a target user according to an embodiment of this specification is shown.

[0018] Figure 6 A schematic diagram illustrating an example of a process for determining predictive behavior according to an embodiment of this specification is shown.

[0019] Figure 7 A flowchart illustrating an example of a process for determining target candidate information according to an embodiment of this specification is shown.

[0020] Figure 8 A flowchart illustrating an example of a model training method for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0021] Figure 9 A schematic diagram illustrating an example of a model training method for recommending items in dialogue, according to an embodiment of this specification.

[0022] Figure 10 A block diagram illustrating an example of an interactive device for recommending items through dialogue, according to an embodiment of this specification.

[0023] Figure 11 A block diagram of an example of an encoding determination unit in an interactive device for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0024] Figure 12 A block diagram showing yet another example of an interactive device for recommending items through dialogue, according to an embodiment of this specification.

[0025] Figure 13 A block diagram of an example of a model training apparatus for dialogue-based item recommendation according to an embodiment of this specification is shown.

[0026] Figure 14 A block diagram illustrating an example of an interactive device for recommending items through dialogue, according to an embodiment of this specification.

[0027] Figure 15 A block diagram of an example of a model training apparatus for dialogue-based item recommendation according to an embodiment of this specification is shown. Detailed Implementation

[0028] The subject matter described herein will be discussed below with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of the embodiments described herein. Various processes or components may be omitted, substituted, or added as needed in the various examples. Furthermore, features described in some examples may be combined in other examples.

[0029] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0030] In this specification, the term "Graph Neural Networks (GNN)" can refer to a connection model that obtains dependencies in a graph by means of information passing between nodes in the network and updates the state of a node by means of its neighbors at any depth.

[0031] In this specification, the term "embedding" can refer to a low-dimensional vector used to represent an object (such as a word, product, text, etc.) that contains semantic information. The properties of such low-dimensional vectors generally allow vectors that are close in distance to correspond to objects with similar meanings.

[0032] In this specification, the term "Q-learning" may refer to a reinforcement learning algorithm based on Markov Decision Processes (MDPs).

[0033] The following will describe in detail, with reference to the accompanying drawings, an interaction method and apparatus for recommending items through dialogue, and a model training method and apparatus for recommending items through dialogue, according to embodiments of this specification.

[0034] Figure 1 An exemplary architecture 100 of an interaction method and apparatus for recommending items in dialogue, and a model training method and apparatus for recommending items in dialogue, according to embodiments of this specification, is shown.

[0035] exist Figure 1In this context, network 110 is used to interconnect terminal device 120 and application server 130.

[0036] Network 110 can be any type of network capable of interconnecting network entities. Network 110 can be a single network or a combination of various networks. In terms of coverage, network 110 can be a local area network (LAN), a wide area network (WAN), etc. In terms of the carrying medium, network 110 can be a wired network, a wireless network, etc. In terms of data switching technology, network 110 can be a circuit-switched network, a packet-switched network, etc.

[0037] Terminal device 120 can be any type of electronic computing device capable of connecting to network 110, accessing servers or websites on network 110, processing data or signals, etc. For example, terminal device 120 can be a desktop computer, laptop computer, tablet computer, smartphone, etc. Although in Figure 1 Only one terminal device is shown in the diagram, but it should be understood that a different number of terminal devices may be connected to network 110.

[0038] In one implementation, terminal device 120 can be used by a user. Terminal device 120 may include an application client (e.g., application client 121) that can provide various services to the user. In one example, the application client may be a shopping application, a financial application, a news application, etc. In some cases, application client 121 may interact with application server 130. For example, application client 121 may transmit user-inputted messages to application server 130 and receive responses associated with those messages from application server 130. However, it should be understood that in other cases, application client 121 may also generate responses to user-inputted messages locally, rather than interacting with application server 130. In this document, "message" can refer to any input information, such as information from user input that represents an acceptance or rejection of recommended items or attributes provided by the application server.

[0039] Application server 130 can connect to candidate information database 140. Candidate information database 140 can include candidate attribute information and candidate item information. In one example, the candidate attribute information can be specific attributes (such as tags like "sports," "stable," and "entertainment"), or feature vectors (such as embeddings) corresponding to specific attributes. Similarly, the candidate item information can be specific item names (such as "mobile phone," "XX fund," or "today's news" sections), or feature vectors (such as embeddings) corresponding to specific items.

[0040] It should be understood that Figure 1All network entities shown are exemplary, and any other network entities may be involved in Architecture 100 depending on the specific application requirements.

[0041] Figure 2 A flowchart of an interaction method 200 for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0042] like Figure 2 As shown in 210, the target user's dialogue history and candidate information set are obtained.

[0043] In this embodiment, the target user's dialogue history and candidate information set can be obtained. The dialogue history can include historical behaviors from at least one round of dialogue and the target user's feedback on those behaviors. The historical behaviors can include asking about target attributes or recommending target items. The feedback can include acceptance or rejection. The candidate information set can include a candidate attribute information set and a candidate item information set.

[0044] In one example, the target user's conversation history can be used To represent. Among them, f m ∈{acc,rej}. The above This can be used to represent the current (t-th round of dialogue) dialogue history. The above h m This can be used to represent the m-th round of dialogue. (The above...) This can be used to represent the entity that performs the aforementioned interactive method for recommending items in a dialogue (e.g., Figure 1 The application server 130 in the example (in the example) and the target user have a history of dialogue in the m-th round. The `ask` above can be used to represent querying for target attributes. The `rec` above can be used to represent recommending target items. The `f` above... m Used to represent the target user's historical behavior in the above m-th round of dialogue. The above acc is used to represent the feedback that the target user will accept as historical behavior for the m-th round of the conversation. The feedback. The `rej` above is used to indicate that the target user will refuse as part of the historical behavior for the `m`th round of the conversation. Feedback.

[0045] In one example, optionally, the aforementioned historical behavior may only distinguish between querying attributes or recommending items, without specifically recording which attribute was queried or which item was recommended. The aforementioned items can include a broad range of tangible or intangible goods. For example, these items can be tangible goods such as mobile phones, clothing, cars, and books, or virtual goods such as prepaid credits, internet data, e-books, and e-tickets, or even funds and information.

[0046] In one example, there is a relationship between item information and attribute information. For instance, an item can have multiple attributes, and correspondingly, the item information corresponding to that item can be associated with the attribute information corresponding to those multiple attributes. As another example, an attribute can be possessed by multiple items simultaneously, and correspondingly, the attribute information corresponding to that attribute can be associated with the item information corresponding to those multiple items.

[0047] In one example, the aforementioned candidate information set can be dynamically adjusted based on the progress of the dialogue with the target user. For instance, the initial candidate information set could be the complete set of candidate attribute information and candidate item information, or it could be composed of a subset of attribute and item information recalled from the full information set based on the target user's user profile or the preferences of users similar to the target user. For example, the aforementioned candidate information set can be modified by adding or deleting corresponding elements from the relevant candidate attribute and candidate item sets based on whether the target user accepts or rejects the corresponding historical behavior.

[0048] In step 220, the current state code of the target user is determined based on the dialogue history and candidate information set.

[0049] In this embodiment, the dialogue history and candidate information set can be encoded in various ways to obtain the target user's current state code. In one example, the item2vec method can be used to encode the dialogue history and candidate information set separately, and then the obtained codes can be fused to obtain the target user's current state code.

[0050] Optionally, refer to Figure 3 , Figure 3 A schematic diagram illustrating an example of a process 300 for determining the current state code of a target user according to an embodiment of this specification is shown.

[0051] like Figure 3 As shown, the aforementioned candidate information set may include a user preference interaction graph (as shown in Figure 311). This user preference interaction graph may include a node set and an adjacency matrix. The node set may include target user nodes, candidate item node sets, and candidate attribute node sets. The adjacency matrix can be used to represent the relationship weights between nodes.

[0052] In one example, the above user preference interaction graph can be represented by G. t =(N t A t This is represented by ) . Among them The above G t This can be used to represent the user preference interaction graph at the current (t-th round of the conversation). The above N tThis can be used to represent the set of nodes in the current user preference interaction graph. The `u` above can be used to represent the target user node. It can be used to represent the current set of candidate attribute nodes. The above V t This can be used to represent the current set of candidate item nodes. The above A t This can be used to represent the current adjacency matrix. Wherein, the aforementioned adjacency matrix A... t elements in The value can be used to represent the size of the association between node i and node j.

[0053] Optionally, refer to the following: Figure 4 , Figure 4 A schematic diagram illustrating an example of a process 400 for determining the values ​​of elements of an adjacency matrix according to an embodiment of this specification is shown.

[0054] like Figure 4 As shown, node 1 can be a target user node. Nodes 2, 3, and 5 can be item nodes. Nodes 4, 6, 7, 8, 9, and 10 can be attribute nodes. The edges ak between nodes can be used to represent the relationships between nodes. Node 3 can be used to represent rejected items. Nodes 4 and 6 can be used to represent rejected attributes. Nodes 8 and 9 can be used to represent accepted attributes. Therefore, candidate item nodes can include nodes 2 and 5, and candidate attribute nodes can include nodes 7, 8, and 9.

[0055] In this embodiment, for the first element that represents the relationship weight between the candidate item node and the candidate attribute node related to the candidate item and / or the relationship between the target user node and the first candidate attribute node, the value of the first element is determined to be a first preset value that represents the existence of a positive correlation.

[0056] The aforementioned first candidate attribute node can be used to represent the candidate attributes that the target user was asked about and accepted. In one example, the correlation between candidate attribute information and candidate item information in the candidate information set can be used to determine whether the corresponding candidate attribute node and candidate item node are related. In another example, the attributes targeted by the target user's accepted historical behavior can be determined based on the target user's dialogue history, and the nodes corresponding to the determined attributes can be identified as the aforementioned first candidate attribute nodes. The elements in the adjacency matrix used to represent the relationships between the nodes can be identified as the first element, and the first element can be assigned a first preset value representing a positive correlation. For example, the first preset value can be 1, 0.5, 10, etc.

[0057] In one example, such as Figure 4 As shown, when the item represented by node 2 possesses the attribute represented by node 9, then the first element corresponding to edge e (e.g.) The value of ) can be the first preset value mentioned above. Similarly, when the item represented by node 5 has the attribute represented by node 7, then the first element corresponding to edge h (e.g. The value of ) can be the first preset value mentioned above.

[0058] In one example, such as Figure 4 As shown, if the attribute represented by node 8 is accepted by the user represented by node 1, then the first element corresponding to edge f (e.g.) The value of ) can be the first preset value mentioned above.

[0059] In this embodiment, for the second element that represents the relationship between the target user node and the second candidate attribute node, the value of the second element is determined to be a second preset value that represents the existence of a negative association.

[0060] The aforementioned second candidate attribute node can be used to represent the candidate attributes that the target user was asked about and rejected. In one example, the attribute targeted by the target user's rejection history can be determined based on the target user's dialogue history, and the node corresponding to the determined attribute can be identified as the second candidate attribute node. The element in the adjacency matrix used to represent the relationship between the nodes can be identified as the second element, and the second element can be assigned a second preset value representing a negative association. For example, the second preset value could be -1, -0.5, -10, etc.

[0061] In one example, such as Figure 4 As shown, if the attribute represented by node 4 is rejected by the user represented by node 1, then the second element corresponding to edge c (e.g.) The value of ) can be the second preset value mentioned above.

[0062] It should be noted that the values ​​of the first and second preset values ​​mentioned above should be numerically compatible (e.g., differing by no more than one order of magnitude), and are generally numerically compatible with the values ​​of the matching degree described below.

[0063] In this embodiment, the third element, which represents the weight of the relationship between the target user node and the candidate item node, is determined to be the matching degree.

[0064] The matching degree mentioned above can be obtained from the vector dot product of the target user and the candidate item, the vector dot product of the target user and the candidate attribute that was asked and accepted, and the vector dot product of the target user and the candidate attribute that was asked and rejected.

[0065] In one example, the candidate attribute information and candidate item information included in the above candidate information set can be vectors corresponding to the candidate attributes and candidate items, respectively. These vectors can be embeddings obtained using various pre-trained models (such as BERT models, transE (Translating Embedding) models, or item2vec, or they can be embeddings obtained by fine-tuning the pre-trained model during joint training with the behavior prediction model and candidate information determination model described below. Similarly, the target user can also obtain corresponding vectors (e.g., embeddings) in a similar manner.

[0066] In one example, the value of the third element mentioned above... It can be Among them, e u e v e p These can be used to represent vectors corresponding to the target user, candidate items, and candidate attributes, respectively. The above... This can be used to represent the set of candidate attributes that have been queried and accepted by the target user up to the current point in time. The above This can be used to represent the set of candidate attributes that have been asked and rejected by the target user up to the current point in time. In one example, the value of the third element above... It can be The above σ() can be used to represent the sigmoid function.

[0067] In one example, such as Figure 4 As shown, the third element corresponding to edge a (e.g.) The value of ) is the inner product of the vectors corresponding to the user represented by node 1 and the item represented by node 2, plus the inner product of the vectors corresponding to the user represented by node 1 and the attribute represented by node 8, and then the inner product of the vectors corresponding to the user represented by node 1 and the attribute represented by node 4 is subtracted.

[0068] It should be noted that the order in which the values ​​of the elements in the above adjacency matrix are determined is not limited.

[0069] In some optional implementations of this embodiment, the other elements of the adjacency matrix can also be assigned a value of 0.

[0070] Based on this, this solution provides a method for determining the numerical values ​​of each element in the adjacency matrix, thereby dynamically constructing the relationship between each element in the candidate information set and the target user, providing a technical foundation for the effective determination of subsequent target candidate information.

[0071] Back Figure 3In step 310, the user preference interaction graph is provided to the graph neural network to obtain the current preference encoding of the target user.

[0072] In this embodiment, a user preference interaction graph (as shown in Figure 311) can be provided to a graph neural network (as shown in Figure 312) to obtain the target user's current preference encoding (as shown in Figure 313). The graph neural network can be, for example, a graph encoder built based on a graph convolutional network architecture. In one example, the graph neural network can be trained together with the behavior prediction model and the candidate information determination model described below.

[0073] In one example, the embedding X of the node in the user preference interaction graph above g It can be done Obtained. Among them, This can represent the new adjacency matrix obtained by combining the above adjacency matrix with a diagonal matrix, which incorporates its own information. It can be used to represent the corresponding diagonal matrix. (l) can be used to represent the layers of the above graph encoder. and These can be used to represent the outputs of the (l-1)th layer (i.e., the penultimate layer) and the (l)th layer (i.e., the last layer) of the above graph encoder, respectively.

[0074] In one example, the target user's current preference encoding It can be This refers to the embedding of the target user node in the aforementioned user preference interaction graph. Therefore, the target user's current preference encoding can be used to characterize the target user's preferences for candidate items and candidate attributes.

[0075] In step 320, the target user's current state code is determined based on the target user's current preference code.

[0076] In this embodiment, the target user's current state code can be determined in various ways based on the target user's current preference code. In one example, the target user's current preference code can be directly determined as the target user's current state code. In another example, other encoding information can be fused with the target user's current preference code to obtain the target user's current state code. The other encoding information may represent, but is not limited to, at least one of the following: the target user's dialogue history, the candidate space of the candidate information set, candidate items and / or candidate attributes in the candidate information set.

[0077] Based on this, this solution provides a method for learning user preferences for items and attributes from graph interaction information based on graph encoders, and improves the representation ability of the target user's current state encoding by fusing this information, which helps to describe user preferences more accurately.

[0078] Optionally, continue to refer to Figure 5 , Figure 5 A flowchart illustrating yet another example of a process 500 for determining the current state code of a target user according to an embodiment of this specification is shown.

[0079] In 510, the dialogue history is converted into a dialogue history embedding.

[0080] In this embodiment, dialogue history can be converted into dialogue history embeddings in various ways. In one example, a pre-trained model (e.g., a BERT model or a transE model) or word2vec can be used to convert the dialogue history into dialogue history embeddings. In one example, regarding the aforementioned dialogue history... You can get the dialogue history embedding Among them, the above These can be compared with the aforementioned dialogue history. h 1 ,h 2 Correspondingly, and so on.

[0081] In step 520, the current dialogue state code is determined based on the dialogue history embedding.

[0082] In this embodiment, the current dialogue state encoding described above can be used to characterize whether the user preferences learned based on the dialogue history are sufficiently accurate for the recommended items. For example, if the target user accepts the requested target attribute for several consecutive rounds, the current dialogue state encoding at this time is used to characterize a preference for recommending items rather than continuing to request attributes.

[0083] In one example, the obtained dialogue history embedding can be provided to a sequence processing model to obtain the current dialogue state encoding. The sequence processing model can include, but is not limited to, at least one of the following: Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM) network, or Gated Recurrent Unit (GRU).

[0084] In one example, the current dialogue state code can be: Among them, the above and These can be used to characterize the dialogue state encoding of the (t-1)th round (previous round) of dialogue and the historical behavior of the (t-1)th round (previous round) of dialogue and the target user's feedback on that historical behavior.

[0085] In 530, the candidate space encoding is determined based on the number of elements contained in the candidate information set.

[0086] In this embodiment, the candidate space encoding can be determined in various ways based on the number of elements contained in the candidate information set. In one example, the number of all elements contained in the candidate information set can be encoded to obtain the candidate space encoding. In another example, the number of candidate item information items contained in the candidate information set can be encoded to obtain the candidate space encoding. In yet another example, the number of candidate attribute information items contained in the candidate information set can be encoded to obtain the candidate space encoding. The above encoding can employ various mapping methods, such as sine / cosine encoding.

[0087] At 540, the current state code of the target user is determined based on the current preference code, the dialogue state code, and the candidate space code.

[0088] In this embodiment, the current preference code, dialogue state code, and candidate space code obtained above can be fused in various ways to obtain the target user's current state code. In one example, the fusion can be element-wise addition. In another example, the fusion can be concatenating vectors. For example, the target user's current state code can be... Among them, the above These can be used to represent the target user's current preference encoding, dialogue state encoding, and candidate space encoding, respectively.

[0089] It should be noted that you can also execute step 530 first, and then execute steps 510 and 520. There is no restriction here.

[0090] Based on this, this solution provides an encoding method that integrates user preferences for items and attributes, current dialogue state, and candidate space. This allows the current state encoding of the target user to integrate the required information as comprehensively as possible, thereby further improving the representation ability of the target user and providing a technical foundation for more accurate prediction of target candidate information.

[0091] Back Figure 2 In step 230, the current state code is provided to the behavior prediction model to obtain the predicted behavior.

[0092] In this embodiment, the obtained current state code can be provided to the behavior prediction model to obtain the predicted behavior. The predicted behavior may include asking the target user for attributes or recommending items. In one example, the behavior prediction model may be a trained binary classification model.

[0093] Optionally, the behavior prediction model described above may include a feedforward neural network (FNN) and a random resampling model. (See reference) Figure 6 , Figure 6 A flowchart illustrating an example of a process 600 for determining predictive behavior according to an embodiment of this specification is shown.

[0094] In step 610, the current state encoding is provided to the feedforward neural network to obtain the initial probabilities of each alternative behavior.

[0095] In this embodiment, the alternative actions may include asking the target user for attributes or recommending items. In one example, the aforementioned neural network can obtain an initial probability of asking the target user for attributes or recommending items to the target user, corresponding to the current state encoding. For example, the initial probability of asking the target user for attributes is 0.7, and the initial probability of recommending items to the target user is 0.3.

[0096] In step 620, the initial probabilities of each alternative behavior are provided to the random resampling model to obtain the transition probabilities of each alternative behavior.

[0097] In this embodiment, the aforementioned random resampling model is used to sample from the categorical distribution to achieve reparameterization. The random resampling model can be, for example, a Gumbel-softmax model. By controlling the temperature parameter of the Gumbel-softmax model, the sharpness of the distribution of transition probabilities for each alternative behavior can be controlled. For example, the transition probability for a target user asking about attributes is 0.95, and the transition probability for a target user recommending items is 0.05.

[0098] At 630, the predicted behavior is determined as the alternative behavior with the highest conversion probability.

[0099] In this embodiment, the predicted behavior can be determined as the candidate behavior with the highest conversion probability. For example, if the conversion probability of asking the target user about attributes is 0.95 and the conversion probability of recommending items to the target user is 0.05, the predicted behavior can be determined as asking the target user about attributes.

[0100] Based on this, a random resampling model is used to improve upon the common practice of directly selecting the candidate behavior with the highest probability. Instead, the initial probability is reparameterized to obtain the transition probability, and then the candidate behavior with the highest transition probability is determined as the prediction behavior. This introduces a certain degree of uncertainty into the prediction process, which helps to correct the prediction behavior when the model training effect is poor, thereby improving the prediction accuracy of subsequent target candidate information.

[0101] Back Figure 2 In step 240, the current state code is provided to the candidate information determination model to obtain target candidate information that matches the predicted behavior.

[0102] In this embodiment, the obtained current state code can be provided to the candidate information determination model to obtain target candidate information matching the predicted behavior. The target candidate information may include candidate attribute information matching the target user determined by the candidate information determination model from the candidate attribute information set matching the predicted behavior, or candidate item information matching the target user determined from the candidate item information set matching the predicted behavior. In one example, the behavior prediction model may be a trained multi-classification model. In one example, when the predicted behavior is to query the target user for attributes, the target candidate information may be the top N candidate attribute information with the highest scores output by the candidate information determination model. In one example, when the predicted behavior is to recommend items to the target user, the target candidate information may be the candidate item information with the highest scores output by the candidate information determination model.

[0103] Optionally, refer to the following: Figure 7 , Figure 7 A flowchart illustrating an example of a process 700 for determining target candidate information according to an embodiment of this specification is shown.

[0104] 710. Determine whether the predicted behavior is recommending items to the target user.

[0105] If the above judgment is correct, proceed with the following recommended steps:

[0106] At 720, the set of accepted attributes is determined from the set of candidate attribute information based on the dialogue history.

[0107] In this embodiment, the set of attributes that the target user was asked for and accepted can be determined based on the dialogue history. In one example, the candidate attribute information set may include attribute A, attribute C, attribute D, attribute F, and attribute G. The attributes that the target user was asked for and accepted may include attribute A and attribute F. Therefore, the aforementioned accepted attribute set may include attribute A and attribute F.

[0108] At 730, a set of alternative item information that matches the attributes in the set of accepted attributes is determined from the set of candidate item information.

[0109] In this embodiment, the aforementioned matching can mean that the item indicated by the item information possesses the attribute. In one example, the item indicated by the item information in the candidate item information set typically possesses all attributes in the accepting attribute set. For example, the candidate item information set may include item x, item y, and item z. Item x possesses attribute A and attribute F. Item y possesses attribute A, attribute C, and attribute F. Item z possesses attribute F and attribute G. Thus, the aforementioned candidate item information set may include item information corresponding to item x and item y. In another example, the item indicated by the item information in the candidate item information set may possess some attributes in the accepting attribute set (e.g., possessing more than 70% of the attributes in the accepting attribute set). In yet another example, the item indicated by the item information in the candidate item information set may possess all attributes in the accepting attribute set and possess one attribute outside the accepting attribute set (i.e., an additional attribute).

[0110] At 740, the current state code is provided to the candidate information determination model to obtain the recommended value corresponding to each candidate item in the candidate item information set.

[0111] In this embodiment, based on the current state encoding, the candidate information determination model can output the recommended value (e.g., the probability score output by logits or softmax) corresponding to each candidate item in the candidate item information set.

[0112] At 750, target item information is selected from the set of candidate item information as target candidate information that matches the predicted behavior.

[0113] In this embodiment, the target item information includes candidate item information whose recommended values ​​meet preset requirements. In one example, a recommended value meeting the preset requirements may exceed a preset recommended value threshold (e.g., 0.7). In another example, a recommended value meeting the preset requirements may be the highest recommended value obtained.

[0114] Based on this, users' conversation history can be used to quickly capture attributes that users are interested in, which can significantly improve the efficiency of identifying target candidate information when the candidate item information space is large.

[0115] In some optional implementations of this embodiment, the following steps may also be performed:

[0116] In 760, the target item information is provided to the target user.

[0117] In one example, target item information can be provided to the target user. This target item information can be provided to the target user in various forms. For example, the most matching target item information can be provided. Another example is that several (e.g., 3, 5) target items can be arranged in a list format, ordered from highest to lowest matching degree.

[0118] It should be noted that the target user to whom the target item information is provided can be the same user as the target user described in step 210 above, or a user using the same client as the target user described in step 210 above; there is no limitation here.

[0119] At 770, determine whether the target user rejects the target item information.

[0120] In one example, feedback from the target user can be received. For instance, the target user can click the "Accept" or "Reject" button. Another example is that the target user can enter a statement such as "No," "No," or "Next" to indicate rejection.

[0121] If the above judgment is correct, proceed with the following steps:

[0122] In 771, determine the additional attributes of the target item information rejected by the target user relative to the set of accepted attributes.

[0123] In one example, the additional attributes of the target item information rejected by the target user relative to the set of accepted attributes are determined. As mentioned earlier, the target item information can be the item information corresponding to item y that has attributes A, C, and F. The additional attribute can be attribute C relative to attributes A and F in the set of accepted attributes.

[0124] In 772, the dialogue history and candidate information set are updated based on the additional attributes.

[0125] In one example, the additional attribute (e.g., attribute C) can be removed from the candidate information set, thereby updating the candidate information set. In another example, the item information corresponding to an item with the additional attribute (e.g., attribute C) can be removed from the candidate item information set, thereby updating the candidate information set. In yet another example, information representing a query about the additional attribute and the target user's rejection of that additional attribute can be used as dialogue history (e.g., New sequence elements (e.g., h) in ) t ).

[0126] Based on this, the rejection of target item information by the target user relative to the additional attributes of the accepted attribute set can be considered equivalent to the rejection of additional attributes, thereby improving the efficiency of determining target candidate information.

[0127] In some optional implementations of this embodiment, the behavior prediction model and the candidate information determination model can be trained based on a first behavior value function for determining the attributes to be asked or the items to be recommended to the target user and a second behavior value function for determining the target candidate information from the candidate information set, respectively.

[0128] In one example, the first-row value function described above can be used to determine the expected reward for querying attributes or recommending items to the target user. The second-row value function described above can be used to determine the expected reward for providing target candidate information to the target user.

[0129] In one example, the first and second rows of the above-mentioned value functions can be implemented as a model, and the specific training method can be found below. Figure 8 or Figure 9 The described method is a model training method for the current first row value determination model and the current second row value determination model.

[0130] use Figures 1-7 The disclosed interactive method for recommending items in dialogue can first determine whether the current behavior is to inquire about attributes or recommend items by providing the encoding of the user's current state to a behavior prediction model. Then, based on the determined current behavior, a corresponding candidate space is determined as a set of candidate attribute information or a set of candidate item information. Next, by providing the encoding of the user's current state to a candidate information determination model, matching target candidate information is determined from the corresponding candidate space. This allows for understanding user preferences during dialogue interaction and inquiring about attributes or recommending items accordingly, without relying too heavily on user profiles. Furthermore, by dividing the interaction process with the user into two stages, the first stage only decides whether to inquire about attributes or recommend items, while the second stage filters the candidate space based on the decision from the first stage to specifically determine which attribute to inquire about or which item to recommend, thereby improving the effectiveness of item recommendations.

[0131] The following is for reference. Figure 8 , Figure 8 A flowchart of a model training method 800 for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0132] like Figure 8 As shown in Figure 810, the sample current state code of the sample user is determined based on the sample user's sample dialogue history and sample candidate information set.

[0133] In this embodiment, the acquired sample user's sample dialogue history and sample candidate information set can be encoded in various ways to obtain the sample user's current state encoding. In one example, training samples consisting of the sample user's sample dialogue history and sample candidate information set can be pre-acquired. The sample dialogue history can include sample historical behaviors from at least one round of dialogue and the sample user's feedback to those behaviors. These historical behaviors can include asking about target attributes or recommending target items. The feedback can include acceptance or rejection. The sample candidate information set can include a sample candidate attribute information set and a sample candidate item information set.

[0134] It should be noted that different sample users can be distinguished, for example, by user identifiers. The relevant descriptions of the sample dialogue history and sample candidate information sets mentioned above can be found in the foregoing. Figure 2 The corresponding description of step 210 in the embodiment will not be repeated here.

[0135] It should also be noted that the specific encoding methods mentioned above can be found in the preceding text. Figure 2 Step 220 of the embodiment Figure 3 and Figure 5 The process for determining the current state code of the target user in the embodiment, steps 300 and 500, and Figure 4 The corresponding descriptions in the embodiments will not be repeated here.

[0136] In step 820, the current state code of the sample is provided to the current first-line value determination model to obtain the first-line value of the current state of the sample under each alternative line of action.

[0137] In this embodiment, the first behavior value determination model can include various models capable of binary classification. The alternative behaviors can include asking the sample user for attributes or recommending items. In one example, the first behavior value determination model can include a first state value function and a first dominance function. In one example, the first state value function and the first dominance function can be implemented as independent multi-layer perceptrons (MLPs). In one example, the first behavior value determination model can also be implemented as other artificial neural networks (ANNs) capable of achieving the aforementioned classification effect.

[0138] In one example, the first row is the value. According to Calculation. Among them, The above s t This can be used to represent the current state encoding of a sample. (The above...) These can be used to represent alternative actions. The `ask` above can be used to indicate a query for an attribute. The `rec` above can be used to indicate a recommended item. This can be used to represent the first state value function. (The above...) It can be used to represent the first dominant function.

[0139] In 830, based on the first value of the current state of the sample under each alternative action, the probability of the current state of the sample choosing each alternative action is determined.

[0140] In this embodiment, based on the first-order value of the sample's current state under each candidate action, the probability of the sample's current state selecting each candidate action can be determined in various ways. In one example, various activation functions (such as softmax) can be used to convert the aforementioned first-order values ​​into corresponding probabilities.

[0141] For example, choosing behavior in the current state of a sample. The probability can be Among them, A 1 ={ask,rec} can refer to the candidate space of alternative behaviors.

[0142] In 840, the probability of a sample choosing each alternative behavior in its current state is transformed into the sample transition probability of choosing each alternative behavior in its current state by sampling from the classification distribution.

[0143] In this embodiment, various random resampling methods can be used to transform the probability of selecting each alternative behavior from the current state of the sample, thereby achieving reparameterization. In one example, the Gumbel-softmax model can be used.

[0144] For example, choosing behavior in the current state of a sample. The sample transition probability can be expressed as follows:

[0145]

[0146] Where Gumbel noise ∈ = -log(-log(x)). x can be obtained by sampling the mean distribution Uniform(0,1). The temperature parameter τ controls the sharpness of the probability distribution of the generated sample transition probabilities. When τ is sufficiently small, the above probability distribution approximates the one-heat distribution, therefore... Close to 0 or 1.

[0147] At 850, based on the current state of the obtained sample, the maximum value of the sample transition probability of each alternative behavior is selected, and the sample candidate item information set that matches the recommended item or the sample candidate attribute information set that matches the query attribute is selected from the sample candidate information set as the sample candidate set.

[0148] In this embodiment, the selected action can be determined based on the maximum value of the sample conversion probabilities obtained above. In one example, when the sample's current state has the highest probability of converting to the user's query attribute, a set of candidate attribute information can be selected from the candidate sample information set as the candidate sample set. Similarly, in another example, when the sample's current state has the highest probability of converting to the user's recommended item, a set of candidate item information can be selected from the candidate sample information set as the candidate sample set.

[0149] In 860, the current state of the sample is encoded and provided to the current second-row value determination model to obtain the second-row value of the current state of the sample under each candidate information in the sample candidate set.

[0150] In this embodiment, the second-behavior value determination model can include various models capable of multi-class classification. The candidate information can be consistent with the sample candidate attribute information or sample candidate item information contained in the sample candidate set. In one example, the second-behavior value determination model can include a second state value function and a second dominance function. In one example, the second state value function and the second dominance function can be implemented as independent multilayer perceptrons. In one example, the second-behavior value determination model can also be implemented as other artificial neural networks capable of achieving the aforementioned classification effect.

[0151] In one example, based on the above Behavior selection, second behavior value The calculation can be performed as follows:

[0152]

[0153] Among them, the above s t This can be used to represent the current state encoding of a sample. (The above...) This can be used to represent the behavior determined based on the value of the first action. (The above...) It can be used to represent alternative information. The above This can be used to represent the second state-value function. (The above...) This can be used to represent the second dominant function. (The above...) This is used to represent the decision mask controlling the candidate set space based on the selected action on the user's query attribute or recommended item. That is, when the selected action is on the user's query attribute, then... The candidate set space representing the alternative information is the sample candidate attribute information set; when the selected action is to recommend items to the user, then... The candidate set space represented by the alternative information is the sample candidate item information set.

[0154] In 870, the parameters of the current first-row value determination model and the current second-row value determination model are adjusted based on the reinforcement learning algorithm to make the sample users accept the recommended sample candidate item information.

[0155] In this embodiment, the first row can be used as the value, with γ as the depreciation rate, based on the delayed reward. The second row is the value As two-stage behaviors of reinforcement learning and The expected reward. The first row above is the value. Equivalent to the first stage of behavior The supervisory signal is used to determine whether to inquire about attributes or recommend items. The second row above is the value. This can be used for the second phase of behavior. The supervision signal is used to determine which attribute to query or which item to recommend. The aforementioned reinforcement learning algorithms may include, for example, Q-learning and Sarsa (state-action-reward-state-action) algorithms.

[0156] In one example, during each round of training, a corresponding reward is given based on the result of the first phase selection. Rewards will be given based on the results of the second stage selection. The candidate action space A for the next round of dialogue can be updated based on the real-time feedback from sample users in each round of dialogue. t+1 Among them, the candidate action space A t+1 This can include the action selection space of the first stage (e.g.) The candidate space) and the information selection space in the second stage (e.g. (The candidate space). Optionally, a replay buffer D can be defined to store the results of each round. During training, a batch of data can be sampled from the replay buffer each time as processed training samples, and the model can be optimized using the following loss function:

[0157]

[0158]

[0159] Where, θ Q ={θ V ,θ A}. This can be achieved using the loss function L1(θ) described above. Q ) and L2(θ QThis is used to optimize the model parameters of the current first-row value determination model and the current second-row value determination model. In one example, this is equivalent to optimizing the model parameters of the first state value function, the first dominance function, the second state value function, and the second dominance function.

[0160] and The target values ​​of the Q-values ​​for the two stages of action can be represented separately, and they can be calculated based on the Bellman Optimality Equation:

[0161]

[0162]

[0163] Among them, A 1 ={ask,rec} can be used to represent the candidate space for the first-stage behavior. This can be used to represent the candidate space for the second-stage behavior under the behavior selected in the first stage. The above... It can include and It can be used to represent the current set of candidate attribute information for samples as a candidate space. It can be used to represent the current set of candidate item information as the candidate space.

[0164] Optionally, when the sample current state encoding of the above sample users involves the above... Figure 3 When using the graph neural network described in the embodiment, the model parameters of the graph neural network can be adjusted simultaneously with the model parameters of the current first row value determination model and the current second row value determination model using the reinforcement learning algorithm described above.

[0165] Optionally, while adjusting the model parameters of the current first row value determination model and the current second row value determination model using the reinforcement learning algorithm, the embeddings corresponding to the target user, candidate item information, and candidate attribute information can also be fine-tuned.

[0166] In one example, a dialogue-based item recommendation model can be obtained by determining the model based on the trained current first action value and the current second action value. This dialogue-based item recommendation model can include a behavior prediction model and a candidate information determination model, respectively derived from the trained current first action value determination model and the current second action value determination model. For example, the behavior prediction model of the dialogue-based item recommendation model can determine whether to query an attribute or recommend an item based on the provided current state code, thereby determining a candidate space from the candidate information set for the candidate information determination model. Then, the candidate information determination model of the dialogue-based item recommendation model can determine the target attribute information or the target item information from the determined candidate space.

[0167] Continue to refer to Figure 9 , Figure 9 A schematic diagram of an example of a model training method 900 for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0168] like Figure 9 As shown, it is possible to determine the sample dialogue history of sample user 1 (e.g., ... Figure 9 (as shown in Figure 911) and sample candidate information set (such as...) Figure 9 As shown in Figure 913, determine the current state code of sample user 1 (e.g., ...). Figure 9 As shown in Figure 920). Among them, the sample candidate information set 913 can be based on the current user preference interaction graph (e.g., ...). Figure 9 As shown in Figure 912, this is confirmed. Figure 9 For an example of nodes in the user preference interaction diagram 912, please refer to the previous section. Figure 4 The corresponding description in the embodiments. Then, the sample current state code 920 can be provided to the first row value determination model (e.g., Figure 9 As shown in Figure 930), the first row value of the current state of the sample under each alternative behavior is obtained (e.g., Figure 9 (As shown in 940). Then, based on the first value 940 of the sample's current state under each alternative behavior, the probability of the sample's current state choosing the recommended item or querying the attribute can be determined (e.g., ...). Figure 9 (As shown in 950). Next, the probability of selecting a recommended item or querying an attribute in the current state of the sample can be converted from a categorical distribution (e.g., 950). Figure 9 Sampling is performed in (as shown in 960) to obtain the sample transition probability of selecting the recommended item or the query attribute in the current state of the sample. Then, in 970, the maximum value of the sample transition probability of each alternative behavior can be selected based on the obtained current state of the sample, and a sample candidate attribute information set (such as...) matching the query attribute can be selected from the sample candidate information set 913. Figure 9As shown in Figure 971, the sample candidate set is selected. It can be understood that when the sample's current state has a greater probability of selecting the recommended item than the query attribute, a sample candidate item information set matching the recommended item (such as...) is selected from the sample candidate information set 913. Figure 9 (As shown in Figure 972) is used as the candidate set of samples. Then, the current state code 920 of the samples can be provided to the second-row value determination model (such as...). Figure 9 As shown in Figure 980), the current state of the sample in the sample candidate set (e.g.) is obtained. Figure 9 The second row of each alternative information shown in Figure 971 contains values ​​(e.g., ...). Figure 9 As shown in Figure 990). In one example, the second row of the above-mentioned value determination model 980 can be based on the embeddings corresponding to each candidate information (e.g., candidate attribute information 7, 8, 9) in the candidate sample set 971. Figure 9 Similarity calculations are performed between the first and second behavior values ​​(as shown in Figure 973) and the embeddings of the sample users to obtain the second behavior value for each candidate information. Then, reinforcement learning algorithms can be used to adjust the parameters of the first behavior value determination model 930 and the second behavior value determination model 980 based on the differences between the obtained first and second behavior values ​​corresponding to the two stages and their respective Q-value target values, thereby achieving training of the first behavior value determination model 930 and the second behavior value determination model 980.

[0169] use Figures 8-9The disclosed model training method for dialogue-based item recommendation divides each round of interaction into two interconnected stages. In the first stage, the method effectively selects the action (asking for attributes or recommending items) based on historical interactions. In the second stage, it sorts the information in the corresponding candidate space determined by the action selected in the first stage to obtain the item that will be successfully recommended. Specifically, by converting the probability into sampling from a classification distribution to obtain the corresponding sample transformation probability, the method avoids the non-differentiability of the discreteness of the action selection in the first stage for model learning. Furthermore, it models the uncertainty of the interaction between the two stages, especially in the early stages of model training when the choice of the action in the first stage often has a high degree of randomness, leading to uncertainty in the interaction between the two stages (e.g., the first stage chooses to ask for attributes, but the goal of the second stage is to recommend a certain item), which helps to accelerate model convergence. Moreover, this scheme designs corresponding rewards for the two stages (i.e., Q1 and Q2 mentioned above) based on reinforcement learning algorithms. It cleverly transforms the final goal (i.e., successfully recommending products to users) which can usually only serve as a supervision signal in the second stage into a supervision signal that can also provide direction for the behavior selection in the first stage during each round of interaction. Compared with using loss functions such as cross-entropy only in the second stage, it can better consider long-term benefits from the perspective of the final goal and avoid getting trapped in local optima.

[0170] Figure 10 A block diagram illustrating an example of an interactive device 1000 for recommending items in a conversation, according to an embodiment of this specification. This device embodiment can be used with... Figures 2-7 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0171] like Figure 10 As shown, the interactive device 1000 for recommending items through dialogue may include an information acquisition unit 1010, an encoding determination unit 1020, a behavior prediction unit 1030, and an information prediction unit 1040.

[0172] The information acquisition unit 1010 is configured to acquire the dialogue history and candidate information set of the target user. The dialogue history includes historical actions from at least one round of dialogue and the target user's feedback on those historical actions. The historical actions include asking about target attributes or recommending target items. The candidate information set includes a candidate attribute information set and a candidate item information set. The operation of the information acquisition unit 1010 can be referenced above. Figure 2 The operation described in section 210.

[0173] The encoding determination unit 1020 is configured to determine the current state encoding of the target user based on the dialogue history and the candidate information set. The operation of the encoding determination unit 1020 can be referred to above. Figure 2 The operation described in section 220.

[0174] Further reference Figure 11 , Figure 11 A block diagram of an example of an encoding determination unit 1100 in an interactive device for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0175] In this embodiment, the aforementioned candidate information set includes a user preference interaction graph. The user preference interaction graph includes a node set and an adjacency matrix. The node set includes target user nodes, candidate item node sets, and candidate attribute node sets. The adjacency matrix is ​​used to represent the relationship weights between nodes.

[0176] In one example, the values ​​of the elements of the adjacency matrix are determined through the following steps: For a first element representing the relationship weight between a candidate item node and a candidate attribute node related to the candidate item, and / or representing the relationship between the target user node and a first candidate attribute node, the value of the first element is determined to be a first preset value representing a positive association, wherein the first candidate attribute node is used to represent the candidate attribute that the target user is asked for and accepted; for a second element representing the relationship between the target user node and a second candidate attribute node, the value of the second element is determined to be a second preset value representing a negative association, wherein the second candidate attribute node is used to represent the candidate attribute that the target user is asked for and rejected; and for a third element representing the relationship weight between the target user node and the candidate item node, the value of the third element is determined to be a matching degree, wherein the matching degree is obtained based on the vector dot product corresponding to the target user and the candidate item, the vector dot product corresponding to the target user and the candidate attribute that was asked for and accepted, and the vector dot product corresponding to the target user and the candidate attribute that was asked for and rejected. The above content can be referenced above. Figure 3 , Figure 4 The corresponding descriptions in the embodiments.

[0177] The aforementioned encoding determination unit 1100 may include a first encoding module 1110 and a second encoding module 1120.

[0178] The first encoding module 1110 is configured to provide the user preference interaction graph to a graph neural network to obtain the current preference encoding of the target user. The operation of the first encoding module 1110 can be referred to above. Figure 3 The operation described in section 310.

[0179] The second encoding module 1120 is configured to determine the current state encoding of the target user based on the target user's current preference encoding. The operation of the second encoding module 1120 can be referred to above. Figure 3 The operation described in section 320.

[0180] In one example, the second encoding module described above can be further configured to: convert the dialogue history into a dialogue history embedding; determine a current dialogue state encoding based on the dialogue history embedding, wherein the current dialogue state encoding is used to characterize whether the user preferences learned based on the dialogue history are sufficiently accurate for the recommended items; determine a candidate space encoding based on the number of elements contained in the candidate information set; and determine the current state encoding of the target user based on the current preference encoding, the dialogue state encoding, and the candidate space encoding. The above content can be referenced above. Figure 5 The corresponding descriptions of 510-540 in the embodiments.

[0181] The behavior prediction unit 1030 is configured to provide the current state code to the behavior prediction model to obtain a predicted behavior. The predicted behavior includes asking the target user for attributes or recommending items. The operation of the behavior prediction unit 1030 can be referred to above. Figure 2 The operation described in section 230.

[0182] In one example, the behavior prediction model described above may include a feedforward neural network and a random resampling model. The behavior prediction unit may be further configured to: provide the current state encoding to the feedforward neural network to obtain initial probabilities for each candidate behavior, wherein candidate behaviors include asking the target user about attributes or recommending items; provide the initial probabilities of each candidate behavior to the random resampling model to obtain transition probabilities for each candidate behavior; and determine the predicted behavior as the candidate behavior with the highest transition probability. The above content can be referenced from the above. Figure 6 The corresponding descriptions of 610-630 in the embodiments.

[0183] Information prediction unit 1040 is configured to provide the current state code to a candidate information determination model to obtain target candidate information matching the predicted behavior. The target candidate information includes candidate attribute information matching the target user determined from the candidate attribute information set matching the predicted behavior, or candidate item information matching the target user determined from the candidate item information set matching the predicted behavior. The operation of information prediction unit 1040 can be referred to above. Figure 2 The operation described in section 240.

[0184] In one example, the aforementioned information prediction unit can be further configured to: in response to the prediction behavior being to recommend items to the target user, perform the following item recommendation steps: determine an acceptance attribute set from the candidate attribute information set based on the dialogue history; determine a set of candidate item information matching the attributes in the acceptance attribute set from the candidate item information set based on the acceptance attribute set; provide the current state code to the candidate information determination model to obtain a recommendation value corresponding to each candidate item information in the candidate item information set; and select target item information from the candidate item information set as target candidate information matching the prediction behavior, wherein the target item information includes candidate item information whose obtained recommendation values ​​meet preset requirements. The above content can be referenced above. Figure 7 The corresponding descriptions of 710-750 in the embodiments.

[0185] In one example, the behavior prediction model and the candidate information determination model can be trained based on a first behavior value function for deciding whether to ask the target user for attributes or recommend items and a second behavior value function for determining target candidate information from the candidate information set, respectively.

[0186] Continue to refer to Figure 12 , Figure 12 A block diagram of yet another example of an interactive device 1200 for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0187] like Figure 12 As shown, the interactive device 1200 for recommending items through dialogue may include an information acquisition unit 1210, an encoding determination unit 1220, a behavior prediction unit 1230, an information prediction unit 1240, an information provision unit 1250, an additional attribute determination unit 1260, and an information update unit 1270.

[0188] Information providing unit 1250 is configured to provide the target item information to the target user. The operation of information providing unit 1250 can be referred to above. Figure 7 The description of the 760's operation.

[0189] The additional attribute determination unit 1260 is configured to determine, in response to the target user rejecting the target item information, additional attributes of the rejected target item information relative to the accepted attribute set. The operation of the additional attribute determination unit 1260 can be referred to above. Figure 7 The operations described for 770 and 771.

[0190] Information update unit 1270 is configured to update the dialogue history and the candidate information set according to the additional attributes. The operation of information update unit 1270 can be referred to above. Figure 7The operation described in section 772.

[0191] The aforementioned information acquisition unit 1210, encoding determination unit 1220, behavior prediction unit 1230, and information prediction unit 1240 can be referenced. Figure 10 The corresponding descriptions of the information acquisition unit 1010, encoding determination unit 1020, behavior prediction unit 1030 and information prediction unit 1040 in the embodiment will not be repeated here.

[0192] Further reference Figure 13 , Figure 13 A block diagram illustrating an example of a model training apparatus 1300 for dialogue-based item recommendation according to an embodiment of this specification is shown.

[0193] like Figure 13 As shown, the model training device 1300 for recommending items in dialogue may include an encoding unit 1310, a first value determination unit 1320, a probability conversion unit 1330, a second value determination unit 1340, and a training unit 1350.

[0194] Encoding unit 1310 is configured to determine the sample current state encoding of the sample user based on the sample user's sample dialogue history and sample candidate information set. The operation of encoding unit 1310 can be referred to above. Figure 8 The operation of 810 is described.

[0195] The first value determination unit 1320 is configured to provide the current state encoding of the sample to the current first behavior value determination model to obtain the first behavior value of the current state of the sample under each alternative behavior. The alternative behaviors include querying the sample user for attributes or recommending items. The operation of the first value determination unit 1320 can be referred to above. Figure 8 The operation of 820 is described.

[0196] The probability transformation unit 1330 is configured to determine the probability of selecting each candidate action for the current state of the sample based on the first action value under each candidate action in the current state of the sample; and to convert the probability of selecting each candidate action for the current state of the sample into the sample transformation probability of selecting each candidate action by sampling from the classification distribution. The operation of the probability transformation unit 1330 can be referred to above. Figure 8 The operation of 830 and 840 is described.

[0197] The second value determination unit 1340 is configured to select the maximum value of the sample transition probability of each candidate behavior based on the obtained current state of the sample, and select a sample candidate item information set that matches the recommended item or a sample candidate attribute information set that matches the query attribute from the sample candidate information set as the sample candidate set; and provide the current state encoding of the sample to the current second behavior value determination model to obtain the second behavior value of the current state of the sample under each candidate information in the sample candidate set. The operation of the second value determination unit 1340 can be referred to above. Figure 8 The operation of 850 and 860 is described.

[0198] Training unit 1350 is configured to adjust the parameters of the current first-behavior value determination model and the current second-behavior value determination model based on a reinforcement learning algorithm, so that the sample user accepts the recommended sample candidate item information. The operation of training unit 1350 can be referred to above. Figure 8 The operation of 870 is described.

[0199] Reference above Figures 1 to 13 This specification describes embodiments of an interaction method and apparatus for recommending items through dialogue, as well as embodiments of a model training method and apparatus for recommending items through dialogue.

[0200] The interactive device and model training device for recommending items through dialogue, as described in the embodiments of this specification, can be implemented in hardware, software, or a combination of hardware and software. Taking software implementation as an example, as a logical device, it is formed by the processor of its host device reading the corresponding computer program instructions from the memory into memory and executing them. In the embodiments of this specification, the interactive device and model training device for recommending items through dialogue can, for example, be implemented using electronic devices.

[0201] Figure 14 A schematic diagram of an interactive device 1400 for recommending items through dialogue, according to an embodiment of this specification, is shown.

[0202] like Figure 14 As shown, the interactive device 1400 for recommending items through dialogue may include at least one processor 1410, a memory (e.g., non-volatile memory) 1420, a RAM 1430, and a communication interface 1440, and the at least one processor 1410, memory 1420, RAM 1430, and communication interface 1440 are connected together via a bus 1450. At least one processor 1410 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0203] In one embodiment, computer-executable instructions are stored in memory that, when executed, cause at least one processor 1410 to: acquire a target user's dialogue history and a candidate information set, wherein the dialogue history includes historical actions in at least one round of dialogue and the target user's feedback to the historical actions, the historical actions including asking about target attributes or recommending target items, and the candidate information set including a candidate attribute information set and a candidate item information set; determine the target user's current state code based on the dialogue history and the candidate information set; provide the current state code to a behavior prediction model to obtain a predicted behavior, wherein the predicted behavior includes asking the target user about attributes or recommending items; and provide the current state code to a candidate information determination model to obtain target candidate information matching the predicted behavior, wherein the target candidate information includes candidate attribute information matching the target user determined from the candidate attribute information set matching the predicted behavior or candidate item information determined from the candidate item information set.

[0204] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1410 to perform the above-described combinations in the various embodiments of this specification. Figure 1-7 The description includes various operations and functions.

[0205] Figure 15 A schematic diagram of a model training device 1500 for recommending items in dialogue, according to an embodiment of this specification, is shown.

[0206] like Figure 15 As shown, the model training device 1500 for recommending items in dialogue may include at least one processor 1510, a memory (e.g., non-volatile memory) 1520, a main memory 1530, and a communication interface 1540, and the at least one processor 1510, memory 1520, main memory 1530, and communication interface 1540 are connected together via a bus 1550. At least one processor 1510 executes at least one computer-readable instruction (i.e., the elements implemented in software above) stored or encoded in the memory.

[0207] In one embodiment, computer-executable instructions are stored in memory, which, when executed, cause at least one processor 1510 to: determine a sample current state code of the sample user based on the sample user's sample dialogue history and a sample candidate information set; provide the sample current state code to a current first behavior value determination model to obtain a first behavior value of the sample current state under each alternative behavior, wherein the alternative behaviors include asking the sample user for attributes or recommending items; determine the probability of the sample current state selecting each alternative behavior based on the first behavior value of the sample current state under each alternative behavior; and convert the probability of the sample current state selecting each alternative behavior into sampling from a classification distribution to obtain... The sample transition probability of each candidate behavior is selected based on the current state of the sample; the maximum value of the sample transition probability of each candidate behavior is selected based on the current state of the sample; a sample candidate item information set matching the recommended item or a sample candidate attribute information set matching the query attribute is selected from the sample candidate information set as the sample candidate set; the current state of the sample is encoded and provided to the current second behavior value determination model to obtain the second behavior value of the current state of the sample under each candidate information in the sample candidate set; the parameters of the current first behavior value determination model and the current second behavior value determination model are adjusted based on the reinforcement learning algorithm so that the sample user accepts the recommended sample candidate item information.

[0208] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1510 to perform the above-described combinations in the various embodiments of this specification. Figure 8 Or the various operations and functions described in 9.

[0209] According to one embodiment, a program product, such as a computer-readable medium, is provided. The computer-readable medium may have instructions (i.e., the elements implemented in software as described above), which, when executed by a computer, cause the computer to perform the above-described combinations of the various embodiments of this specification. Figure 1-9 The description includes various operations and functions.

[0210] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0211] In this case, the program code itself, which can be read from a readable medium, can perform the functions of any of the above embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0212] The computer program code required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB, .NET, and Python; conventional procedural programming languages ​​such as C, Visual Basic 2003, Perl, COBOL 2002, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service, such as Software as a Service (SaaS).

[0213] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0214] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0215] Not all steps and units in the above process and system structure diagrams are mandatory; some steps or units can be omitted as needed. The execution order of each step is not fixed and can be determined as required. The device structure described in the above embodiments can be a physical structure or a logical structure. That is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0216] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" over other embodiments. Detailed descriptions are included for the purpose of providing an understanding of the described techniques. However, these techniques may be practiced without these detailed descriptions. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0217] The optional embodiments of the present specification have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present specification are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present specification, various simple modifications can be made to the technical solutions of the embodiments of the present specification, and these simple modifications all fall within the protection scope of the embodiments of the present specification.

[0218] The foregoing description of this specification is provided to enable any person skilled in the art to implement or use the content of this specification. Various modifications to the content of this specification will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of protection of this specification. Therefore, this specification is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. An interactive method for recommending items through dialogue, comprising: Acquire the target user's dialogue history and candidate information set, wherein the dialogue history includes historical behavior in at least one round of dialogue and the target user's feedback on the historical behavior, the historical behavior includes asking about target attributes or recommending target items, and the candidate information set includes a candidate attribute information set and a candidate item information set; The current state code of the target user is determined based on the dialogue history and the candidate information set; The current state code is provided to the behavior prediction model to obtain the predicted behavior, wherein the predicted behavior includes asking the target user for attributes or recommending items; and The current state code is provided to the candidate information determination model to obtain target candidate information that matches the predicted behavior. The target candidate information includes candidate attribute information that matches the target user determined from the candidate attribute information set that matches the predicted behavior, or candidate item information determined from the candidate item information set. The candidate information set includes a current user preference interaction graph, which comprises a node set and an adjacency matrix. The node set includes target user nodes, a current candidate item node set, and a current candidate attribute node set. The adjacency matrix represents the relationship weights between nodes. The user preference interaction graph is dynamically updated based on the target attributes or target items targeted by the target user's historical behavior of accepting or rejecting corresponding actions. Determining the current state code of the target user based on the dialogue history and the candidate information set includes: The user preference interaction graph is provided to a graph neural network to obtain the current preference code of the target user; and The current state code of the target user is determined based on the target user's current preference code.

2. The interaction method as described in claim 1, wherein, The values ​​of the elements of the adjacency matrix are determined through the following steps: For the first element representing the relationship weight between the candidate item node and the candidate attribute node related to the candidate item and / or representing the relationship between the target user node and the first candidate attribute node, the value of the first element is determined to be a first preset value representing the existence of a positive correlation, wherein the first candidate attribute node is used to represent the candidate attribute that the target user is asked for and accepted. For the second element characterizing the relationship between the target user node and the second candidate attribute node, the value of the second element is determined to be a second preset value characterizing the existence of a negative association, wherein the second candidate attribute node is used to characterize the candidate attributes that the target user is asked about and rejected; and For the third element that characterizes the weight of the relationship between the target user node and the candidate item node, the value of the third element is determined to be the matching degree, wherein the matching degree is obtained based on the vector dot product of the target user and the candidate item, the vector dot product of the target user and the queried and accepted candidate attribute, and the vector dot product of the target user and the queried and rejected candidate attribute.

3. The interaction method as described in claim 1, wherein, Determining the current state code of the target user based on the target user's current preference code includes: Convert the dialogue history into a dialogue history embedding; The current dialogue state code is determined based on the dialogue history embedding, wherein the current dialogue state code is used to characterize whether the user preferences learned based on the dialogue history are sufficiently accurate for the recommended items; Based on the number of elements contained in the candidate information set, the candidate space encoding is determined; and The current state code of the target user is determined based on the current preference code, the dialogue state code, and the candidate space code.

4. The interaction method as described in claim 1, wherein, The step of providing the current state encoding to the candidate information determination model to obtain target candidate information that matches the prediction behavior includes: In response to the prediction action of recommending items to the target user, the following item recommendation steps are performed: The set of accepted attributes is determined from the candidate attribute information set based on the dialogue history; Based on the set of accepted attributes, a set of alternative item information that matches the attributes in the set of accepted attributes is determined from the set of candidate item information. The current state code is provided to the candidate information determination model to obtain the recommended value corresponding to each candidate item in the candidate item information set; and Target item information is selected from the set of candidate item information as target candidate information that matches the prediction behavior, wherein the target item information includes candidate item information whose obtained recommendation values ​​meet preset requirements.

5. The interaction method as described in claim 4, wherein, The method further includes: Provide the target item information to the target user; In response to the target user rejecting the target item information, determine the additional attributes of the rejected target item information relative to the accepted attribute set, and Update the dialogue history and the candidate information set based on the additional attributes.

6. The interaction method as described in claim 1, wherein, The behavior prediction model and the candidate information determination model are respectively trained based on a first behavior value function for deciding whether to ask the target user for attributes or recommend items and a second behavior value function for determining target candidate information from the candidate information set.

7. The interaction method as described in any one of claims 1 to 6, wherein, The behavior prediction model includes a feedforward neural network and a random resampling model. The step of providing the current state code to the behavior prediction model to obtain the predicted behavior includes: The current state encoding is provided to the feedforward neural network to obtain the initial probability of each alternative behavior, wherein the alternative behavior includes asking the target user for attributes or recommending items. The initial probabilities of each candidate action are provided to the random resampling model to obtain the transition probabilities of each candidate action; and The predicted behavior is selected as the alternative behavior with the highest conversion probability.

8. A model training method for recommending items in dialogue, comprising: The sample current state code of the sample user is determined based on the sample user's sample dialogue history and sample candidate information set; The current state of the sample is encoded and provided to the current first behavior value determination model to obtain the first behavior value of the current state of the sample under each alternative behavior, wherein the alternative behaviors include asking the sample user for attributes or recommending items. Based on the value of the first action of the current state of the sample under each alternative action, determine the probability of the current state of the sample choosing each alternative action; The probability of a sample choosing each alternative behavior in its current state is converted into the sample transition probability of choosing each alternative behavior in its current state by sampling from the classification distribution. Based on the current state of the obtained samples, select the maximum value of the sample conversion probability of each alternative behavior, and select the sample candidate item information set that matches the recommended item or the sample candidate attribute information set that matches the query attribute from the sample candidate information set as the sample candidate set. The current state encoding of the sample is provided to the current second-row value determination model to obtain the second-row value of the current state of the sample under each candidate information in the sample candidate set; and The parameters of the current first-row value determination model and the current second-row value determination model are adjusted based on the reinforcement learning algorithm so that the sample user accepts the recommended sample candidate item information.

9. An interactive device for recommending items through dialogue, comprising: An information acquisition unit is configured to acquire the dialogue history and candidate information set of a target user. The dialogue history includes historical behaviors in at least one round of dialogue and the target user's feedback on the historical behaviors. The historical behaviors include asking about target attributes or recommending target items. The candidate information set includes a candidate attribute information set and a candidate item information set. The candidate information set includes a current user preference interaction graph. The user preference interaction graph includes a node set and an adjacency matrix. The node set includes target user nodes, a current candidate item node set, and a current candidate attribute node set. The adjacency matrix is ​​used to represent the relationship weights between nodes. The user preference interaction graph is dynamically updated based on whether the target user accepts or rejects the target attribute or target item targeted by the corresponding historical behavior. The encoding determination unit is configured to determine the current state encoding of the target user based on the dialogue history and the candidate information set; A behavior prediction unit is configured to provide the current state code to a behavior prediction model to obtain a predicted behavior, wherein the predicted behavior includes asking the target user for attributes or recommending items; and An information prediction unit is configured to provide the current state code to a candidate information determination model to obtain target candidate information that matches the prediction behavior, wherein the target candidate information includes candidate attribute information that matches the target user determined from the candidate attribute information set that matches the prediction behavior, or candidate item information determined from the candidate item information set. The encoding determination unit is further configured to provide the user preference interaction graph to the graph neural network to obtain the current preference encoding of the target user; and to determine the current state encoding of the target user based on the current preference encoding of the target user.

10. A model training device for recommending items in dialogue, comprising: The encoding unit is configured to determine the sample current state encoding of the sample user based on the sample user's sample dialogue history and sample candidate information set; The first value determination unit is configured to provide the current state encoding of the sample to the current first behavior value determination model to obtain the first behavior value of the current state of the sample under each alternative behavior, wherein the alternative behaviors include asking the sample user for attributes or recommending items. The probability transformation unit is configured to determine the probability of selecting each alternative behavior in the current state of the sample based on the first behavior value of the current state of the sample under each alternative behavior; and to transform the probability of selecting each alternative behavior in the current state of the sample into the sample transformation probability of selecting each alternative behavior by sampling from the classification distribution. The second value determination unit is configured to select the maximum value of the sample transition probability of each candidate behavior based on the obtained current state of the sample; select a sample candidate item information set that matches the recommended item or a sample candidate attribute information set that matches the query attribute from the sample candidate information set as the sample candidate set; provide the current state encoding of the sample to the current second behavior value determination model to obtain the second behavior value of the current state of the sample under each candidate information in the sample candidate set; and The training unit is configured to adjust the parameters of the current first-behavior value determination model and the current second-behavior value determination model based on a reinforcement learning algorithm, so that the sample user accepts the recommended sample candidate item information.

11. An interactive device for recommending items through dialogue, comprising: At least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the interaction method as described in any one of claims 1 to 7.

12. A model training device for recommending items in dialogue, comprising: At least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the model training method as described in claim 8.

13. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the interaction method as described in any one of claims 1 to 7 or the model training method as described in claim 8.

14. A computer program product comprising a computer program that is executed by a processor to implement the interactive method as described in any one of claims 1 to 7 or the model training method as described in claim 8.

Images