Product shopping guide processing method, device, electronic equipment, medium and program product

Abstract

The disclosed embodiments provide a product shopping guide processing method, device, electronic device, medium and program product. The method includes: obtaining the current round of dialogue information of the user; using a dialogue state tracking model to process the current round of dialogue information to obtain the current dialogue state, the current dialogue state includes the current probability distribution of each product attribute in multiple product attributes, each current probability distribution is used to characterize the current probability of each product attribute value in one or more product attribute values ​​corresponding to the product attribute; using a strategy decision model to process the current dialogue state to obtain the current response action, the current response action includes a response action for continuing the dialogue or a response action for product recommendation, the strategy decision model and the dialogue state tracking model are jointly trained and generated using a first training sample, the first training sample includes a multi-round dialogue information sample of each first sample user in multiple first sample users; executing the current response action to obtain a product shopping guide processing result.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more specifically, to a product shopping guide processing method, device, electronic device, medium, and program product. Background Art

[0002] With the continuous development of computer technology, products are recommended to users through product shopping guides to improve sales conversion rates.

[0003] In the related art, product shopping guides are usually implemented by using a search-based dialogue method or a recommendation method. Among them, the search-based dialogue method predicts the user's purchase intention through multiple rounds of dialogue information with the user, and recommends products to the user based on the predicted purchase intention. The recommendation-based method recommends products to the user based on the user's historical behavior records.

[0004] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: the sales conversion rate of product shopping guides implemented by using the related art is not high. Summary of the invention

[0005] In view of this, the embodiments of the present disclosure provide a product shopping guide processing method, device, electronic device, medium and program product.

[0006] One aspect of an embodiment of the present disclosure provides a product shopping guide processing method, including: obtaining a user's current round of dialogue information; using a dialogue state tracking model to process the current round of dialogue information to obtain a current dialogue state, wherein the current dialogue state includes a current probability distribution of each product attribute among multiple product attributes, wherein each of the current probability distributions is used to characterize the current probability of each of one or more product attribute values ​​corresponding to the product attribute; using a strategy decision model to process the current dialogue state to obtain a current response action, wherein the current response action includes a response action for continuing the dialogue or a response action for product recommendation, wherein the strategy decision model and the dialogue state tracking model are jointly trained and generated using a first training sample, wherein the first training sample includes a multi-round dialogue information sample of each first sample user among multiple first sample users; and executing the current response action to obtain a product shopping guide processing result.

[0007] Another aspect of an embodiment of the present disclosure provides a product shopping guide processing device, including: an acquisition module, used to acquire a user's current round of dialogue information; a first processing module, used to use a dialogue state tracking model to process the above-mentioned current round of dialogue information to obtain a current dialogue state, wherein the above-mentioned current dialogue state includes a current probability distribution of each product attribute among multiple product attributes, wherein each of the above-mentioned current probability distributions is used to characterize the probability of each of the above-mentioned product attribute values ​​in one or more product attribute values ​​corresponding to the above-mentioned product attribute; a second processing module, used to use a policy decision model to process the above-mentioned current dialogue state to obtain a current response action, wherein the above-mentioned current response action includes a response action for continuing the dialogue or a response action for product recommendation, wherein the above-mentioned policy decision model and the above-mentioned dialogue state tracking model are jointly trained and generated using a first training sample, wherein the above-mentioned first training sample includes a multi-round dialogue information sample of each first sample user among multiple first sample users; and an execution module, used to execute the above-mentioned current response action to obtain a product shopping guide processing result.

[0008] Another aspect of an embodiment of the present disclosure provides an electronic device, including: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0009] Another aspect of an embodiment of the present disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method described above.

[0010] Another aspect of an embodiment of the present disclosure provides a computer program product, the computer program product comprising a computer program, and the computer program is used to implement the above-mentioned method when executed by a processor.

[0011] According to an embodiment of the present disclosure, by obtaining the current round of dialogue information of the user, the current round of dialogue information is processed by a dialogue state tracking model to obtain the current dialogue state, the current dialogue state includes the current probability distribution of each product attribute in a plurality of product attributes, the current dialogue state is processed by a policy decision model to obtain the current response action, the current response action includes a response action for continuing the dialogue or a response action for product recommendation, the policy decision model and the dialogue state tracking model are jointly trained and generated using the first training sample, and the current response action is executed to obtain the product shopping guide processing result. Since the current dialogue state obtained based on the dialogue state tracking model is processed by the policy decision model, it can be determined whether the current response action is a response action that requires continued communication with the user to clarify the user's needs, or a response action for product recommendation, thereby achieving the recommendation of appropriate products to the user at the appropriate time, thereby at least partially overcoming the technical problem of low sales conversion rate of product shopping guides using related technologies, and improving the sales conversion rate of product shopping guides. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0013] Figure 1 An exemplary system architecture to which the product shopping guide method according to an embodiment of the present disclosure can be applied is schematically shown;

[0014] Figure 2 A flowchart of a product shopping guide processing method according to an embodiment of the present disclosure is schematically shown;

[0015] Figure 3 A schematic diagram schematically shows a current round of dialogue information according to an embodiment of the present disclosure;

[0016] Figure 4 A schematic diagram schematically shows a response action for continuing a conversation according to an embodiment of the present disclosure;

[0017] Figure 5 A schematic diagram schematically shows a response action for product recommendation according to an embodiment of the present disclosure;

[0018] Figure 6 A flowchart schematically shows another product shopping guide processing method according to an embodiment of the present disclosure;

[0019] Figure 7 The following schematically shows an application diagram of a product shopping guide processing method according to an embodiment of the present disclosure;

[0020] Figure 8 A block diagram schematically shows a product shopping guide processing device according to an embodiment of the present disclosure; and

[0021] Fig. 9 A block diagram of a method for implementing product shopping guide processing according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0024] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0025] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0026] In the related art, the inventors found that because the search dialogue method does not take into account the user's historical behavior records, which are usually used to discover the user's potential interests, some products with high potential interests may be missed, thereby reducing the sales conversion rate. Because the recommendation method cannot obtain the user's current demand information, the recommended products may not meet the user's needs, thereby reducing the sales conversion rate.

[0027] In the process of implementing the concept of the present disclosure, the inventors found that the way to improve the sales conversion rate is to recommend suitable products to users at the right time. Here, recommending products to users at the right time means recommending products after fully understanding the current demand information of users. Recommending suitable products means recommending products that meet the current demand information of users and also include the potential interests of users. The inventors also discovered that the user's current demand information can be obtained based on the search dialogue method, and the user's potential interests can be obtained based on the recommendation method. Therefore, the two methods can be used in combination, but it is also found that directly combining the two methods cannot solve the problem. This is because the premise of recommending suitable products to users is to obtain sufficient current demand information that can be used to recommend products to users. If sufficient current demand information has not been obtained, products are recommended to users, and the recommended products may not meet the user's actual needs. On the contrary, if sufficient current demand information has been obtained, products can be recommended to users. Instead of making product recommendations, continuing to communicate with users to continue to obtain current demand information will cause unnecessary resource consumption and may also cause user disgust. Therefore, in the dialogue with the user, at the current time node, there is a question of whether it is necessary to continue to communicate with the user based on the search dialogue method to obtain the user's current demand information to further clarify the user's current needs, or whether it is necessary to recommend suitable products to users based on the recommendation method.

[0028] In order to solve the problems existing in the process of combining the two methods, the inventor proposes a technical solution that combines the two implementation methods based on a strategy decision model, that is, the embodiment of the present disclosure provides a product shopping guide processing method, a product shopping guide processing device, and an electronic device that can apply the method. The method includes a prediction process and an execution process. In the prediction process, the user's current round of dialogue information is obtained, and the current round of dialogue information is processed by a dialogue state tracking module to obtain the current dialogue state. The current dialogue state includes the current probability distribution of each product attribute in multiple product attributes, and each current probability distribution is used to characterize the current probability of each product attribute value in one or more product attribute values ​​corresponding to the product attribute. The current dialogue state is processed by the strategy decision module to obtain the current response action, and the current response action includes a response action for continuing the dialogue or a response action for product recommendation. The strategy decision model and the dialogue state tracking model are jointly trained and generated using the first training sample, and the first training sample includes a multi-round dialogue information sample of each first sample user in multiple first sample users. In the execution process, the current response action is executed to obtain the product shopping guide processing result.

[0029] Figure 1 The exemplary system architecture 100 to which the product shopping guide method according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0030] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0031] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0032] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0033] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received shopping guide dialogue request of the user, and feed back the current response action to the terminal device.

[0034] It should be noted that the product shopping guide processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the product shopping guide processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The product shopping guide processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Correspondingly, the product shopping guide processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Alternatively, the product shopping guide processing method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or can also be executed by other terminal devices different from the terminal devices 101, 102, or 103. Accordingly, the product shopping guide processing device provided in the embodiment of the present disclosure may also be disposed in the terminal device 101 , 102 , or 103 , or in other terminal devices different from the terminal device 101 , 102 , or 103 .

[0035] For example, the dialogue state tracking model and the policy decision model may be originally stored in any one of the terminal devices 101, 102, or 103 (for example, the terminal device 101, but not limited thereto), or stored in an external storage device and may be imported into the terminal device 101. Then, the terminal device 101 may locally execute the product shopping guide processing method provided by the embodiment of the present disclosure, or send the dialogue state tracking model and the policy decision model to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the dialogue state tracking model and the policy decision model execute the product shopping guide processing method provided by the embodiment of the present disclosure.

[0036] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0037] Figure 2 The following schematically shows a flow chart of a product shopping guide processing method according to an embodiment of the present disclosure.

[0038] like Figure 2As shown, the method includes operations S210 to S240.

[0039] In operation S210, the user's current turn dialogue information is obtained.

[0040] According to an embodiment of the present disclosure, current round dialogue information generated by a dialogue with a user can be obtained, wherein the user can conduct a dialogue in at least one of voice, text, and image. The current round dialogue information may include multiple product attributes for a product, and the current round dialogue information may include multiple groups of dialogues.

[0041] For example, if a user enters the dialogue interface by triggering a dialogue request control on an application and enters product demand information for a product in a text box, the user's current round of dialogue information is generated in response to the product demand information entered by the user. Figure 3 A schematic diagram of current round of dialogue information according to an embodiment of the present disclosure is schematically shown. Figure 3 The user needs to obtain product recommendations for soybean milk machines. The user's current round of dialogue information may include two groups of dialogues with the user.

[0042] In operation S220, the current round of dialogue information is processed using a dialogue state tracking model to obtain a current dialogue state, wherein the current dialogue state includes a current probability distribution of each product attribute in a plurality of product attributes, wherein each current probability distribution is used to characterize a current probability of each product attribute value in one or more product attribute values ​​corresponding to the product attribute.

[0043] According to an embodiment of the present disclosure, the dialogue state tracking model can be generated based on the training of the recurrent neural network model. Since the current round of dialogue information can include multiple product attributes, each product attribute can have a corresponding recurrent neural network model.

[0044] After obtaining the user's current round of dialogue information, the current round of dialogue information can be processed using the dialogue state tracking model to obtain the current round of dialogue state. The current round of dialogue state can include the current probability distribution of each product attribute among multiple product attributes, each product attribute can correspond to one or more product attribute values, each product attribute value has a corresponding current probability, and the probability of the product attribute value can represent the probability of the product attribute value being the product attribute value.

[0045] For example, if the product attribute is color, the product attribute values ​​corresponding to color include red, yellow, and blue. The current probability corresponding to red is 0.3, the current probability corresponding to yellow is 0.1, and the current probability corresponding to blue is 0.6. The current probability distribution of color is [0.3 0.1 0.6].

[0046] In operation S230, the current dialogue state is processed using the policy decision model to obtain a current response action, wherein the current response action includes a response action for continuing the dialogue or a response action for product recommendation, wherein the policy decision model and the dialogue state tracking model are jointly trained and generated using a first training sample, wherein the first training sample includes multiple rounds of dialogue information samples of each first sample user among multiple first sample users.

[0047] In operation S240, the current response action is executed to obtain a product shopping guide processing result.

[0048] According to an embodiment of the present disclosure, the policy decision model can be used to determine the current response action according to the current dialogue state. The policy decision model and the dialogue state tracking model can be generated by joint training using the first training sample, that is, the conditions for completing the training of the policy decision model and the dialogue state tracking model are the same. The policy decision model can be a policy network model.

[0049] According to an embodiment of the present disclosure, a response action for continuing a conversation can represent the need to continue a conversation with the user to further clarify the user's current demand information. A response action for product recommendation can represent the recommendation of a product to the user. Among them, the current vector can be obtained by processing the current conversation state using a policy decision model, and the current vector includes a recommendation probability and a predicted probability of each product attribute in a plurality of product attributes, and the current response action is determined based on the current vector, wherein the recommendation probability can be used to represent the probability of making a product recommendation. Determining the current response action based on the current vector may include determining the maximum probability based on the recommendation probability and a plurality of predicted probabilities, and if the maximum probability is the predicted probability, the current response action may be determined as a response action for continuing the conversation. If the maximum probability is the recommended probability, the current response action may be determined as a response action for product recommendation.

[0050] According to an embodiment of the present disclosure, if the current response action is a response action for continuing the conversation, the product attribute corresponding to the maximum probability can be determined as the target product attribute, and the target product attribute can be determined as the reply information for continuing the conversation with the user. If the current response action is a response action for product recommendation, the user's feature information and the candidate product's feature information can be obtained, and the product recommendation model is used to process the user's feature information, the candidate product's feature information and the current conversation state to obtain a recommended product.

[0051] According to the technical solution of the embodiment of the present disclosure, by obtaining the current round of dialogue information of the user, the current round of dialogue information is processed by the dialogue state tracking model to obtain the current dialogue state, the current dialogue state includes the current probability distribution of each product attribute in multiple product attributes, the current dialogue state is processed by the policy decision model to obtain the current response action, the current response action includes a response action for continuing the dialogue or a response action for product recommendation, the policy decision model and the dialogue state tracking model are jointly trained and generated using the first training sample, and the current response action is executed to obtain the product shopping guide processing result. Since the current dialogue state obtained based on the dialogue state tracking model is processed by the policy decision model, it can be determined whether the current response action is a response action that requires continued communication with the user to clarify the user's needs, or a response action for product recommendation, so as to achieve the recommendation of suitable products to the user at the right time, thereby at least partially overcoming the technical problem that the sales conversion rate of product shopping guides using related technologies is not high, and the sales conversion rate of product shopping guides is improved.

[0052] According to an embodiment of the present disclosure, using a policy decision model to process the current dialog state to obtain a current response action may include the following operations.

[0053] The current dialog state is processed using the policy decision model to obtain a current vector, wherein the current vector includes the current recommendation probability and the current predicted probability of each of the multiple product attributes. According to the current vector, the current response action is determined.

[0054] According to an embodiment of the present disclosure, the current recommendation probability can be used to characterize the probability of making a product recommendation in the current round. The current prediction probability of each product attribute can be used to characterize the prediction probability of the product attribute in the current round. The maximum probability can be determined based on the current recommendation probability and multiple current prediction probabilities, and the current response action can be determined based on the maximum probability.

[0055] For example, if the number of product attributes includes L, the current prediction probability of each product attribute can be expressed as B j , where j∈{1, 2, ..., L-1, L}. The current recommendation probability can be expressed as B0. The current vector can be expressed as [B0, B1, B2, ..., B L-1 , B L ].

[0056] According to an embodiment of the present disclosure, determining a current response action according to a current vector may include the following operations.

[0057] The maximum probability is determined from the current recommendation probability and the multiple current prediction probabilities. When the maximum probability is determined to be the current prediction probability, the current response action is determined to be a response action for continuing the conversation. When the maximum probability is determined to be the current recommendation probability, the current response action is determined to be a response action for product recommendation.

[0058] According to an embodiment of the present disclosure, the current recommendation probability is compared with the current predicted probability of each product attribute in a plurality of product attributes to determine the maximum probability. If the maximum probability is the current recommendation probability, it can be determined that the current response action is the response action for product recommendation. If the maximum probability is a current predicted probability in a plurality of current predicted probabilities, it can be determined that the current response action is the response action for continuing the conversation.

[0059] According to an embodiment of the present disclosure, if the number of maximum probabilities is greater than one, it can be determined according to actual conditions. For example, if the maximum probability includes one or more of the current recommendation probability and multiple current predicted probabilities, the response action recommended by the product can be determined as the current response action.

[0060] According to an embodiment of the present disclosure, the current response action includes a response action for continuing the conversation. Executing the response action to obtain a product shopping guide processing result may include the following operations.

[0061] Determine a target product attribute, wherein the target product attribute is a product attribute corresponding to the maximum probability. Determine the target product attribute as reply information for continuing the conversation with the user.

[0062] According to an embodiment of the present disclosure, if the maximum probability is the current predicted probability, the product attribute corresponding to the current predicted probability can be determined as the target product attribute, and the target product attribute can be used as reply information to continue the conversation with the user, that is, the information replied to the user is the target product attribute. Figure 4 A schematic diagram of a response action for continuing a conversation according to an embodiment of the present disclosure is schematically shown. Figure 3 On the basis of Figure 4 The target product attribute determined in the analysis is wash-free. Based on the target product attribute, the reply message for continuing the conversation with the user is determined to be "Do you need wash-free?"

[0063] According to an embodiment of the present disclosure, if the maximum probability is the current prediction probability, and the number of current prediction probabilities that are the maximum probability is greater than one, then any current prediction probability can be selected from multiple current prediction probabilities, and the product attribute corresponding to the current prediction probability can be used as the target product attribute.

[0064] According to an embodiment of the present disclosure, the current response action includes a response action for product recommendation.

[0065] Executing a response action to obtain a product shopping guide processing result may include the following operations.

[0066] Acquire the user's characteristic information and the candidate product's characteristic information. Use the product recommendation model to process the user's characteristic information, the candidate product's characteristic information and the current conversation state to obtain a recommended product, wherein the product recommendation model is generated by training using the second training sample, wherein the second training sample includes characteristic information of each second sample user among multiple second sample users, a conversation state sample and a real interest product, and characteristic information of each sample product among multiple sample products.

[0067] According to an embodiment of the present disclosure, if the current response action is a response action for product recommendation, the user's feature information and the candidate product's feature information can be obtained, the user's feature information, the candidate product's feature information and the current dialogue state are vectorized to obtain the user's feature vector, the candidate product's feature vector and the current dialogue state vector, and the user's feature vector, the candidate product's feature vector and the current dialogue state vector are concatenated to obtain a target feature vector. The target feature vector is processed using a product recommendation model to obtain a recommended product. Figure 5 A schematic diagram of a response action for product recommendation according to an embodiment of the present disclosure is schematically shown. Figure 3 On the basis of Figure 5 Product recommendations are made by sending users web links to related products.

[0068] According to an embodiment of the present disclosure, the product recommendation model is generated by training with the second training sample, and may include the following operations.

[0069] The factorization machine model is trained using the second training sample to obtain a product recommendation model.

[0070] According to an embodiment of the present disclosure, obtaining a conversation state sample of each second sample user among multiple second sample users may include: for each second sample user, obtaining a conversation information sample corresponding to the second sample user, inputting the conversation information sample into a conversation state tracking model, and outputting a conversation state sample, wherein the conversation state sample includes a probability distribution of each product attribute among multiple product attributes.

[0071] According to an embodiment of the present disclosure, the feature information of each second sample user and the feature information of each sample product are vectorized to obtain a feature vector of each second sample user and a product feature vector of each sample product. The feature vector of each second sample user, the feature information of each sample product and the conversation state sample of each second sample user are spliced ​​to obtain multiple training feature vectors, and the Factorization Machine (FM) model is trained using multiple training feature vectors to obtain a product recommendation model. That is, multiple training feature vectors are input into the Factorization Machine model, and the predicted interest products of each second sample user are output. The predicted interest products and the real interest products of each second sample user are input into the loss function of the Factorization Machine model, and the output value of the loss function is output. The model parameters of the Factorization Machine model are adjusted according to the output value of the loss function until the loss function converges, and the trained Factorization Machine model is used as the product recommendation model. Among them, the Factorization Machine model can be trained by the stochastic gradient descent method, the alternating least squares method or the Markov chain Monte Carlo inference method.

[0072] For example, the second training sample includes U second sample users and I sample products. The feature vector of the second sample user m can be expressed as u m ={0, 0, ..., 1, ..., 0}, where u m The dimension corresponding to the second sample user m is represented by 1. The feature vector of sample product n can be represented as i n ={0, 0, ..., 1, ..., 0}, where i n The dimension corresponding to the sample product n is represented by 1. The second sample user m’s dialogue state sample in round t can be represented by s tm Represents. Training feature vector x mnt It can be expressed as

[0073] According to an embodiment of the present disclosure, the policy decision model and the dialog state tracking model are jointly trained and generated using the first training sample, which may include the following operations.

[0074] For each first sample user, each round of dialogue information samples in the multiple rounds of dialogue information samples is input into multiple recurrent neural network models respectively, and the predicted probability distribution of each product attribute in the multiple product attributes is output, wherein each recurrent neural network model is used to output the predicted probability distribution of the corresponding product attribute. For each round of dialogue information sample, the current round dialogue state is generated according to the predicted probability distribution of each product attribute in the multiple product attributes. The current round dialogue state is input into the policy network model to output the current round vector corresponding to the current round dialogue, wherein the current round vector includes the current round recommendation probability and the current round prediction probability of each product attribute in the multiple product attributes. According to the current round vector, the response action corresponding to the current round dialogue is determined. The response action corresponding to the current round dialogue is executed to obtain the user's feedback result for the response action, wherein the response action corresponding to the current round dialogue includes a response action for continuing the dialogue or a response action for product recommendation. A feedback score corresponding to the feedback result is determined. The response probability and feedback score of the multiple rounds of dialogue are input into the objective function, and the output value of the objective function is output, wherein the response probability is the probability corresponding to the response action, and the response probability is the current round recommendation probability or the current round prediction probability. The network parameters of multiple recurrent neural network models and policy network models are adjusted according to the output values ​​until the objective function converges. The multiple recurrent neural network models obtained after training are used as the dialogue state tracking model, and the policy network model obtained after training is used as the policy decision model.

[0075] According to an embodiment of the present disclosure, a product shopping guide based on a search dialogue mode and a product shopping guide based on a recommendation mode can be combined through a policy decision model, wherein the policy decision model and the dialogue state tracking model can be obtained by training a policy network model and multiple recurrent neural network models using reinforcement learning, and the outputs of the multiple recurrent neural network models are the inputs of the training policy network model. The recurrent neural network model can include a long short-term memory neural network model.

[0076] According to an embodiment of the present disclosure, the elements of standard reinforcement learning may include actions, states, rewards, and strategies. Reinforcement learning is an unsupervised environment in which an agent interacts with the environment. Each time the agent performs an action, it receives feedback from the environment, which is the state and reward corresponding to the action. The goal of reinforcement learning is to seek an optimal strategy in the environment through continuous updating of actions and states to accumulate the maximum reward, that is, the output value of the objective function converges, that is, the output value of the objective function is maximized. Reinforcement learning may include a policy gradient algorithm.

[0077] According to the embodiments of the present disclosure, the policy decision model and the dialogue state tracking model can be obtained by training the policy network model and multiple recurrent neural network models using the policy gradient algorithm. The policy gradient algorithm directly selects actions based on the current state, which is a round update, that is, the state, action and reward of each step are recorded in the round. After a round is completed, the policy network model predicts the action of each state in the round, and performs cross entropy with the executed actions. If an action gets more rewards, the probability of the action increasing, and if an action gets less rewards, the probability of the action decreasing.

[0078] The policy network model may be a neural network model including two fully connected layers, the input of the neural network model is the current round of dialogue state, that is, the output of the dialogue state tracking model including multiple long short-term memory neural network models, and the output of the neural network model is the current round vector, that is, the current round recommendation probability and the current round prediction probability of each product attribute in multiple product attributes. Usually, the neural network model updates the network parameters according to the loss function, and strictly speaking, the policy gradient algorithm is error-free because its output is only the probability of selection. In order to update the network parameters, it can be implemented according to the output value of the objective function, where the objective function can be understood as the expectation of the cumulative reward from the initial round of dialogue to the current round of dialogue. The input parameters of the objective function may include the response probability and the feedback score. The feedback score may be a score corresponding to the feedback result, and the feedback result may be the feedback result of the user for the response action obtained by executing the response action corresponding to the current round of dialogue, and the response action may be determined according to the current round vector. The response action can be determined based on the current round vector, and may include determining the maximum probability from the current round recommendation probability and multiple current round prediction probabilities. When the maximum probability is determined to be the current round prediction probability, the current round response action is determined to be a response action for continuing the conversation. When the maximum probability is determined to be the current round recommendation probability, the current round response action is determined to be a response action for product recommendation.

[0079] After obtaining the response probability and feedback score of each round of dialogue in multiple rounds of dialogue from the initial round to the current round, multiple response probabilities and multiple feedback scores can be input into the objective function to output the output value of the objective function. The response probability and feedback score of each round of dialogue in the multiple rounds of dialogue are multiplied to obtain the product result of this round, and multiple product results of this round are added to obtain the output value of the objective function. On this basis, since the farther the reward is, the less influence it has on the current state, a discount parameter can be added to affect the attenuation. Therefore, the product result of this round can be determined as follows: multiply the response probability, feedback score and discount factor of each round of dialogue in the multiple rounds of dialogue to obtain the product result of this round. After obtaining the output value of the objective function, the network parameters of multiple recurrent neural network models and policy network models can be adjusted according to the output value until the objective function converges, and the multiple recurrent neural network models obtained after training are used as dialogue state tracking models, and the neural network model obtained after training is used as a policy decision model.

[0080] According to an embodiment of the present disclosure, the current round vector of the tth round can be determined by the following formula (1).

[0081]

[0082] Among them, S t represents the current round vector of round t. jt represents the probability distribution of product attribute j in the dialogue state of the tth round, j∈{1, 2, ..., L-1, L}, and L represents the number of product attributes.

[0083] The objective function can be determined by the following formula (2).

[0084]

[0085] Among them, η(θ) represents the objective function, θ represents the learning parameters, that is, the network parameters of the neural network model and multiple long short-term memory neural network models. π represents the strategy, t represents the round, T represents the total number of rounds, t∈{1,2,......,T-1,T}, A t represents the action in round t, S t Represents the state of round t. t represents the feedback score of the tth time. J represents the discount parameter, J∈[0,1]. Indicates that in state S t Next, according to the strategy π(A t |S t ) determines the response probability of the response action in round t.

[0086] According to the embodiments of the present disclosure, the strategy decision model obtained by utilizing reinforcement learning training can combine the product shopping guides based on the search dialogue method and the product shopping guides based on the recommendation method, thereby solving the problem of whether it is necessary to continue to communicate with the user based on the search dialogue method to obtain the user's current demand information to further clarify the user's current needs at the current time node, or whether it is necessary to recommend suitable products to the user based on the recommendation method.

[0087] According to an embodiment of the present disclosure, determining a feedback score corresponding to a feedback result may include the following operations.

[0088] In the case where the feedback result is determined to be the purchase of the recommended product, the feedback score corresponding to the feedback result is determined to be the first reward score. In the case where the feedback result is determined to be browsing the recommended product and not purchasing the product, the feedback score corresponding to the feedback result is determined to be the second reward score, wherein the first reward score is greater than the second reward score. In the case where the feedback result is determined to be continuing the conversation or not browsing the recommended product, the feedback score corresponding to the feedback result is determined to be the first penalty score. In the case where the feedback result is determined to be terminating the conversation, the feedback score corresponding to the feedback result is determined to be the second penalty score, wherein the first penalty score is greater than the second penalty score.

[0089] According to an embodiment of the present disclosure, determining a corresponding feedback score according to a feedback result can be divided into the following three situations.

[0090] Case 1: Incentives under recommendations. If after executing this round of response actions, the user's feedback result for this round of response actions is to purchase the recommended product or browse the recommended product, it can be considered that this round of response actions has received good feedback from the user, and the model can be motivated through reinforcement learning. If it is determined that the feedback result is to purchase the recommended product, the feedback score corresponding to the feedback result can be determined as the first reward score. If it is determined that the feedback result is to browse the recommended product and not purchase the product, the feedback score corresponding to the feedback result can be determined as the second reward score, where the reward score can be a positive number and the first reward score is greater than the second reward score. For example, if in the tth round of dialogue, the user purchases the recommended product, the first reward score can be R t = 5. In the tth round of conversation, if the user browses but does not buy the recommended product, the second reward score can be R t =1.

[0091] Case 2: Penalty when the number of conversation rounds increases. If the feedback result is to continue the conversation, it can be said that there are currently no recommended products, which increases the number of conversation rounds. In this case, it can be considered that in the current round of conversation, the system did not clearly define the user's demand for the product, and the model will be punished through reinforcement learning. Alternatively, if there are currently recommended products and the feedback result is that the recommended products have not been browsed, it can also be considered that in the current round of conversation, the system did not clearly define the user's demand for the product, and the model will be punished through reinforcement learning. Exemplarily, if the current round is t, t=5 and the system has no recommended products, or the system has recommended products and the user has not browsed the recommended products, then the first penalty score can be R t =1.

[0092] Case 3: Penalty for terminating the conversation. If the feedback result is to terminate the conversation, it can be considered that the user has a bad experience with the system, and the model will be punished through reinforcement learning. For example, if in the tth round of conversation, the user actively terminates the conversation, then the second penalty score can be R t =3.

[0093] Figure 6 The flowchart schematically shows another product shopping guide processing method according to an embodiment of the present disclosure.

[0094] like Figure 6 As shown, the method includes operations S601 to S611.

[0095] In operation S601, the user's current turn dialogue information is obtained.

[0096] In operation S602, the current round of dialogue information is processed using a dialogue state tracking model to obtain a current dialogue state.

[0097] In operation S603, the current dialog state is processed using the policy decision model to obtain a current vector, wherein the current vector includes a current recommendation probability and a current predicted probability of each of the multiple product attributes.

[0098] In operation S604, a maximum probability is determined from the current recommendation probability and a plurality of current prediction probabilities.

[0099] In operation S605, it is determined whether the maximum probability is the recommendation probability; if so, operation S606 is performed; if not, operation S607 is performed.

[0100] In operation S606, it is determined that the current response action is a response action for product recommendation, and operation S608 is performed.

[0101] In operation S607, it is determined that the current response action is a response action for continuing the conversation, and operation S610 is performed.

[0102] In operation S608, characteristic information of the user and characteristic information of the candidate products are acquired.

[0103] In operation S609, the feature information of the user, the feature information of the candidate products, and the current dialog state are processed using the product recommendation model to obtain a recommended product.

[0104] In operation S610 , a target product attribute is determined, wherein the target product attribute is a product attribute corresponding to a maximum probability.

[0105] In operation S611, the target product attribute is determined as reply information for continuing the dialogue with the user.

[0106] According to an embodiment of the present disclosure, Figure 7 The application diagram of a product shopping guide processing method according to an embodiment of the present disclosure is schematically shown.

[0107] Figure 7 The output of the dialogue state tracking model can be used as the input of the policy decision model and the product recommendation model. The dialogue state tracking model can include k long short-term memory neural network models, and k can be a positive integer.

[0108] According to the technical solution of the embodiment of the present disclosure, since the current dialogue state obtained based on the dialogue state tracking model is processed by the policy decision model, it is possible to determine whether the current response action requires continued communication with the user to clarify the user's needs, or whether it is a response action to call the product recommendation model to recommend products. Among them, the current dialogue state reflecting the user's current demand information can be obtained based on the dialogue state tracking model, and the product recommendation model combines the user's feature information reflecting the user's historical behavior record and the current dialogue state reflecting the user's current demand information when recommending products, so that the recommended products not only meet the user's current needs, but also reflect the user's potential interests, and achieve the recommendation of suitable products to users at the right time. Therefore, it at least partially overcomes the technical problem of low sales conversion rate of product shopping guides using related technologies, and improves the sales conversion rate of product shopping guides.

[0109] Figure 8 A block diagram of a product shopping guide processing device according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 8 As shown, the product shopping guide processing device 800 includes an acquisition module 810, a first processing module 820, a second processing module 830 and an execution module 840.

[0111] The acquisition module 810 , the first processing module 820 , the second processing module 830 and the execution module 840 are communicatively connected.

[0112] The acquisition module 810 is used to acquire the user's current round of dialogue information.

[0113] The first processing module 820 is used to use the dialogue state tracking model to process the current round of dialogue information to obtain the current dialogue state, wherein the current dialogue state includes the current probability distribution of each product attribute in multiple product attributes, wherein each current probability distribution is used to characterize the probability of each product attribute value in one or more product attribute values ​​corresponding to the product attribute.

[0114] The second processing module 830 is used to use the strategy decision model to process the current dialogue state to obtain the current response action, wherein the current response action includes a response action for continuing the dialogue or a response action for product recommendation, wherein the strategy decision model and the dialogue state tracking model are jointly trained and generated using the first training sample, wherein the first training sample includes multiple rounds of dialogue information samples of each first sample user among multiple first sample users.

[0115] The execution module 840 is used to execute the current response action to obtain the product shopping guide processing result.

[0116] According to the technical solution of the embodiment of the present disclosure, by obtaining the current round of dialogue information of the user, the current round of dialogue information is processed by the dialogue state tracking model to obtain the current dialogue state, the current dialogue state includes the current probability distribution of each product attribute in multiple product attributes, the current dialogue state is processed by the policy decision model to obtain the current response action, the current response action includes a response action for continuing the dialogue or a response action for product recommendation, the policy decision model and the dialogue state tracking model are jointly trained and generated using the first training sample, and the current response action is executed to obtain the product shopping guide processing result. Since the current dialogue state obtained based on the dialogue state tracking model is processed by the policy decision model, it can be determined whether the current response action is a response action that requires continued communication with the user to clarify the user's needs, or a response action for product recommendation, so as to achieve the recommendation of suitable products to the user at the right time, thereby at least partially overcoming the technical problem that the sales conversion rate of product shopping guides using related technologies is not high, and the sales conversion rate of product shopping guides is improved.

[0117] According to an embodiment of the present disclosure, the second processing module 830 may include a first processing submodule and a first determining submodule.

[0118] The first processing submodule is used to process the current dialogue state using the policy decision model to obtain a current vector, wherein the current vector includes a current recommendation probability and a current predicted probability of each product attribute in a plurality of product attributes.

[0119] The first determination submodule is used to determine a current response action according to a current vector.

[0120] According to an embodiment of the present disclosure, the first determining submodule may include a first determining unit, a second determining unit, and a third determining unit.

[0121] The first determining unit is used to determine the maximum probability from the current recommendation probability and multiple current prediction probabilities.

[0122] The second determining unit is used to determine the current response action as a response action for continuing the conversation when the maximum probability is determined to be the current predicted probability.

[0123] The third determining unit is used to determine the current response action as a response action for product recommendation when the maximum probability is determined to be the current recommendation probability.

[0124] According to an embodiment of the present disclosure, the current response action includes a response action for continuing the conversation.

[0125] The execution module 840 may include a second determination submodule and a third determination submodule.

[0126] The second determination submodule is used to determine a target product attribute, wherein the target product attribute is a product attribute corresponding to the maximum probability.

[0127] The third determination submodule is used to determine the target product attribute as reply information for continuing the dialogue with the user.

[0128] According to an embodiment of the present disclosure, the current response action includes a response action for product recommendation.

[0129] The execution module 840 may include an acquisition submodule and a second processing submodule.

[0130] The acquisition submodule is used to obtain the characteristic information of the user and the characteristic information of the candidate products.

[0131] The second processing submodule is used to use the product recommendation model to process the user's feature information, the feature information of the candidate products and the current conversation state to obtain recommended products, wherein the product recommendation model is generated by training using the second training sample, wherein the second training sample includes the feature information of each second sample user among multiple second sample users, the conversation state sample and the real interest product, and the feature information of each sample product among multiple sample products.

[0132] According to an embodiment of the present disclosure, the product recommendation model is generated by training with the second training sample, and may include the following operations.

[0133] The factorization machine model is trained using the second training sample to obtain a product recommendation model.

[0134] According to an embodiment of the present disclosure, the policy decision model and the dialog state tracking model are jointly trained and generated using the first training sample, which may include the following operations.

[0135] For each first sample user, each round of dialogue information samples in the multiple rounds of dialogue information samples is input into multiple recurrent neural network models respectively, and the predicted probability distribution of each product attribute in the multiple product attributes is output, wherein each recurrent neural network model is used to output the predicted probability distribution of the corresponding product attribute. For each round of dialogue information sample, the current round dialogue state is generated according to the predicted probability distribution of each product attribute in the multiple product attributes. The current round dialogue state is input into the policy network model to output the current round vector corresponding to the current round dialogue, wherein the current round vector includes the current round recommendation probability and the current round prediction probability of each product attribute in the multiple product attributes. According to the current round vector, the response action corresponding to the current round dialogue is determined. The response action corresponding to the current round dialogue is executed to obtain the user's feedback result for the response action, wherein the response action corresponding to the current round dialogue includes a response action for continuing the dialogue or a response action for product recommendation. A feedback score corresponding to the feedback result is determined. The response probability and feedback score of the multiple rounds of dialogue are input into the objective function, and the output value of the objective function is output, wherein the response probability is the probability corresponding to the response action, and the response probability is the current round recommendation probability or the current round prediction probability. The network parameters of multiple recurrent neural network models and policy network models are adjusted according to the output values ​​until the objective function converges. The multiple recurrent neural network models obtained after training are used as the dialogue state tracking model, and the policy network model obtained after training is used as the policy decision model.

[0136] According to an embodiment of the present disclosure, determining a feedback score corresponding to a feedback result may include the following operations.

[0137] In the case where the feedback result is determined to be the purchase of the recommended product, the feedback score corresponding to the feedback result is determined to be the first reward score. In the case where the feedback result is determined to be browsing the recommended product and not purchasing the product, the feedback score corresponding to the feedback result is determined to be the second reward score, wherein the first reward score is greater than the second reward score. In the case where the feedback result is determined to be continuing the conversation or not browsing the recommended product, the feedback score corresponding to the feedback result is determined to be the first penalty score. In the case where the feedback result is determined to be terminating the conversation, the feedback score corresponding to the feedback result is determined to be the second penalty score, wherein the first penalty score is greater than the second penalty score.

[0138] According to the modules, submodules, and units of the embodiments of the present disclosure, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, submodules, and units of the embodiments of the present disclosure, any one or more thereof can be split into multiple modules for implementation. According to the modules, submodules, and units of the embodiments of the present disclosure, any one or more thereof can be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a System on a Chip, a System on a Substrate, a System on a Package, an Application Specific Integrated Circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in any appropriate combination of any of them. Alternatively, according to the modules, submodules, and units of the embodiments of the present disclosure, one or more thereof can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0139] For example, any multiple of the acquisition module 810, the first processing module 820, the second processing module 830, and the execution module 840 can be combined in one module / submodule / unit for implementation, or any one of the modules / submodules / units can be split into multiple modules / submodules / units. Alternatively, at least part of the functions of one or more modules / submodules / units in these modules / submodules / units can be combined with at least part of the functions of other modules / submodules / units and implemented in one module / submodule / unit. According to an embodiment of the present disclosure, at least one of the acquisition module 810, the first processing module 820, the second processing module 830, and the execution module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the acquisition module 810, the first processing module 820, the second processing module 830 and the execution module 840 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0140] It should be noted that the product shopping guide processing device part in the embodiment of the present disclosure corresponds to the product shopping guide processing method part in the embodiment of the present disclosure. The description of the product shopping guide processing device part specifically refers to the product shopping guide processing method part, which will not be repeated here.

[0141] Fig. 9 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0142] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0143] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0144] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0145] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, submodule, unit, etc. described above can be implemented by a computer program module.

[0146] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0147] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device.

[0148] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0149] According to the technical solution of the embodiment of the present disclosure, by obtaining the current round of dialogue information of the user, the current round of dialogue information is processed by the dialogue state tracking model to obtain the current dialogue state, the current dialogue state includes the current probability distribution of each product attribute in multiple product attributes, the current dialogue state is processed by the policy decision model to obtain the current response action, the current response action includes a response action for continuing the dialogue or a response action for product recommendation, the policy decision model and the dialogue state tracking model are jointly trained and generated using the first training sample, and the current response action is executed to obtain the product shopping guide processing result. Since the current dialogue state obtained based on the dialogue state tracking model is processed by the policy decision model, it can be determined whether the current response action is a response action that requires continued communication with the user to clarify the user's needs, or a response action for product recommendation, so as to achieve the recommendation of suitable products to the user at the right time, thereby at least partially overcoming the technical problem that the sales conversion rate of product shopping guides using related technologies is not high, and the sales conversion rate of product shopping guides is improved.

[0150] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains a program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the product shopping guide processing method provided by the embodiment of the present disclosure.

[0151] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0152] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0153] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0155] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A product shopping guide processing method, comprising: Get the user's current round of conversation information; Processing the current round of dialogue information using a dialogue state tracking model to obtain a current dialogue state, wherein the current dialogue state includes a current probability distribution of each of a plurality of product attributes, wherein each of the current probability distributions is used to represent a current probability of each of one or more product attribute values ​​corresponding to the product attribute; Processing the current dialogue state using a policy decision model to obtain a current response action, wherein the current response action includes a response action for continuing the dialogue or a response action for product recommendation, wherein the policy decision model and the dialogue state tracking model are generated by jointly training a policy network model and a plurality of recurrent neural network models using a first training sample, wherein the policy decision model corresponds to the policy network model, the dialogue state tracking model corresponds to the plurality of recurrent neural network models, network parameters of the policy network model and the plurality of recurrent neural network models are adjusted according to the same objective function output value, and the first training sample includes a plurality of rounds of dialogue information samples of each first sample user among a plurality of first sample users; and The current response action is executed to obtain a product shopping guide processing result.

2. The method according to claim 1, wherein: The using the policy decision model to process the current dialog state to obtain the current response action includes: Processing the current dialog state using a policy decision model to obtain a current vector, wherein the current vector includes a current recommendation probability and a current predicted probability of each of the multiple product attributes; and The current response action is determined according to the current vector.

3. The method according to claim 2, wherein: The determining the current response action according to the current vector includes: Determining a maximum probability from the current recommendation probability and a plurality of the current prediction probabilities; In the case where the maximum probability is determined to be the current predicted probability, determining the current response action as a response action for continuing the conversation; and In the case where the maximum probability is determined to be the current recommendation probability, the current response action is determined to be a response action for product recommendation.

4. The method according to claim 3, wherein: The current response action includes a response action for continuing the conversation; The executing the response action to obtain a product shopping guide processing result includes: determining a target product attribute, wherein the target product attribute is a product attribute corresponding to the maximum probability; and The target product attribute is determined as reply information for continuing the conversation with the user.

5. The method according to claim 1 or 3, wherein: The current response action includes a response action for product recommendation; The executing the response action to obtain a product shopping guide processing result includes: Acquiring characteristic information of the user and characteristic information of the candidate products; and The feature information of the user, the feature information of the candidate product and the current conversation state are processed by a product recommendation model to obtain a recommended product, wherein the product recommendation model is generated by training using a second training sample, wherein the second training sample includes feature information of each second sample user among multiple second sample users, a conversation state sample and a real interest product, and feature information of each sample product among multiple sample products.

6. The method according to claim 5, wherein: The product recommendation model is generated by training using the second training sample, and includes: The factorization machine model is trained using the second training sample to obtain the product recommendation model.

7. The method according to claim 1, wherein: The strategy decision model and the dialog state tracking model are jointly trained and generated using the first training sample, including: For each of the first sample users, input each round of dialogue information samples in the multiple rounds of dialogue information samples into multiple recurrent neural network models respectively, and output a predicted probability distribution of each product attribute in a plurality of product attributes, wherein each of the recurrent neural network models is used to output a predicted probability distribution of the corresponding product attribute; For each round of the dialogue information sample, generating a dialogue state for this round according to the predicted probability distribution of each product attribute among the multiple product attributes; Input the current round of dialogue state into the policy network model to output a current round vector corresponding to the current round of dialogue, wherein the current round vector includes the current round recommendation probability and the current round prediction probability of each product attribute in the multiple product attributes; Determine a response action corresponding to the current round of dialogue according to the current round vector; Executing the response action corresponding to the current round of dialogue to obtain a user's feedback result for the response action, wherein the response action corresponding to the current round of dialogue includes a response action for continuing the dialogue or a response action for product recommendation; Determining a feedback score corresponding to the feedback result; Inputting the response probability and feedback score of multiple rounds of dialogue into the objective function, and outputting the output value of the objective function, wherein the response probability is the probability corresponding to the response action, and the response probability is the current round recommendation probability or the current round prediction probability; Adjusting network parameters of the multiple recurrent neural network models and the policy network model according to the output value until the objective function converges; and The multiple recurrent neural network models obtained after training are used as the dialogue state tracking model, and the policy network model obtained after training is used as the policy decision model.

8. The method according to claim 7, wherein: The determining a feedback score corresponding to the feedback result includes: In the case where it is determined that the feedback result is a purchase recommendation product, determining the feedback score corresponding to the feedback result as a first reward score; In the case where it is determined that the feedback result is browsing the recommended product and the product is not purchased, determining the feedback score corresponding to the feedback result as the second reward score, wherein the first reward score is greater than the second reward score; In a case where it is determined that the feedback result is to continue the conversation or not browse the recommended product, determining the feedback score corresponding to the feedback result as a first penalty score; In the case where it is determined that the feedback result is to terminate the conversation, a feedback score corresponding to the feedback result is determined as a second penalty score, wherein the first penalty score is greater than the second penalty score.

9. A product shopping guide processing device, comprising: The acquisition module is used to obtain the user's current round of dialogue information; A first processing module, configured to process the current round of dialogue information using a dialogue state tracking model to obtain a current dialogue state, wherein the current dialogue state includes a current probability distribution of each of a plurality of product attributes, wherein each of the current probability distributions is used to represent a probability of each of one or more product attribute values ​​corresponding to the product attribute; a second processing module, configured to process the current dialog state using a policy decision model to obtain a current response action, wherein the current response action includes a response action for continuing the dialog or a response action for product recommendation, wherein the policy decision model and the dialog state tracking model are generated by jointly training a policy network model and a plurality of recurrent neural network models using a first training sample, wherein the policy decision model corresponds to the policy network model, the dialog state tracking model corresponds to the plurality of recurrent neural network models, network parameters of the policy network model and the plurality of recurrent neural network models are adjusted according to the same objective function output value, and the first training sample includes a plurality of rounds of dialog information samples of each first sample user among a plurality of first sample users; and An execution module is used to execute the current response action to obtain a product shopping guide processing result.

10. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein the computer program is used to implement the method according to any one of claims 1 to 8 when executed by a processor.

Images