Machine customer service training system and method, voice reply method, and electronic device

Abstract

The application provides a machine customer service training system and method, a voice reply method and an electronic device. The machine customer service training system comprises a machine customer service model, a user model, a reward parameter configuration component and a termination component. The user model is used to generate a plurality of first predicted texts according to a first text output by the machine customer service model and a historical communication text of the first text. The machine customer service model is used to randomly determine a target predicted text from the plurality of first predicted texts, and generate a second predicted text according to the target predicted text and the historical communication text. The reward parameter configuration component is used to configure a first positive reward parameter for the machine customer service model when the current conversation between the user model and the machine customer service model ends successfully. The termination component is used to terminate the training of the machine customer service model when the number of training times of the machine customer service model is greater than a threshold value, so as to obtain a trained machine customer service model. The application can train a high-quality machine customer service model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a machine customer service training system and method, a voice reply method and an electronic device. BACKGROUND

[0002] In a voice dialogue, the machine customer service will have a voice dialogue with the user, and extract key texts from the voice input by the user, such as asking the user's name, phone number, address, and the like. In order to enable the machine customer service to efficiently collect the key texts, the model training of the machine customer service is required.

[0003] At present, in the training process of the machine customer service, a rule-based user simulator is usually provided for the machine customer service, the rule-based user simulator has a dialogue with the machine customer service to achieve the purpose of training the machine customer service. However, the rule-based user simulator has a fixed dialogue mode with the machine customer service, which reduces the strategy diversity of the machine customer service, and makes the trained machine customer service unable to meet various users, thereby reducing the service quality of the machine customer service and the user experience. SUMMARY

[0004] The aspects of the present application provide a machine customer service training system and method, a voice reply method and an electronic device to improve the service quality of the machine customer service.

[0005] The first aspect of the embodiment of the present application provides a machine customer service training system, comprising a machine customer service model, a user model, a reward parameter configuration component and a termination component, wherein:

[0006] The user model is configured to obtain a first text output by the machine customer service model and a historical communication text of the first text, and generate a plurality of first predicted texts according to the first text and the historical communication text, the plurality of first predicted texts being replies to the first text, and the plurality of first predicted texts being simulated outputs of a plurality of users by the user model;

[0007] The machine customer service model is configured to randomly determine one of the plurality of first predicted texts as a target predicted text, and generate a second predicted text according to the target predicted text and the historical communication text, the second predicted text being a reply to the target predicted text;

[0008] The reward parameter configuration component is configured to, when the second predicted text indicates that the dialogue between the user model and the machine customer service model is ended and successful, configure a first positive reward parameter to the machine customer service model, and the first positive reward parameter is used to adjust the model parameters of the machine customer service model;

[0009] The termination component is configured to determine the number of training times of the machine customer service model, and terminate the training of the machine customer service model when the number of training times is greater than a threshold number of times, to obtain a trained machine customer service model.

[0010] The second aspect of the embodiments of the present application provides a machine customer service training method applied to the machine customer service training system of the first aspect. The machine customer service training method comprises the following steps: a user model acquires a first text output by a machine customer service model and historical communication texts of the first text, and generates a plurality of first predicted texts according to the first text and the historical communication texts. The plurality of first predicted texts are replies to the first text, and the plurality of first predicted texts are simulated by the user model and output by a plurality of users. The machine customer service model randomly determines one of the plurality of first predicted texts as a target predicted text, generates a second predicted text according to the target predicted text and the historical communication texts, and the second predicted text is a reply to the target predicted text. A reward parameter configuration component configures a first positive reward parameter for the machine customer service model when the second predicted text indicates that the user model and the machine customer service model end this dialogue successfully. The first positive reward parameter is used to adjust the model parameters of the machine customer service model. A termination component terminates the training of the machine customer service model when the number of training times of the machine customer service model is determined and the number of training times is greater than a threshold value, and obtains a trained machine customer service model.

[0011] The third aspect of the embodiments of the present application provides a voice reply method, which comprises the following steps: acquiring user text information, wherein the user text information is a result of voice recognition on user voice information; inputting the user text information into a machine customer service model trained in the second aspect to perform text processing, and obtaining reply text information; converting the reply text information into reply voice information and outputting the reply voice information to reply to the user voice information.

[0012] The fourth aspect of the embodiments of the present application provides an electronic device, which comprises a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the machine customer service training method of the second aspect or the voice reply method of the third aspect is implemented.

[0013] The embodiments of the present application are applied to a voice dialogue scene, and the machine customer service training system provided by the embodiments comprises a machine customer service model, a user model, a reward parameter configuration component, and a termination component. The user model simulates a plurality of users, outputs a plurality of first predicted texts for a first text output by the machine customer service, and then provides diversified input texts for the machine customer service model to train the machine customer service model, so that the trained machine customer service model can serve a variety of users, and the service quality of the machine customer service and the user experience are improved. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and their descriptions serve to explain the present application and do not constitute improper limitations on the present application. In the drawings:

[0015] Figure 1 A structural block diagram of a machine customer service training system provided by an exemplary embodiment of the present application is intended to be provided.

[0016] Figure 2 A structural block diagram of another machine customer service training system provided by an exemplary embodiment of the present application is intended to be provided.

[0017] Figure 3 A step flowchart of a machine customer service training method provided by an exemplary embodiment of the present application is intended to be provided.

[0018] Figure 4 A step flowchart of another machine customer service training method provided by an exemplary embodiment of the present application is intended to be provided.

[0019] Figure 5 A structural schematic diagram of an electronic device provided by an exemplary embodiment of the present application is intended to be provided. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in conjunction with the specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work are within the scope of protection of the present application.

[0021] In view of the low service quality of machine customer service in the existing voice dialogue scene, the machine customer service training system provided in the embodiments of the present application includes a machine customer service model, a user model, a reward parameter configuration component and a termination component. The user model outputs a plurality of first predicted texts for a first text output by the machine customer service by simulating a plurality of users, thereby providing the machine customer service model with diversified input texts for training of the machine customer service model, and thereby enabling the trained machine customer service model to serve a variety of users, improving the service quality of the machine customer service and the user experience.

[0022] The technical solutions provided by the embodiments of the present application will be described in detail below in conjunction with the drawings.

[0023] Figure 1 A structural schematic diagram of a machine customer service training system 10 provided by an exemplary embodiment of the present application is intended to be provided. As shown in the figure, the machine customer service training system 10 includes a user model 11, a machine customer service model 12, a reward parameter configuration component 13 and a termination component 14. Further optionally, as shown in the figure, the machine customer service training system 10 further includes a storage component 15. Figure 1 Figure 1

[0024] ​​In this embodiment, the deployment implementation manner of the user model 11, the machine service model 12, the reward parameter configuration component 13, the termination component 14 and the storage component 15 is not limited. Alternatively, these components are implemented in a distributed manner, and the overall system function is implemented by means of a cloud computing system. For example, the user model 11, the machine service model 12, the reward parameter configuration component 13, the termination component 14 and the storage component 15 can be deployed in the cloud, for example, in a data center or a central cloud system, and implemented as a cloud server. With respect to the cloud deployment, the machine service training system can also be deployed on a regular server, a cloud server or a server array, etc.

[0025] The machine service training system provided in the embodiments of this application can be applied to various voice dialogue industries, for example, vehicle navigation, intelligent household appliances and machine service, etc. In addition, the trained machine service can also realize intelligent question answering, open field chatting and task type dialogue with users. The machine service model trained by using the machine service training system provided in the embodiments of this application can meet the individualized dialogue needs of various users and improve the user experience.

[0026] For example, when the machine service training system provided in the embodiments of this application is applied to a service system of an e-commerce industry, the trained machine service can meet the service consultation needs of various users. For another example, when the machine service training system provided in the embodiments of this application is applied to intelligent household appliances, the trained machine service can effectively play question answering games, chat, etc. with various users.

[0027] No matter what the voice dialogue industry is, the dialogue tasks of the voice dialogue industry are various, and the users for dialogue with the machine service are also various. In the embodiments of this application, the division of dialogue tasks is not limited. In this embodiment, the machine service training system can simulate the dialogue between different users and the machine service model, realize the training of the machine service model, and then enable the trained machine service to serve various users and improve the user experience.

[0028] In the machine customer service training system of the embodiment, the user model 11 is configured to obtain a first text output by the machine customer service model 12 and a historical communication text of the first text, and generate a plurality of first predicted texts according to the first text and the historical communication text, the plurality of first predicted texts being replies to the first text, and the plurality of first predicted texts being simulated outputs of a plurality of users by the user model; the machine customer service model 12 is configured to randomly determine one of the plurality of first predicted texts as a target predicted text, generate a second predicted text according to the target predicted text and the historical communication text, and the second predicted text being a reply to the target predicted text; the reward parameter configuration component 13 is configured to, when the second predicted text indicates that the user model 11 and the machine customer service model 12 end this dialogue successfully, configure a first positive reward parameter to the machine customer service model, and the first positive reward parameter being used to adjust the model parameters of the machine customer service model; and the termination component 14 is configured to determine the number of training times of the machine customer service model, and terminate the training of the machine customer service model when the number of training times is greater than a threshold number of times, to obtain the trained machine customer service model.

[0029] In the embodiments of the present application, the user model 11 and the machine customer service model 12 are both reinforcement learning models. The reinforcement learning model is a learning process in which an “agent” interacts with its “external environment” and optimizes the decision-making of the agent, so that the agent can make reasonable decisions to achieve specific goals under given problems. The essence of reinforcement learning is that the “agent” learns how to make decisions (i.e., map a given state to a reasonable action) to maximize the reward signal (positive reward parameter). In addition, reinforcement learning does not have labeled training data as in supervised learning, and the mapping of state to action is optimized by the interaction of the agent and the external environment. During the optimization process, the agent continuously adjusts the mapping of state to action through the reward signal (positive reward parameter or negative reward parameter) until the desired decision is learned. Therefore, the feature of reinforcement learning is interaction, and the desired strategy is learned through repeated trial and error.

[0030] For example, if the user model 11 is an “agent”, the machine customer service model 12 is the “external environment” of the user model 11, and the user model 11 can be trained. If the machine customer service model 12 is an “agent”, the user model 11 is the “external environment” of the machine customer service model 12, and the machine customer service model 12 can be trained. In the embodiments of the present application, the user model 11 and the machine customer service model 12 are jointly trained to obtain a high-quality machine customer service model 12.

[0031] Further, the first text can be a first question text that the machine service model 12 first outputs at this time of training, or can be a dialogue text that the machine service model 12 outputs at the last round of dialogue of this time of training. Here, one training is a training that the machine service model 12 and the user model 11 complete one dialogue task. One round of dialogue refers to one question and one reply of the machine service model 12 and the user model 11 in the one training process.

[0032] For example, referring to Table 1, six rounds of dialogue are performed in one training. In the embodiment of the present application, the first text refers to the first text in one training, which can be output by the user model or the machine service model. In Table 1, the first text is output by the user model, the machine service model replies to the first text of the user model to obtain the first text, completes the first round of dialogue, then inputs the first text into the user model to enter the second round of dialogue, the user model outputs a plurality of first predicted texts, inputs one of the first predicted texts as a target predicted text into the machine service model, the machine service model outputs a second predicted text, and completes the second round of dialogue. Then, the second predicted text is input as the first text into the user model to perform the third round of dialogue. In this way, six rounds of dialogue are completed in Table 1, and the dialogue task is completed. Moreover, the last round (the sixth round) of the machine service model outputs the second predicted text “thank you”, indicating that the current dialogue is completed, and the machine service model collects the telephone number “1881069532” provided by the user model, indicating that the current dialogue is successful. The first positive reward parameter can be configured for the machine service model, which is used to adjust the model parameters of the machine service model, and encourages the machine service model to reply to the user model in the manner of Table 1.

[0033] Table 1

[0034]

[0035] For example, referring to Table 2, five rounds of dialogue are performed in another training. Table 2 corresponds to that the dialogue task is not completed. Moreover, the last round (the fifth round) of the machine service model outputs the second predicted text “I am a robot, goodbye”, indicating that the current dialogue is completed, and the machine service model does not collect the telephone number provided by the user model, indicating that the current dialogue fails. The negative reward parameter can be configured for the machine service model, which is used to adjust the model of the machine service model, so that the machine service model avoids replying to the user model in the manner of Table 2.

[0036] Table 2

[0037]

[0038] For another example, referring to Table 3, a further training is performed, 5 rounds of dialogues are performed, Table 3 completes the dialogue task and the dialogue is successful, the first positive reward parameter can be configured for the machine customer service model, which is used to adjust the model parameters of the machine customer service model to encourage the machine customer service model to reply to the user model in the manner of Table 3.

[0039] Table 3

[0040]

[0041]

[0042] In the embodiment of the present application, the storage component 15 is connected with the user model 11 and the machine customer service model 12, and the data generated by the user model 11 and the machine customer service model 12 can be stored in the storage component 15. The user model 11 and the machine customer service model 12 can also obtain data from the storage component 15 in the training process.

[0043] In the embodiment, referring to Figure 2 , the user model 11 includes: a first understanding module 111, configured to obtain the first text output by the machine customer service model 12, and generate a first customer service action and a first customer service slot value according to the first text, the first customer service action being a predicted action of an initial customer service action, the initial customer service action being an action adopted by the machine customer service model to generate the first text, the first customer service slot value being a predicted slot value of an initial customer service slot value, the initial customer service slot value being a slot value adopted by the machine customer service model to generate the first text; a plurality of first strategy modules (A1 to An), configured to obtain historical dialogue texts, and respectively simulate different users to generate a first user action and a first user slot value according to the first customer service action, the first customer service slot value and the historical dialogue texts; and a first natural text generation module 112, configured to generate a corresponding first predicted text according to the first user action and the first user slot value, the first strategy module corresponding to the first predicted text in a one-to-one manner, and the first predicted text being a natural language of the first user slot value.

[0044] Referring to Table 4, for example, a plurality of customer service actions and a plurality of user actions are provided, and the plurality of user actions can be pre-configured to the user model 11, and the plurality of customer service actions can be pre-configured to the machine customer service model 12. Then, in the training process, the machine customer service model 12 generates the first text according to the initial customer service action and the initial customer service slot value, the first understanding module 111 generates the first customer service action and the first customer service slot value according to the first text, the first customer service action is a prediction of the initial customer service action, and the first customer service slot value is a prediction of the initial customer service slot value. Wherein, the purpose of training requires that the first customer service action output by the first understanding module 111 tends to be close to the initial customer service action, and the first customer service slot value tends to be close to the initial customer service slot value, and the loss value of the first customer service action generated by the first understanding module 111 of the trained user model 11 according to the first text is less than a preset first loss value threshold, and the loss value of the first customer service slot value generated is less than a preset second loss value threshold.

[0045] Table 4

[0046]

[0047] In addition, referring to Figure 2 , a plurality of first strategy modules (A1 to An) are provided, each first strategy module simulates one of the users, and different first strategy modules generate different first user actions and first user slot values for the same first customer service slot value, first customer service text and historical dialogue text. After the first user action and the first user slot value are obtained by the first natural text generation module 112, the first user slot value is naturalized according to the first user action, and then the first predicted text corresponding to each first strategy module (such as B1 to Bn in Figure 2 In the embodiments of the present application, the plurality of first strategy modules simulating different users can enable the machine customer service model obtained by training to successfully complete the dialogue with different users. In addition, the plurality of first strategy modules are arranged in the same user model, which not only provides the plurality of first strategy modules with the diversity of simulating different users to train the machine customer service model, but also enables the plurality of first strategy modules to share the first understanding module 111 and the first natural text generation module 112, thereby reducing the parameter amount of the model when the user model 11 and the machine customer service model 12 are jointly trained, and improving the training efficiency of the machine customer service model 12.

[0048] The historical communication text includes historical dialogue text. The historical dialogue text refers to the text output by the user model 11 (the first text and the target prediction text) and the text output by the machine customer service model 12 before the current round of dialogue in the current training. Referring to Table 1, if the first text is “Please tell me your mobile number” in the first round of dialogue, there is no historical dialogue text. If the first text is “Is it 188106?” in the third round of dialogue, the historical dialogue text is the text of the first and second rounds of dialogue “For this problem, reply my phone, please tell me your mobile number, please remember my phone number, please tell me your mobile number”.

[0049] In an optional embodiment, referring to Figure 2 The first natural text generation module 112 includes a changing unit configured to change the first user slot value to obtain a second user slot value, and a first natural text generation unit configured to generate a first prediction text corresponding to the first user action and the second user slot value, the first prediction text being a natural language version of the second user slot value.

[0050] The changing unit simulates a situation in which an automatic speech recognition technology (ASR) makes a mistake in recognizing user voice information during user dialogue. The changing unit specifically processes at least one character in the first user slot value by replacing, deleting, or adding a character, so as to change the first user slot value. For example, a digit of a mobile phone number is added, a digit of a mobile phone number is deleted, or a mobile phone number is changed. The changing unit improves the fault tolerance of the machine customer service model and enhances the training of the machine customer service model.

[0051] The first natural text generation unit outputs the first prediction text based on the second user slot value or the third user slot value by using a template. The first user action is used to determine the template. For example, if the first user action is “inform_update” in Table 4, the template is “Yes ~ ~, no, yes ~ ~”. If the third user slot value is “178 18”, the third user slot value replaces the placeholder “~ ~” in the template, and the output first prediction text is “Yes 178, no, yes 18”. In the embodiment of the present application, multiple templates can be configured for each user action in advance, and then one template is selected according to the parameters of the first natural text generation unit during prediction. After the machine customer service model is trained, the corresponding template can be determined according to the user action in the actual use process.

[0052] Referring to Figure 2The machine customer service model 12 comprises: a second understanding module 121 configured to generate a second user action and a third user slot value according to the historical dialogue text and the target prediction text, the second user action being a prediction of the first user action, and the third user slot value being a prediction of the second user slot value; a state tracking module 122 configured to determine a second customer service slot value according to the second user action, the third user slot value, and the historical customer service slot value, the second customer service slot value being a reply to the third user slot value; a second strategy module 123 configured to determine a second customer service action according to the second user action and the second customer service slot value, the second customer service action being a reply action to the second user action; and a second natural text generation module 124 configured to generate corresponding second prediction text according to the second customer service action and the second customer service slot value, the second prediction text being a natural language of the second customer service slot value.

[0053] The historical dialogue text further comprises a historical customer service slot value. The historical customer service slot value is a customer service slot value used by the machine customer service model to generate the historical prediction text. For example, referring to Table 1, if the prediction text generated by the machine customer service model historically comprises “Please tell me your mobile phone number, please tell me your mobile phone number”, the historical customer service slot value is the slot value corresponding to the generation of “Please tell me your mobile phone number” by the machine customer service model. In addition, the historical customer service slot value can be only the customer service slot value generated by the machine customer service in the last round of dialogue.

[0054] Further, in the training process, the second user action generated by the second understanding module 121 is a prediction of the first user action, and the third user slot value generated is a prediction of the second user slot value. The training purpose of the machine customer service model is to make the second user action generated by the second understanding module 121 approach the first user action, and the third user slot value generated approach the second user slot value. Finally, the loss value of the second user action generated by the second understanding module 121 and the first user action is less than a preset third loss value threshold, and the loss value of the third user slot value generated and the second user slot value is less than a preset fourth loss value threshold.

[0055] In addition, the second customer service slot value generated by the state tracking module 122 is a predicted slot value obtained according to the second user action, the third user slot value, and the historical customer service slot value, to reply to the third user slot value. The second strategy module 123 can determine the second customer service action according to the second user action and the second customer service slot value, so that the second natural text generation module generates the second prediction text according to the second customer service action and the second customer service slot value.

[0056] Referring to Figure 2In the process of constructing the machine customer service training system 13, the user model 11 (including: the first understanding module 111, a plurality of first strategy modules A1-An, the first natural text generation module 112), the machine customer service model 12 (including: the second understanding module 121, the state tracking module 122, the second strategy module 123, and the second natural text generation module 124), the reward parameter configuration component 13, the termination component 14, and the storage component 15 are constructed. Then, the user model 11 is configured with initial model parameters, and the machine customer service model is configured with initial model parameters, wherein the initial model parameters of the user model 11 include initial parameters that each module has, and the initial model parameters of the machine customer service model also include initial parameters that each module has. In the training process, the user model 11 and the machine customer service model 12 are configured with reward parameters (positive reward parameters or negative reward parameters) through the supervision of the reward parameter configuration component 14, so that the user model 11 and the machine customer service model 12 adjust their respective model parameters according to the reward parameters, achieving the effect of reinforcement. The finally trained user model 11 can simulate a variety of users, and the machine customer service model 12 can serve a variety of users. The termination component 14 can count the number of training times of the machine customer service model, and terminate the training of the machine customer service model when the number of training times is greater than a number threshold. The number threshold can be set as needed. In addition, the termination component 14 can also terminate the training of the machine customer service model according to other conditions, for example, the machine customer service model successfully completes the dialogue in a plurality of consecutive training processes, and / or the loss value of any two first predicted texts in a plurality of consecutive training processes of the machine customer service model is greater than a loss value threshold, and / or the second predicted text output by the machine customer service training model is different from the historical predicted text.

[0057] In the above optional embodiment, with reference to Figure 2 , Table 1 and Table 4, the following is illustrated:

[0058] The first round of dialogue: the user model 11 outputs the text c1 (reply my phone number for this question), the second understanding module 121 of the machine customer service model 12 generates the user action d1 (requirement) and the user slot value e1 (reply phone number) according to the text c1 (no historical dialogue text), and the state tracking module 122 determines the customer service slot value f1 (phone number) according to the user action d1 and the user slot value e1 (no historical customer service slot value). The second strategy module 123 determines the customer service action g1 (request) according to the user action d1 and the customer service slot value f1, and the second natural text generation module 124 determines the text c2 (the first text of Table 1, such as What is your mobile phone number?) according to the customer service action g1 and the customer service slot value f1. The first round of dialogue in Table 1 is completed.

[0059] The second round of dialogue: the first understanding module 111 generates the customer action g2 (request) and the customer slot value f2 (phone number) according to the text c2. The first strategy module A1 generates the user action d21 (offer) and the user slot value e21 (number) according to the customer action g2 and the customer slot value f2, and the historical dialogue text h1 (text c1 and text c2). The first strategy module A2 generates the user action d22 (inform_normal) and the user slot value e22 (1881069532) according to the customer action g2 and the customer slot value f2, and the historical dialogue text h1 (text c1 and text c2). The first strategy module An generates the user action d2n (inform_update) and the user slot value e2n (178, 188) according to the customer action g2 and the customer slot value f2, and the historical dialogue text h1 (text c1 and text c2). The change unit does not process the user slot value e21, and the first natural text generation unit obtains the text c31 (please remember my phone number) according to the user action d21 and the user slot value e21. The change unit changes the user slot value e22 to obtain the user slot value e32 (1881079532), and the first natural text generation unit obtains the text c32 (the number is 1881079532) according to the user action d22 and the user slot value e32. The change unit changes the user slot value e23 to obtain the user slot value e33 (178, 18), and the first natural text generation unit obtains the text c33 (the number is 178, not 18) according to the user action d23 and the user slot value e33. The second understanding module 121 randomly determines the text c31 as the target predicted text, and generates the user action d3 (requirement, requirement) and the user slot value e4 (number) according to the text c31 and the historical dialogue text (text c1 and text c2). The state tracking module 122 determines the customer slot value f2 (phone number) according to the user action d3, the user slot value e4 and the historical customer slot value (customer slot value f1). The second strategy module 123 determines the customer action g3 (request) according to the user action d3 and the customer slot value f2. The second natural text generation module 124 determines the text c4 (is the second predicted text of the second round of dialogue machine customer model output in table 1: please tell me your mobile phone number) according to the customer action g3 and the customer slot value f2. The second round of dialogue in table 1 is completed.

[0060] In addition, the dialogue process of the third round to the sixth round can refer to the second round described above, which will not be repeated here.

[0061] Further, with reference to Figure 2The reward parameter configuration component 13 is further configured to: determine a loss value of any two of the plurality of first predicted texts according to a preset loss function; and configure a first negative reward parameter for the user model if the loss value is less than a loss value threshold, the first negative reward parameter being used to adjust a model parameter of the user model.

[0062] Specifically, the prediction loss function can be L2loss (mean square error loss function). Wherein, the model parameter of the user model is adjusted according to the loss value, that is, when the loss value is less than the loss value threshold, a negative reward parameter is configured for the user model, and when the loss value is greater than the loss value threshold, a positive reward parameter is configured for the user model, so as to encourage the plurality of first predicted texts output by the user model to be different, thereby improving the diversity of the first predicted texts output by the user model.

[0063] In an optional embodiment, the reward parameter configuration component 13 is further configured to: configure a second negative reward parameter for the machine service model and a second positive reward parameter for the user model when the second predicted text indicates that the current dialogue between the user model and the machine service model ends and fails, the second negative reward parameter being used to adjust a model parameter of the machine service model, and the second positive reward parameter being used to adjust a model parameter of the user model.

[0064] Specifically, when the current dialogue of the machine service model fails, such as when the machine service model fails to collect preset information or fails to provide useful information to the user model, a second negative reward parameter is configured for the machine service model to adjust the model parameter of the machine service model, so that the machine service model can more effectively complete the dialogue when dialoguing with the user model, and the dialogue is successful. Configuring a second positive reward parameter for the user model can encourage the user model to dialogue with the machine service model in this way, thereby improving the service capability of the machine service model.

[0065] In an optional embodiment, in the embodiments of the present application, the reward parameter configuration component 13 is further configured to: obtain at least one historical predicted text of the second predicted text, and configure a third negative reward parameter for the machine service model if the second predicted text is the same as the historical predicted text, the third negative reward parameter being used to adjust a model parameter of the machine service model, and the historical predicted text being a predicted text historically output by the machine service model.

[0066] Specifically, the historical predicted text and the second predicted text are the same dialogue text. For example, referring to Table 1, the second predicted text "Please tell me your mobile phone number" output by the machine service model in the second round of dialogue is the same as the predicted text "Please tell me your mobile phone number" output by the machine service model in the first round of dialogue, and then the machine service model is configured with the third negative reward parameter to adjust the model parameters of the machine service model, so that the machine service model avoids the same reply in a dialogue, improves the diversity of the text output by the machine service model, and enables the trained machine service model to output different information to the user in a dialogue with the user, thereby improving the user experience.

[0067] Further, after the machine service model is trained, the quality of the machine service model can be tested. Specifically, the machine service model is used to interact with the user model for multiple dialogues, and each dialogue is interacted for a fixed number of rounds. The dialogue path of each dialogue is counted, wherein the dialogue path is obtained by splicing the user actions and the service actions in time sequence (for example, user action→service action→user action→service action), the user actions in the test process are output by the first strategy module, and the service actions are output by the second strategy module. Then, the total number of dialogue paths, the number of each dialogue path, and the proportion of the first number to the total number are counted. The types of dialogue paths can intuitively reflect the dialogue diversity. The more the types of dialogue paths, the richer the dialogue, and the higher the dialogue diversity. The variance of the proportion of each dialogue path to the total number can also be calculated, and the dialogue diversity is determined by the variance. The smaller the variance, the more balanced the proportion of each dialogue path, and the better the strategy diversity of the system. If the dialogue diversity of the trained machine service model after the test meets the requirements, the machine service model can be configured in the product to serve real users. If the requirements are not met, the training can be continued until the requirements are met.

[0068] The machine service model trained by the embodiments of the present application can correctly guide the dialogue to obtain a higher dialogue success rate in the shortest number of rounds. Furthermore, the machine service model can specifically tolerate errors and successfully complete dialogues with different and various users. In addition, the machine service model can give different replies to the same user text to improve the user experience. Finally, the present application can simulate various user models and set various rewards to realize that a machine service model can interact with multiple simulated users to train the machine service model, so that the trained machine service model has robustness and can be applied to various dialogue products.

[0069] The embodiment of the application is applied to a voice dialogue scene, and provides a machine service training system, which comprises a machine service model, a user model, a reward parameter configuration component and a termination component. The user model outputs a plurality of first predicted texts for a first text output by the machine service in simulation of a plurality of users, so as to provide diversified input texts for the machine service model for training of the machine service model, and thus the machine service model obtained through training can serve a plurality of users, and the service quality of the machine service and the user experience are improved.

[0070] In the embodiment of the application, in addition to providing a machine service training system, a machine service training method is also provided, which is applied to any one of the machine service training systems and can train a machine service model for various voice tasks, as shown in the figure, the method comprises the following steps. Figure 3

[0071] S301, the user model acquires the first text output by the machine service model and the historical communication text of the first text, and generates a plurality of first predicted texts according to the first text and the historical communication text.

[0072] The plurality of first predicted texts are replies to the first text, and the plurality of first predicted texts are output by the user model simulating a plurality of users. S301 comprises: acquiring the first text output by the machine service model, and generating a first service action and a first service slot value according to the first text, the first service action being a predicted action of an initial service action, the initial service action being an action adopted by the machine service model to generate the first text, the first service slot value being a predicted slot value of an initial service slot value, the initial service slot value being a slot value adopted by the machine service model to generate the first text; acquiring the historical dialogue text, and simulating different users according to the first service action, the first service slot value and the historical dialogue text to generate a first user action and a first user slot value; and generating a corresponding first predicted text according to the first user action and the first user slot value, the first strategy module corresponding to the first predicted text one by one, and the first predicted text being a natural language of the first user slot value.

[0073] Further, generating the corresponding first predicted text according to the first user action and the first user slot value comprises: changing the first user slot value to obtain a second user slot value; and generating the corresponding first predicted text according to the first user action and the second user slot value, the first predicted text being a natural language of the second user slot value.

[0074] S302, the machine service model randomly determines one of the plurality of first predicted texts as a target predicted text, and generates a second predicted text according to the target predicted text and the historical communication text.

[0075] ​The second predicted text is a reply to the target predicted text. S302 includes: generating a second user action and a third user slot value according to the historical dialogue text and the target predicted text, the second user action being a prediction of the first user action, and the third user slot value being a prediction of the second user slot value; determining a second customer service slot value according to the second user action, the third user slot value, and the historical customer service slot value, the second customer service slot value being a reply to the third user slot value; determining a second customer service action according to the second user action and the second customer service slot value, the second customer service action being a reply action to the second user action; and generating a corresponding second predicted text according to the second customer service action and the second customer service slot value, the second predicted text being a natural language of the second customer service slot value.

[0076] S303, the reward parameter configuration component configures a first positive reward parameter for the machine customer service model when the second predicted text indicates that the current dialogue between the user model and the machine customer service model ends successfully.

[0077] The first positive reward parameter is used to adjust the model parameters of the machine customer service model.

[0078] S304, the termination component determines the number of training times of the machine customer service model, and terminates the training of the machine customer service model when the number of training times is greater than a threshold number of times, to obtain a trained machine customer service model.

[0079] In an optional embodiment, the machine customer service training method further includes: the reward parameter configuration component determines a loss value of any two of the plurality of first predicted texts according to a preset loss function; and if the loss value is less than a loss value threshold, a first negative reward parameter is configured for the user model, the first negative reward parameter being used to adjust the model parameters of the user model.

[0080] In an optional embodiment, the machine customer service training method further includes: the reward parameter configuration component configures a second negative reward parameter for the machine customer service model and a second positive reward parameter for the user model when the second predicted text indicates that the current dialogue between the user model and the machine customer service model ends unsuccessfully, the second negative reward parameter being used to adjust the model parameters of the machine customer service model, and the second positive reward parameter being used to adjust the model parameters of the user model.

[0081] In an optional embodiment, the machine customer service training method further includes: the reward parameter configuration component obtains at least one historical predicted text of the second predicted text, and configures a third negative reward parameter for the machine customer service model if the second predicted text is the same as the historical predicted text, the third negative reward parameter being used to adjust the model parameters of the machine customer service model, and the historical predicted text being a predicted text historically output by the machine customer service model.

[0082] The machine customer service training method provided in the embodiments of the present application provides a machine customer service training system in a voice dialogue scene, which comprises a machine customer service model, a user model and a reward parameter configuration component. The user model outputs a plurality of first predicted texts for a first text output by the machine customer service by simulating a plurality of users, thereby providing the machine customer service model with diversified input texts for training of the machine customer service model, and thereby enabling the machine customer service model obtained through the training to serve a plurality of users, and improving the service quality of the machine customer service and the user experience.

[0083] In the embodiments of the present application, a voice reply method is also provided, which is applied to the machine customer service model obtained through any one of the machine customer service training methods, and can serve a plurality of users, such as Figure 4 as shown in the figure, the method comprises the following steps.

[0084] S401, obtaining user text information.

[0085] The user text information is a result of voice recognition on user voice information.

[0086] S402, inputting the user text information into the machine customer service model for text processing to obtain reply text information.

[0087] The machine customer service model is obtained through the machine customer service training method.

[0088] S403, converting the reply text information into reply voice information and outputting the reply voice information to reply to the user voice information.

[0089] The voice reply method provided in the embodiments of the present application can give the user expected reply voice information based on various user text information, and improve the user experience.

[0090] In addition, some of the processes described in the above embodiments and the accompanying drawings contain a plurality of operations appearing in a specific order, but it should be clearly understood that these operations can be executed in the order appearing in the text or in parallel, and the serial numbers are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the "first", "second" and the like in the text are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0091] Figure 5 A structural schematic diagram of an electronic device is provided for the exemplary embodiments of the present application. The electronic device is used to run the upper body voice reply method. As Figure 5As shown, the electronic device includes a memory 54 and a processor 55.

[0092] The memory 54 is configured to store computer programs and can be configured to store various other data to support operations on the electronic device. The memory 54 can be an Object Storage Service (OSS).

[0093] The memory 54 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk, or an optical disk.

[0094] The processor 55 is coupled to the memory 54 and is configured to execute computer programs in the memory 54 for: obtaining, by a user model, a first text output by a machine customer service model and a historical communication text of the first text, and generating a plurality of first predicted texts according to the first text and the historical communication text, the plurality of first predicted texts being replies to the first text, the plurality of first predicted texts being simulated by the user model as a plurality of user outputs; randomly determining, by the machine customer service model, one of the plurality of first predicted texts as a target predicted text, generating a second predicted text according to the target predicted text and the historical communication text, the second predicted text being a reply to the target predicted text; configuring, by a reward parameter configuration component, a first positive reward parameter to the machine customer service model when the second predicted text indicates that the user model and the machine customer service model end this time of conversation successfully; adjusting, by the reward parameter configuration component, a model parameter of the machine customer service model according to the first positive reward parameter; and terminating, by a termination component, training of the machine customer service model when a training number of the machine customer service model is determined and the training number is greater than a number threshold, to obtain a trained machine customer service model.

[0095] Further optionally, the processor 55, when generating the first predicted text corresponding to the first user action and the first user slot value according to the first user action and the first user slot value, is specifically configured to: change the first user slot value to obtain a second user slot value; and generate the first predicted text corresponding to the first user action and the second user slot value, the first predicted text being a natural language of the second user slot value.

[0096] Further optionally, the processor 55, when generating the first predicted text corresponding to the first user action and the first user slot value according to the first user action and the first user slot value, is specifically configured to: change the first user slot value to obtain a second user slot value; and generate the first predicted text corresponding to the first user action and the second user slot value, the first predicted text being a natural language of the second user slot value.

[0097] Further optionally, the processor 55, when generating the second predicted text according to the target predicted text and the historical communication text, is specifically configured to: generate a second user action and a third user slot value according to the historical dialogue text and the target predicted text, the second user action being a prediction of the first user action, and the third user slot value being a prediction of the second user slot value; determine a second customer slot value according to the second user action, the third user slot value and the historical customer slot value, the second customer slot value being a reply to the third user slot value; determine a second customer action according to the second user action and the second customer slot value, the second customer action being a reply action to the second user action; and generate the second predicted text corresponding to the second customer action and the second customer slot value, the second predicted text being a natural language of the second customer slot value.

[0098] In an optional embodiment, the processor 55 is further configured to cause the reward parameter configuration component to configure the machine customer service model with a first positive reward parameter when the second predicted text indicates that the user model and the machine customer service model end this dialogue successfully, until the number of training of the machine customer service model is greater than a number threshold, to obtain a trained machine customer service model.

[0099] In an optional embodiment, the processor 55 is further configured to cause the reward parameter configuration component to determine a loss value of any two of the plurality of first predicted texts according to a preset loss function, and if the loss value is less than a loss value threshold, configure a first negative reward parameter to the user model, the first negative reward parameter being used to adjust a model parameter of the user model.

[0100] In an optional embodiment, the processor 55 is further configured to cause the reward parameter configuration component to, when the second predicted text indicates that the user model and the machine customer service model end this time of conversation and fail, configure a second negative reward parameter to the machine customer service model and a second positive reward parameter to the user model, the second negative reward parameter being used to adjust a model parameter of the machine customer service model, and the second positive reward parameter being used to adjust a model parameter of the user model.

[0101] In another optional embodiment, the processor 55 is further configured to obtain user text information, the user text information being a result of speech recognition on the user voice information, perform text processing on the machine customer service model trained above by using the user text information to obtain reply text information, and output the reply text information to reply to the user voice information.

[0102] Further, as shown in Figure 5 , the electronic device further includes a firewall 51, a load balancer 52, a communication component 56, a power supply component 58, and other components. Figure 5 Some components are only schematically shown in the figure, and it does not mean that the electronic device only includes Figure 5 the components shown in the figure.

[0103] The electronic device provided by the embodiments of the present application, in a voice conversation scenario, provides a machine customer service training system, which includes a machine customer service model, a user model, a reward parameter configuration component, and a termination component. The user model outputs a plurality of first predicted texts for a first text output by the machine customer service by simulating a plurality of users, thereby providing diversified input texts for the machine customer service model to train the machine customer service model, and thereby enabling the trained machine customer service model to serve a variety of users, improving the service quality of the machine customer service and the user experience.

[0104] Correspondingly, the embodiments of the present application also provide a computer readable storage medium storing a computer program, when the computer program / instruction is executed by a processor, causing the processor to implement the steps in the method shown in Figure 3 or Figure 4 .

[0105] Correspondingly, the embodiments of the present application also provide a computer program product, including a computer program / instruction, when the computer program / instruction is executed by a processor, causing the processor to implement the steps in the method shown in Figure 3 or Figure 4 .

[0106] The communication component in the aforementioned Figure 5 is configured to facilitate wired or wireless communication between a device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, a 2G, 3G, 4G / LTE, 5G, or the like mobile communication network, or a combination thereof. In an example embodiment, the communication component receives a broadcast signal or a broadcast-related text from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0107] The power component in the aforementioned Figure 5 supplies power to various components of the device where the power component is located. The power component can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power component is located.

[0108] Those skilled in the art will understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0109] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 The flowchart illustrations and / or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatuses, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart illustrations and / or block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable procedures or functions. In some embodiments, the flowchart illustrations and / or block diagrams can include Figure 1 The flowchart illustrations and / or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatuses, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart illustrations and / or block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable procedures or functions. In some embodiments, the flowchart illustrations and / or block diagrams can include

[0110] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams.Figure 1 one or more processes and / or blocks Figure 1 the function(s) specified in the block or blocks.

[0111] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more processes and / or blocks Figure 1 the function(s) specified in the block or blocks.

[0112] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0113] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0114] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "comprising," "including," and any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements in the list, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0116] The above merely provides an example of the present application, but is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A machine customer service training system, characterized by, The method comprises the following steps: The user model is used for obtaining the first text output by the machine customer service model and historical communication text of the first text, and generating a plurality of first predicted texts according to the first text and the historical communication text, the plurality of first predicted texts being replies to the first text, and the plurality of first predicted texts being simulated outputs of a plurality of users by the user model; The machine customer service model is used for randomly determining one of the plurality of first predicted texts as a target predicted text, and generating a second predicted text according to the target predicted text and the historical communication text, the second predicted text being a reply to the target predicted text; The reward parameter configuration component is used for configuring a first positive reward parameter for the machine customer service model when the second predicted text indicates that the current dialogue of the user model and the machine customer service model ends successfully, and the first positive reward parameter is used for adjusting the model parameter of the machine customer service model; The termination component is used for determining the number of training times of the machine customer service model, and terminating the training of the machine customer service model when the number of training times is greater than a threshold number, to obtain a trained machine customer service model; The historical communication text comprises historical dialogue text, and the user model comprises: The first understanding module is used for obtaining the first text output by the machine customer service model, and generating a first customer service action and a first customer service slot value according to the first text, the first customer service action being a predicted action of an initial customer service action, the initial customer service action being an action adopted by the machine customer service model for generating the first text, and the first customer service slot value being a predicted slot value of an initial customer service slot value, the initial customer service slot value being a slot value adopted by the machine customer service model for generating the first text; A plurality of first strategy modules are used for obtaining the historical dialogue text, and respectively simulating different users to generate a first user action and a first user slot value according to the first customer service action, the first customer service slot value and the historical dialogue text; The first natural text generation module is used for generating a corresponding first predicted text according to the first user action and the first user slot value, the first strategy module corresponding to the first predicted text one by one, and the first predicted text being a natural language of the first user slot value. The first natural text generation module comprises:

2. The machine customer service training system of claim 1, wherein, A changing unit is used for changing the first user slot value to obtain a second user slot value; A first natural text generation unit is used for generating the corresponding first predicted text according to the first user action and the second user slot value, the first predicted text being a natural language of the second user slot value. The historical communication text further comprises a historical customer service slot value, the historical customer service slot value being a customer service slot value adopted by the machine customer service model for historically generating a predicted text, and the machine customer service model comprising:

3. The machine customer service training system of claim 2, wherein, ​ a second understanding module, configured to generate a second user action and a third user slot value according to the historical dialogue text and the target prediction text, the second user action being a prediction of the first user action, and the third user slot value being a prediction of the second user slot value; a state tracking module, configured to determine a second customer service slot value according to the second user action, the third user slot value, and the historical customer service slot value, the second customer service slot value being a reply to the third user slot value; a second strategy module, configured to determine a second customer service action according to the second user action and the second customer service slot value, the second customer service action being a reply action to the second user action; a second natural text generation module, configured to generate a corresponding second prediction text according to the second customer service action and the second customer service slot value, the second prediction text being a natural language of the second customer service slot value.

4. The machine customer service training system according to any one of claims 1 to 3, characterized in that, The reward parameter configuration component is further configured to: determine a loss value of any two first prediction texts in the plurality of first prediction texts according to a preset loss function; if the loss value is less than a loss value threshold, configure a first negative reward parameter to the user model, the first negative reward parameter being used to adjust a model parameter of the user model.

5. The machine customer service training system according to any one of claims 1 to 3, characterized in that, The reward parameter configuration component is further configured to: if the second prediction text indicates that the current dialogue between the user model and the machine customer service model ends and fails, configure a second negative reward parameter to the machine customer service model and a second positive reward parameter to the user model, the second negative reward parameter being used to adjust a model parameter of the machine customer service model, and the second positive reward parameter being used to adjust a model parameter of the user model.

6. The machine customer service training system according to any one of claims 1 to 3, characterized by, The reward parameter configuration component is further configured to: obtain at least one historical prediction text of the second prediction text, and if the second prediction text is the same as the historical prediction text, configure a third negative reward parameter to the machine customer service model, the third negative reward parameter being used to adjust a model parameter of the machine customer service model, the historical prediction text being a prediction text historically output by the machine customer service model.

7. A machine customer service training method characterized by, The machine customer service training system and the machine customer service training method are applied to any one of claims 1 to 6, and the machine customer service training method comprises: the user model obtains a first text output by the machine customer service model and historical communication text of the first text, and generates a plurality of first prediction texts according to the first text and the historical communication text, the plurality of first prediction texts being replies to the first text, and the plurality of first prediction texts being simulated by the user model; the machine customer service model randomly determines one of the plurality of first prediction texts as a target prediction text, and generates a second prediction text according to the target prediction text and the historical communication text, the second prediction text being a reply to the target prediction text; The reward parameter configuration component configures a first positive reward parameter for the machine customer service model for adjusting model parameters of the machine customer service model when the second predicted text indicates that the current conversation of the user model and the machine customer service model ends successfully. The termination component determines a number of training times of the machine customer service model, and terminates training of the machine customer service model when the number of training times is greater than a threshold number of times, to obtain a trained machine customer service model.

8. A voice reply method, characterized by, The method comprises: obtaining user text information, the user text information being a result of speech recognition on user speech information; inputting the user text information into the machine customer service model trained in claim 7 for text processing to obtain reply text information; converting the reply text information into reply speech information and outputting the reply speech information to reply to the user speech information.

9. An electronic device, comprising: The computer program product comprises a processor, a memory, and a computer program stored on the memory and executable on the processor, and the processor implements the machine customer service training method of claim 7 or the speech reply method of claim 8 when executing the computer program. The reward parameter configuration component configures a first positive reward parameter for the machine customer service model for adjusting model parameters of the machine customer service model when the second predicted text indicates that the current conversation of the user model and the machine customer service model ends successfully. The termination component determines a number of training times of the machine customer service model, and terminates training of the machine customer service model when the number of training times is greater than a threshold number of times, to obtain a trained machine customer service model. The method comprises: obtaining user text information, the user text information being a result of speech recognition on user speech information; inputting the user text information into the machine customer service model trained in claim 7 for text processing to obtain reply text information; converting the reply text information into reply speech information and outputting the reply speech information to reply to the user speech information. The computer program product comprises a processor, a memory, and a computer program stored on the memory and executable on the processor, and the processor implements the machine customer service training method of claim 7 or the speech reply method of claim 8 when executing the computer program.

Images