Training methods for intelligent customer service models, model-based dialogue methods, and equipment

Abstract

The present disclosure provides a training method for an intelligent customer service model, a model-based dialogue method, and a device, which relate to technical fields such as deep learning and natural language processing. The training method of the intelligent customer service model includes: adding a scene placeholder to the dialogue context based on the dialogue scene to which the dialogue context belongs in the historical customer service dialogue; generating a sample dialogue sentence according to the dialogue context containing the scene placeholder, and using the reply sentence corresponding to the dialogue context as the annotated reply sentence; inputting the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and using the annotated reply sentence and the predicted reply sentence to fine-tune the pre-trained first generative language model to obtain a second generative language model; and determining the intelligent customer service model based on the second generative language model. The above technical solution can improve the reply quality of intelligent customer service.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of deep learning, natural language processing, etc. Specifically, it relates to a training method for an intelligent customer service model, a model-based dialogue method, and a device. Background Art

[0002] The business marketing consulting scenario refers to the dialogue between merchants and traffic users, which meets the needs of traffic users on the one hand and the conversion needs of merchants on the other. This scenario has long had several pain points, including high customer service labor costs, low efficiency of manual customer service, and lack of customer service for small and medium-sized customers. These problems lead to inefficient and low conversion of consulting services in business scenarios.

[0003] Therefore, how to improve the quality of consulting services is very important. Summary of the invention

[0004] The present invention provides a training method for an intelligent customer service model, a model-based dialogue method, and a device.

[0005] According to one aspect of the present disclosure, a method for training an intelligent customer service model is provided, comprising:

[0006] Add a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation;

[0007] Generate sample dialogue sentences according to the dialogue context containing scene placeholders, and use the reply sentences corresponding to the dialogue context as the annotated reply sentences;

[0008] Inputting the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and fine-tuning the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model;

[0009] An intelligent customer service model is determined based on the second generative language model.

[0010] According to one aspect of the present disclosure, a conversation method based on an intelligent customer service model is provided, comprising:

[0011] Based on the dialogue scene to which the current dialogue context belongs, add a scene placeholder to the current dialogue context;

[0012] Generate a current dialogue sentence according to the current dialogue context containing the scene placeholder, and input the current dialogue sentence into the second generative language model in the intelligent customer service model to obtain at least two current reply sentences;

[0013] For each current reply sentence, input the current dialogue sentence and the current reply sentence into the second feedback model in the intelligent customer service model to obtain the quality of the current reply sentence;

[0014] A target response sentence is selected from among the current response sentences based on the quality of each current response sentence.

[0015] According to one aspect of the present disclosure, a training device for an intelligent customer service model is provided, comprising:

[0016] A placeholder module is used to add a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation;

[0017] A sample generation module, used to generate sample dialogue sentences according to the dialogue context containing scene placeholders, and use the reply sentence corresponding to the dialogue context as the annotated reply sentence;

[0018] A generative model training module, configured to input the sample dialogue sentence into a pre-trained first generative language model, obtain a predicted reply sentence output by the first generative language model, and fine-tune the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model;

[0019] A customer service model module is used to determine an intelligent customer service model based on the second generative language model.

[0020] According to one aspect of the present disclosure, a conversation device based on an intelligent customer service model is provided, comprising:

[0021] A placeholder module, used to add a scene placeholder to the current dialogue context based on the dialogue scene to which the current dialogue context belongs;

[0022] A reply generation module, used to generate a current dialogue sentence according to the current dialogue context containing the scene placeholder, and input the current dialogue sentence into the second generative language model in the intelligent customer service model to obtain at least two current reply sentences;

[0023] A reply quality module, for each current reply statement, inputting the current dialogue statement and the current reply statement into a second feedback model in the intelligent customer service model to obtain the quality of the current reply statement;

[0024] The reply selection module is used to select a target reply sentence from each current reply sentence according to the quality of each current reply sentence.

[0025] According to another aspect of the present disclosure, there is provided an electronic device, the electronic device comprising:

[0026] at least one processor; and

[0027] a memory communicatively connected to the at least one processor; wherein,

[0028] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method provided by any embodiment of the present disclosure.

[0029] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method provided by any embodiment of the present disclosure.

[0030] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method provided according to any embodiment of the present disclosure is implemented.

[0031] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0033] Figure 1 is a flow chart of a training method for an intelligent customer service model provided according to an embodiment of the present disclosure;

[0034] Figure 2a is a flowchart of another training method for an intelligent customer service model provided according to an embodiment of the present disclosure;

[0035] Figure 2b and Figure 2c is a schematic diagram of a conversation context provided according to an embodiment of the present disclosure;

[0036] Figure 3a is a flowchart of another training method for an intelligent customer service model provided according to an embodiment of the present disclosure;

[0037] Figure 3b is a schematic diagram of a fine-tuning principle of a generation model provided according to an embodiment of the present disclosure;

[0038] Figure 4a is a flowchart of another training method for an intelligent customer service model provided according to an embodiment of the present disclosure;

[0039] Figure 4b and Figure 4c All of them are schematic diagrams of the fine-tuning principle of a feedback model provided according to an embodiment of the present disclosure;

[0040] Figure 5 is a flow chart of a conversation method based on an intelligent customer service model provided according to an embodiment of the present disclosure;

[0041] Figure 6 It is a structural schematic diagram of a training device for an intelligent customer service model provided according to an embodiment of the present disclosure;

[0042] Figure 7 is a structural diagram of a conversation device based on an intelligent customer service model provided according to an embodiment of the present disclosure;

[0043] Figure 8 It is a block diagram of an electronic device used to implement the training method of the intelligent customer service model or the conversation method based on the intelligent customer service model of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] Figure 1 1 is a flow chart of a method for training an intelligent customer service model according to an embodiment of the present disclosure. The method is applicable to the case where a generative language model is applied to intelligent customer service. The method can be executed by a training device for an intelligent customer service model, which can be implemented in software and / or hardware and can be integrated into an electronic device. Figure 1 As shown, the training method of the intelligent customer service model of this embodiment may include:

[0045] S101, adding a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation;

[0046] S102, generating a sample dialogue sentence according to the dialogue context containing the scene placeholder, and taking the reply sentence corresponding to the dialogue context as the marked reply sentence;

[0047] S103, inputting the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and fine-tuning the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model;

[0048] S104: Determine an intelligent customer service model based on the second generative language model.

[0049] In the consultation scenario of the object to be promoted, the traffic side can provide query requirements, and the owner of the object to be promoted can provide targeted replies, which can meet the query requirements of the traffic side on the one hand and the conversion requirements of the owner of the object to be promoted on the other hand. The reply statements in the historical customer service dialogue can be provided by the dialogue flow robot or by the manual customer service, which is not specifically limited in the embodiments of this application.

[0050] The historical customer service dialogue is a multi-round dialogue, including the question statement queried by the traffic side and the reply statement provided by the customer service. For each reply statement in the historical customer service dialogue, the conversation statement before the reply statement can be used as the dialogue context (context), and the reply statement can be used as the marked reply statement (response) corresponding to the dialogue context, that is, the dialogue context can include sub-question statements and / or sub-reply statements, and the dialogue context can include multiple sub-question statements and multiple sub-reply statements. Take the three-round dialogue Q1R1R2Q2R3 as an example, where Qi is the i-th question statement, Ri is the i-th reply statement, and i is a natural number. Q1 can be used as the dialogue context of R1, Q1R1 as the dialogue context of R2, and Q1R1R2Q2 as the dialogue context of R3. The dialogue context of R1 only includes sub-question statements, and the dialogue contexts of R2 and R3 include both sub-question statements and sub-reply statements.

[0051] Among them, the dialogue scene to which the dialogue context belongs is used to characterize the conversation stage to which the dialogue context belongs. A corresponding scene placeholder can be configured for each dialogue scene, and the scene placeholder can be a preset string. Taking the opening conversation stage as an example, the corresponding opening placeholder can be BOS. By introducing the scene placeholder into the dialogue context, the dialogue context has the attribute of the conversation stage, so that the intelligent customer service model has the communication ability in the complex environment of each dialogue scene, thereby improving the quality of intelligent customer service.

[0052] Specifically, for each reply statement in the historical customer service conversation, the reply statement can be used as the marked reply statement, and the conversation sentence before the reply statement can be used as the corresponding conversation context to determine the conversation scene to which the conversation context belongs, and the corresponding scene placeholder is added to the conversation context to obtain a conversation context containing the scene placeholder, and a sample conversation sentence is generated based on the conversation context containing the scene placeholder.

[0053] The sample dialogue sentence is input into the pre-trained first generative language model to obtain the predicted reply sentence output by the first generative language model; a loss function is constructed according to the predicted reply sentence and the labeled reply sentence, and the loss function is used to perform supervised fine-tuning training (Supervised FineTuning, SFT) on the pre-trained first generative language model to obtain a second generative language model; the second generative language model can be directly used as an intelligent customer service model, or a feedback model can be constructed based on the second generative sentence model to sort multiple reply sentences output by the second generative language model, and the second generative language model and the feedback model are both used as sub-models in the intelligent customer service model, that is, an intelligent customer service model including the second generative language model and the feedback model is obtained. By fine-tuning the pre-trained first generative language model to obtain the second generative language model, the huge time-consuming process of training the model from scratch can be avoided, thereby improving the efficiency of building the intelligent customer service model.

[0054] The technical solution provided by the disclosed embodiment obtains the conversation context and the corresponding reply sentences based on the historical customer service conversations, adds a scene placeholder to the conversation context according to the conversation scene to which the conversation context belongs, obtains the conversation context containing the scene placeholder, generates sample conversation sentences according to the conversation context containing the scene placeholder, uses the sample conversation sentences and the conversation context as training sets to fine-tune the pre-trained generative model, and obtains an intelligent customer service model based on the generative language model, which can not only improve the training efficiency of the intelligent customer service model, but also improve the service quality of the intelligent customer service model.

[0055] Figure 2a is a flow chart of another training method of an intelligent customer service model provided according to an embodiment of the present disclosure. Figure 2a The training method of the intelligent customer service model of this embodiment may include:

[0056] S201, obtaining a conversation context based on historical customer service conversations; the conversation context includes a sub-question statement and a sub-reply statement;

[0057] S202, for a sub-reply statement in the conversation context, determine the conversation scenario to which the sub-reply statement belongs, and if there is no corresponding sub-question statement before the sub-reply statement, add a scene placeholder for the sub-reply statement according to the conversation scenario. Add a scene placeholder to the conversation context based on the conversation scenario to which the conversation context belongs in the historical customer service conversation;

[0058] S203, generating a sample dialogue sentence according to the dialogue context containing the scene placeholder, and taking the reply sentence corresponding to the dialogue context as the annotated reply sentence;

[0059] S204, inputting the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and fine-tuning the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model;

[0060] S205: Determine an intelligent customer service model based on the second generative language model.

[0061] In an optional implementation, the scene placeholder is used to organize the dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

[0062] In the consulting scenario, it is necessary not only to meet the needs and experience of the traffic side, but also to achieve the goal of guiding conversion. Customer service continuous questions and follow-up questions are very common, and the generative dialogue systems in related technologies, such as chatGPT and Wenxinyiyan, are all strictly ordered alternating role dialogues, which brings great challenges to the implementation of the entire generative dialogue model. By refining the analysis of speech intervals and distinguishing dialogue scenarios (i.e., speech modes), generative intelligent customer service has the marketing ability in a variety of complex dialogue scenarios, getting rid of the strict alternating rounds of passive response settings of generative dialogue robots, so that the model greatly meets the needs of consulting scenarios.

[0063] Among them, the dialogue context may include sub-question sentences and sub-reply sentences. The sub-question sentences and sub-reply sentences in the dialogue context are divided into different dialogue stages to obtain the dialogue scenes to which the sub-question sentences and sub-reply sentences belong. For each sub-reply sentence in the dialogue context, it is determined whether the sub-reply sentence lacks a corresponding sub-question sentence. If so, a scene placeholder may be added before the sub-reply sentence according to the dialogue scene to which the sub-reply sentence belongs. Specifically, for the sub-reply sentence in the dialogue context, it is determined to which dialogue scene the sub-reply sentence belongs, and whether the sub-reply sentence lacks a sub-question sentence. If the sub-reply sentence lacks a sub-question sentence, a corresponding scene placeholder is added before the sub-reply sentence as a supplement to obtain a dialogue context containing a scene placeholder, that is, a dialogue context of alternating questions and answers.

[0064] By adding scene placeholders to sub-response statements that lack sub-question statements, the scene placeholders can be used as a supplement to the missing sub-question statements, thereby organizing the dialogue context into alternating dialogues between sub-question statements and sub-response statements, which is adapted to the generative dialogue model. In other words, by introducing scene placeholders to organize the dialogue context into alternating dialogues, the dialogue context is freed from the passive setting of strict alternating turns, and the flexibility of the dialogue context is improved, so that the dialogue model can meet the needs of marketing scenarios. Moreover, by introducing scene placeholders as supplementary sub-question statements, the logic of the dialogue scene can be learned, further improving the quality of the dialogue.

[0065] In an optional implementation, the dialogue scene includes at least one of the following: an opening scene, a guiding scene, or a follow-up scene, and each dialogue scene has a different scene placeholder, and the scene placeholder is used to enable the first generative language model to learn the language of the dialogue scene.

[0066] Among them, the opening scene is used for identity introduction and belongs to the welcome stage; the guiding scene is used to meet the query needs of the traffic side through continuous dialogue with the traffic side; the follow-up scene is used to continue to extend the dialogue and guide conversion. For example, the traffic side can be asked to stay in touch in the follow-up scene. The scene placeholders of each dialogue scene are different. By introducing different scene placeholders for each dialogue scene, the first generative large language model can learn the words corresponding to the dialogue scene, so that the second generative large language model can understand and output the corresponding dialogue scene, further improving the dialogue quality.

[0067] refer to Figure 2b Taking the dialogue context including welcome, sub-reply sentence 1, sub-question sentence 1, sub-reply sentence 2, sub-reply sentence 3, sub-question sentence 2, sub-reply sentence 4, and sub-reply sentence 5 as an example, the welcome and sub-reply sentence 1 can be divided into opening scene 21, sub-question sentence 1, sub-reply sentence 2, and sub-reply sentence 3 can be divided into guiding scene 22, and sub-reply sentence 5 can be divided into follow-up scene 23 according to the dialogue content and dialogue interval. Figure 2c , sub-reply statement 1, sub-reply statement 3, and sub-reply statement 5 all lack corresponding sub-question statements. An opening placeholder BOS can be added to sub-reply statement 1, an introductory placeholder EOS can be added to sub-reply statement 3, and an inquiry placeholder AOS can be added to sub-reply statement 5. It should be noted that the same scene placeholders can be used in different dialogue scenes. For example, the same placeholder BOS can be used in the opening scene and the inquiry scene, thereby reducing the amount of calculation for model fine-tuning and improving the efficiency of building the intelligent customer service model.

[0068] The technical solution provided by the disclosed embodiment, by introducing scene placeholders for sub-reply statements in the dialogue context, on the one hand, organizes the dialogue context into a dialogue of alternating sub-question statements and sub-reply statements, so that the dialogue context gets rid of the passive setting of strict alternating turns, thereby improving the flexibility of the dialogue context; on the other hand, it also enables the intelligent customer service model to learn the speech logic of each dialogue scenario, thereby further improving the quality of intelligent customer service.

[0069] Figure 3a is a flowchart of another training method of an intelligent customer service model provided according to an embodiment of the present disclosure. Figure 3a The training method of the intelligent customer service model of this embodiment may include:

[0070] S301, adding a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation;

[0071] S302, determining dialogue knowledge for the dialogue context based on owner-side information of the object to be promoted and traffic-side information corresponding to the dialogue context;

[0072] S303, taking the dialogue context and dialogue knowledge containing the scene placeholder as sample dialogue sentences, and taking the reply sentence corresponding to the dialogue context as the annotated reply sentence;

[0073] S304, inputting the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and fine-tuning the pre-trained first generative language model using the annotated reply sentence and the predicted reply sentence to obtain a second generative language model;

[0074] S305: Determine an intelligent customer service model based on the second generative language model.

[0075] The intelligent customer service dialogue model serves hundreds of thousands of merchants in different industries and businesses. Although the mainstream generative models in the industry can support role customization, they are generally only implemented in single personality scenarios. In actual complex businesses, how a single model can provide personalized and differentiated generative dialogue capabilities for hundreds of thousands of merchants in thousands of industries has become a major difficulty. We also need to think about how to make the model understand the business of a brand new customer, so that customers who have never used consulting can also use intelligent customer service. The disclosed embodiment proposes multi-dimensional knowledge slots based on the generative dialogue large model to create a customer service setting that is "one thousand faces for one thousand people".

[0076] Among them, the owner of the object to be promoted can be the merchant served by the intelligent customer service dialogue model, and the traffic side corresponding to the dialogue context can be the user with consulting needs. For the owner side of the object to be promoted and the traffic side corresponding to the dialogue context, knowledge slots of multiple dimensions are obtained respectively, and the business information of the owner side, the information of the traffic side, personalized role cognition, traffic business scope, etc. can be injected into the sample dialogue sentence to obtain a high-quality merchant portrait and a closed set of private domain information. By introducing the knowledge slots on the owner side during the training and reasoning process of the generative model, the generative model's cognition of different owners and different industries is strengthened, so that the model can learn the information differences in different personalized scenarios, so as to generate customer service capabilities that are more in line with the current scenario. Traffic business scope setting injection: During the training and reasoning process of the generative model, the traffic side and business point information are introduced as knowledge in the generative model input, so as to increase the perception of the business scope and optimize the model's migration ability in consulting new businesses.

[0077] Specifically, for the owner of the object to be promoted and the traffic side of the conversation context, the information of the corresponding knowledge slots is obtained to obtain the conversation knowledge, and the conversation knowledge is introduced into the sample conversation sentence, so that the sample conversation sentence includes not only the conversation context (context) containing the scene placeholder, but also the conversation knowledge (knowledge), thereby enhancing the intelligent conversation model's perception of the owner of the object to be promoted and the traffic side of the conversation context, further improving the quality of customer service, thereby better meeting the needs of the traffic side and improving the conversion rate of the object to be promoted. It should be noted that the owner side information and traffic side information used in the embodiments of the present disclosure must first obtain the use authority granted by the owner side and the traffic side, and obtain the consent of the corresponding parties before use.

[0078] refer to Figure 3b , using the owner-side information of the object to be promoted and the traffic-side information 31 corresponding to the conversation context, determine the conversation knowledge knowledge for the conversation context, and obtain the conversation context context and reply statement response based on the historical customer service conversation 32; using the conversation context and conversation knowledge to generate sample conversation sentences, using the corresponding reply statement response as the labeled reply sentence to obtain the training set of the first generative language model 33, using the training set to fine-tune the first generative language model 33 to obtain the second generative language model 34.

[0079] In an optional implementation, the owner-side information of the object to be promoted includes at least one of the following: the name, portrait, business scope or private domain information of the owner of the object to be promoted; the traffic-side information corresponding to the conversation context includes at least one of the following: the query statement on the traffic side or the business point to be queried.

[0080] The knowledge slots on the owner side of the object to be promoted may include at least one of the following: personalized role settings such as the name, portrait, special selling points, business scope settings, and private domain information of the owner of the object to be promoted. The business scope may include the main business, the industry to which it belongs, the primary industry, the secondary industry, etc. Taking the legal industry as an example, the business scope may be set to only do divorce litigation, not demolition litigation; the special selling points may be the number of years of employment, whether there are tertiary doctors to do the array, etc.; the private domain information may be the unit price of the object to be promoted. The portrait of the owner of the object to be promoted can be constructed using a large language model, which can be applied to the relevant businesses of various merchants (such as intelligent customer service, intelligent consulting, intelligent question and answer library, etc.). The portrait construction process can be as follows: input the prompt and the conversation content into the pre-trained large language model, output the portrait of the owner side, such as the main business, the name of the merchant, the special selling points, and the private domain information (price information). After the portrait is extracted, the business information and portrait of the merchant are aggregated to complete the construction of a high-quality merchant portrait and a closed set of private domain information.

[0081] Traffic side information may include query statements on the traffic side and business points that need to be queried, which are used to indicate the search intent and business scope on the traffic side, so as to avoid deviations in model processing. By introducing customer service side and industry side information as knowledge in the generative model input during the training and reasoning process of the generative model, the generative model's cognition of different customers and industries is strengthened, so that the model can learn the information differences in different personalized scenarios, thereby generating customer service capabilities that are more in line with the current scenario. In addition, by introducing traffic side and business point information as knowledge, the perception of the business scope is enhanced, and the model's migration capabilities to new consultations are optimized.

[0082] The technical solution provided by the disclosed embodiment determines the dialogue knowledge based on the owner side information of the object to be promoted and the flow side information corresponding to the dialogue context, and introduces the dialogue knowledge into the sample dialogue sentence, so that the sample dialogue sentence includes not only the dialogue context containing the scene placeholder, but also the dialogue knowledge), thereby enhancing the intelligent dialogue model's perception of the owner of the object to be promoted and the flow side of the dialogue context, and further improving the quality of customer service based on the generative dialogue big model. In other words, by introducing multi-dimensional knowledge slots, a customer service setting with "thousands of faces" is created.

[0083] In an optional implementation, the method further includes at least one of the following: using a conversation quality inspection model to determine the quality of historical customer service conversations, and screening the historical customer service conversations based on the quality; screening the historical customer service conversations based on the conversion effects of the historical customer service conversations; screening the historical customer service conversations based on the industry to which the to-be-promoted objects in the historical customer service conversations belong.

[0084] Due to the serious marketing attributes of advertising dialogue scenes, uneven customer service capabilities, and the existence of manufacturer robots and plug-in tools, the massive amount of historical customer service dialogues contains a large number of low-quality, repetitive words that ignore the traffic experience. In order to solve the problem of uneven historical dialogues, we continuously collect, clean and build high-quality marketing dialogue data sets to cut off the interference of low-quality data on the model from the source of the data.

[0085] In the disclosed embodiments, the conversation granularity filtering and sampling can be performed according to the conversation level quality in the following ways: a. Score a large amount of historical customer service conversations through a conversation quality inspection model to obtain the quality of the historical customer service conversations, and filter out historical customer service conversations with quality below a threshold value; b. Screen the historical customer service conversations according to the a posteriori conversion effect, the actual speech situation on the traffic side, etc. For example, filter out historical customer service conversations with poor conversion effects. If the conversion effect is good, but the traffic side is directly connected, filtering is also required; c. For different industry attributes, sample the high-quality conversations at the head in proportion and send them to the next link, so as to balance the samples of each industry and enable the customer service conversation model to be applied to different industries.

[0086] In an optional embodiment, the method further includes at least one of the following: determining the similarity between different sub-reply statements in the conversation context, and deduplicating each sub-reply statement based on the similarity; based on the reply rule of the large language model, selecting a sub-reply statement that does not comply with the reply rule from the conversation context, and filtering out the sub-reply statement.

[0087] In the disclosed embodiment, the reply statements in the historical customer service dialogue can be further processed to determine whether the reply statements are suitable as the content of the model learning. The sub-reply statements in the historical customer service dialogue can be filtered in the following ways: a. Determine the dialogue scene to which the sub-reply statement belongs, sample different dialogue scenes in proportion, and balance the sub-reply statements of each dialogue scene; determine the similarity between different sub-reply statements, and deduplicate the sub-reply statements according to the similarity; based on the reply rules of the large language model, filter out the sub-reply statements that do not meet the reply rules, that is, filter out the sub-reply statements that are not suitable as model output, for example, "I will call you directly" in the manual customer service scene is not suitable as model output. By deduplicating and filtering the reply statements in the historical customer service replies, the quality of the retained historical customer service dialogues can be further improved, thereby improving the quality of the training set of the intelligent customer service model, and improving the construction efficiency and prediction quality of the intelligent customer service model.

[0088] Figure 4a is a flowchart of another training method of an intelligent customer service model provided according to an embodiment of the present disclosure. Figure 4a The training method of the intelligent customer service model of this embodiment may include:

[0089] S401, adding a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation;

[0090] S402, generating a sample dialogue sentence according to the dialogue context containing the scene placeholder, and taking the reply sentence corresponding to the dialogue context as the marked reply sentence;

[0091] S403, inputting the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and fine-tuning the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model;

[0092] S404, inputting the sample dialogue sentence into a second generative language model to obtain at least two predicted reply sentences;

[0093] S405, constructing a training set for a first feedback model according to the ranking information between at least two predicted reply sentences; the first feedback model is obtained by adding a fully connected layer to the second generative language model;

[0094] S406, fine-tuning the first feedback model using the training set to obtain a second feedback model;

[0095] S407: Use the second generative language model and the second feedback model as sub-models in an intelligent customer service model.

[0096] The first feedback model can be used to score the reply sentences generated by the second generative language model by adding a fully connected layer at the end of the second generative language model as the output layer of the first feedback model. The second generative language model may include multiple layers of transformers, the input of the second generative language model is the dialogue context, and the output is the reply sentence. A fully connected layer is added at the end of the second generative language model as the first feedback model, and the first feedback model can be hot-started based on the second generative language model.

[0097] Specifically, for any sample dialogue sentence, the sample dialogue sentence can be input into the second generative language model multiple times to obtain multiple predicted reply sentences; the predicted reply sentences are sorted based on the scoring rules and the sorting rules to obtain the order between different predicted reply sentences. Among them, the scoring rules and the sorting rules can be analyzed by the business dialogue to uniformly summarize the business standards from the perspective of user experience and conversion guidance, including three different perspectives such as rationality, guidance and marketing. According to the sorting information between each predicted reply sentence, a training set is constructed for the first feedback model, and the first feedback model is fine-tuned and trained using the training set to obtain the second feedback module, and the second generative language model and the second feedback model are used as sub-models in the intelligent customer service model, that is, an intelligent customer service model including the second generative language model and the second feedback model is obtained. By hot starting the first feedback model on the basis of the second generative language model, and constructing a training set for the first feedback model based on the sorting information between the predicted reply sentences, and fine-tuning the first feedback model using the training set to obtain the second feedback model, the sorting accuracy of the second feedback model for each predicted reply sentence can be improved, thereby further improving the quality of intelligent customer service.

[0098] refer to Figure 4b , fine-tune the first generative language model using sample dialogue sentences and annotated response sentences to obtain a second generative language model; construct a first feedback model (not shown) based on the second generative language model, input the sample dialogue sentences into the second generative language model multiple times to obtain multiple predicted response sentences R1, R2, and R3, and obtain the order between the predicted response sentences, construct a training set for the first feedback model according to the order between the predicted response sentences, and use the training set to fine-tune the first feedback model to obtain the second feedback model.

[0099] Combination Figure 4b and Figure 4c After building the second feedback model, the output of the second feedback model can also be sampled, such as by performing rejection sampling, to obtain a new training set for the generative language model, and the new training set can be used to fine-tune the second generative language model until the second generative language model and the second feedback model meet the requirements of the marketing scenario.

[0100] In an optional embodiment, the training set includes sample pairs, the sample pairs include positive samples and negative samples, the positive samples include sample dialogue sentences and first predicted response sentences, the negative samples include sample dialogue sentences and second predicted response sentences, and the first predicted response sentence is ranked better than the second predicted response sentence.

[0101] By introducing positive samples and negative samples into the training set of the first feedback model, the gap between the quality scores of positive samples and negative samples is widened during the fine-tuning of the first feedback model. In other words, the relative quality scores between positive samples and negative samples are learned during the fine-tuning of the first feedback model to obtain the second feedback model, thereby improving the accuracy of the second feedback model.

[0102] Specifically, for any sample dialogue sentence, the second generative language module can output multiple predicted reply sentences, sort the predicted reply sentences, and construct positive and negative sample pairs for the first feedback model based on the sorting information. Taking the priority order of multiple predicted reply sentences as R3>R1>R5>…>R1 as an example,<R3,R1> They are used as the first predicted reply sentence and the second predicted reply sentence respectively, that is, the sample dialogue sentence + R3 is used as the positive sample in the sample pair, and the sample dialogue sentence + R1 is used as the negative sample in the sample pair. The sample dialogue sentence + R3 is input into the first feedback model to obtain the first quality score, and the sample dialogue sentence and R1 are input into the first feedback model to obtain the second quality score. The first feedback model is fine-tuned and updated according to the difference between the first quality score and the second quality score to obtain the second feedback model.

[0103] In an optional embodiment, the training set includes sample triplets, the sample triplets include a first positive sample, a second positive sample and a negative sample; the first positive sample includes a sample dialogue sentence and a first predicted reply sentence, the second positive sample includes a second sample dialogue sentence and a second predicted reply sentence, and the negative sample includes a sample dialogue sentence and a third predicted reply sentence; the ranking of the first predicted reply sentence is the same as the ranking of the second predicted reply sentence, and is better than the ranking of the third predicted reply sentence.

[0104] The sample dialogue sentence may have multiple reply sentences with similar meaning and quality, and these reply sentences can be classified into one category. The training set of the first feedback model can be a sample triple, which includes two positive samples and one negative sample. Among them, the first predicted reply sentence in the first positive sample and the second predicted reply sentence in the second positive sample have the same ranking, and both are better than the third predicted reply sentence in the negative sample.

[0105] Specifically, all sample triplets can be input into the first feedback model, so that the first feedback model outputs the quality score of each predicted reply sentence respectively, that is, a first quality score, a second quality score, and a third quality score are obtained; by narrowing the gap between the first quality score and the second quality score and widening the gap between the first quality score and the third quality score, the distance within the class is made smaller than the distance between classes, thereby further improving the sorting accuracy of the second feedback model.

[0106] Since the intelligent customer service model has carried out very detailed business modeling for the dialogue in the marketing scenario, it has built a set of excellent generative model capabilities for the consulting dialogue field from various angles such as data, knowledge, capabilities, and effects. On this basis, it has completed the construction of an online generative intelligent customer service system and launched a new generation of generative customer service robots, leading the industry in realizing the generational change of intelligent customer service. Intelligent customer service has the following characteristics: 1. Strong dialogue flexibility: Compared with general intelligent customer service systems, this intelligent customer service is built based on a generative model and can generate responses in real time according to the context; 2. High dialogue quality: The intelligent customer service model is based on a multi-round dialogue model with knowledge internalization, and through high-quality data sampling, it completes multiple rounds of training, effectively ensuring the quality of the dialogue; 3. High scene fit: The intelligent customer service model can effectively perceive the traffic attributes of commercial dialogues and personalized merchant information, and has marketing capabilities in complex environments such as opening, continuous speaking, and silent questioning; 4. High business achievement: Through a large amount of manually labeled data to train the feedback model, the overall system completes the alignment of business goals through the feedback model, so that the intelligent customer service has very good business results and can take into account both user experience and conversion guidance. Various experiments have confirmed that the service quality and conversion effect of this intelligent customer service are better than the average level of dialogue flow robots and manual customer service in the same scenario.

[0107] The technical solution provided by the embodiment of the present disclosure obtains a second feedback model by hot-starting the first feedback model based on the second generative language model, and constructing sample pairs and sample triples for the first feedback model to fine-tune the first feedback model, so that the second feedback model can accurately sort the predicted reply sentences, thereby improving the quality of the intelligent customer service model.

[0108] Figure 5 1 is a flow chart of a conversation method based on an intelligent customer service model provided according to an embodiment of the present disclosure. The method is applicable to the case where intelligent customer service is provided based on a generative language model. The method can be executed by a conversation device based on an intelligent customer service model, which can be implemented in software and / or hardware and can be integrated into an electronic device. Figure 5 As shown, the conversation method based on the intelligent customer service model of this embodiment may include:

[0109] S501, adding a scene placeholder to the current dialogue context based on the dialogue scene to which the current dialogue context belongs;

[0110] S502, generating a current dialogue sentence according to the current dialogue context containing the scene placeholder, and inputting the current dialogue sentence into a second generative language model in the intelligent customer service model to obtain at least two current reply sentences;

[0111] S503, for each current reply sentence, input the current dialogue sentence and the current reply sentence into a second feedback model in the intelligent customer service model to obtain the quality of the current reply sentence;

[0112] S504: Select a target response sentence from each current response sentence according to the quality of each current response sentence.

[0113] Among them, the current dialogue context may be a question statement or a reply statement that has existed in multiple rounds of dialogue. The dialogue scene to which the current dialogue context belongs is used to characterize the conversation stage to which the current dialogue context belongs. A corresponding scene placeholder may be configured for each dialogue scene, and the scene placeholder may be a preset string. By introducing the scene placeholder into the current dialogue context, the current dialogue context has a conversation stage attribute. The current dialogue statement is generated according to the current dialogue context containing the scene placeholder, and the current dialogue statement may be input into the second generative language model in the intelligent customer service model for multiple times to obtain at least two current reply statements; for each current reply statement, the current dialogue statement and the current reply statement are used as inputs to the second feedback model in the intelligent customer service model to obtain the quality score of the current reply statement; the quality scores of the current reply statements are compared, and the current reply statement with the highest quality score is selected as the target reply statement that the intelligent customer service model responds to the current dialogue context.

[0114] The technical solution provided by the embodiment of the present disclosure can improve the response quality of the intelligent customer service by adopting the second generative language model in the intelligent customer service model to generate multiple predicted reply sentences for the current dialogue sentence, and adopting the second feedback model in the intelligent customer service model to perform quality scoring on the multiple predicted reply sentences, and selecting the target reply sentence from the multiple predicted reply sentences according to the quality scoring results.

[0115] In an optional implementation, the scene placeholder is used to organize the current dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

[0116] By introducing scene placeholders to organize the current dialogue context into alternating dialogues, the current dialogue context is freed from the passive setting of strict alternating turns, which improves the flexibility of the current dialogue context and enables the dialogue model to meet the needs of marketing scenarios.

[0117] In an optional implementation, the dialogue scene includes at least one of the following: an opening scene, a guide scene, or a follow-up scene, and the scene placeholders of each dialogue scene are different.

[0118] Among them, the opening scene is used for identity introduction and belongs to the welcome stage; the guiding scene meets the query needs of the traffic side through continuous dialogue with the traffic side; the follow-up scene is used to continue to extend the dialogue and guide conversion. For example, the traffic side can be asked to stay in touch in the follow-up scene. The scene placeholders of each dialogue scene are different. By introducing the scene placeholders of the opening scene, guiding scene or follow-up scene in the previous dialogue context, the intelligent customer service model has the marketing capabilities in the complex environment of each dialogue scene, which can improve the quality of dialogue.

[0119] In an optional implementation, adding a scene placeholder to the current dialogue context based on the dialogue scene to which the current dialogue context belongs includes: determining the dialogue scene to which a sub-dialogue statement belongs based on the pause duration between different sub-dialogue statements in the current dialogue context; and adding a scene placeholder to the sub-dialogue statement based on the dialogue scene.

[0120] In the disclosed embodiment, by analyzing the pause duration between different sub-dialogue sentences in the current dialogue context, it is determined whether the sub-dialogue sentence belongs to a guiding scenario or a questioning scenario, and a scene placeholder of the corresponding scenario is added to the sub-dialogue sentence. By adding a scene placeholder to the sub-dialogue sentence, the intelligent customer service model can use the speech logic corresponding to the scene placeholder to reply, thereby improving the reply quality.

[0121] In an optional implementation, generating a current dialogue statement based on a current dialogue context containing scene placeholders includes: determining current dialogue knowledge for the current dialogue context based on owner-side information of the object to be promoted and traffic-side information corresponding to the current dialogue context; and using the current dialogue context containing scene placeholders and the current dialogue knowledge as the current dialogue statement.

[0122] Obtain knowledge slots of multiple dimensions for the owner side of the object to be promoted and the traffic side corresponding to the conversation context. By introducing knowledge slots on the owner side in the reasoning process of the generative model, the generative model's cognition of different owners and different industries is strengthened, so that the model can learn the information differences in different personalized scenarios, thereby generating customer service capabilities that are more in line with the current scenario. Traffic business scope setting injection: By introducing traffic side and business point information as knowledge, the business scope perception is increased and strengthened, and the model's migration capabilities in consulting new businesses are optimized.

[0123] In an optional implementation, the owner-side information of the object to be promoted includes at least one of the following: the name, portrait, business scope or private domain information of the owner of the object to be promoted; the traffic-side information corresponding to the current conversation context includes at least one of the following: a query statement on the traffic side or a business point to be queried.

[0124] Among them, the knowledge slots on the owner side of the object to be promoted may include at least one of the following: personalized role settings such as the name, portrait, special selling points, business scope settings, and private domain information of the owner of the object to be promoted. By introducing customer service and industry information as knowledge in the generation model input, the generation model's cognition of different customers and industries is strengthened, so that the model can respond based on personalized scenarios, thereby improving the matching between the response and the personalized scenario.

[0125] Figure 6 1 is a schematic diagram of a structure of a training device for an intelligent customer service model provided according to an embodiment of the present disclosure. This embodiment is applicable to the case where a generative language model is applied to intelligent customer service. The device can be implemented in software and / or hardware. Figure 6 As shown, the training device 600 of the intelligent customer service model of this embodiment may include:

[0126] A placeholder module 610, configured to add a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation;

[0127] A sample generation module 620, for generating a sample dialogue sentence according to the dialogue context containing the scene placeholder, and using the reply sentence corresponding to the dialogue context as the marked reply sentence;

[0128] A generative model training module 630 is used to input the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and fine-tune the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model;

[0129] The customer service model module 640 is used to determine an intelligent customer service model based on the second generative language model.

[0130] In an optional implementation, the scene placeholder is used to organize the dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

[0131] In an optional implementation, the dialogue scene includes at least one of the following: an opening scene, a guiding scene, or a follow-up scene, and each dialogue scene has a different scene placeholder, and the scene placeholder is used to enable the first generative language model to learn the language of the dialogue scene.

[0132] In an optional implementation, the placeholder module 610 includes:

[0133] A context unit, used to obtain a conversation context based on historical customer service conversations; the conversation context includes a sub-question statement and a sub-reply statement;

[0134] The placeholder unit is used to determine the dialogue scene to which the sub-reply statement belongs in the dialogue context, and if there is no corresponding sub-question statement before the sub-reply statement, add a scene placeholder for the sub-reply statement according to the dialogue scene.

[0135] In an optional implementation, the sample generation module 620 includes:

[0136] A knowledge unit, configured to determine dialog knowledge for the dialog context based on owner-side information of the object to be promoted and traffic-side information corresponding to the dialog context;

[0137] The sample generation unit is used to use the dialogue context containing the scene placeholder and the dialogue knowledge as the sample dialogue sentence, and use the reply sentence corresponding to the dialogue context as the labeled reply sentence.

[0138] In an optional implementation, the owner-side information of the object to be promoted includes at least one of the following: the name, portrait, business scope or private domain information of the owner of the object to be promoted; the traffic-side information corresponding to the conversation context includes at least one of the following: the query statement on the traffic side or the business point to be queried.

[0139] In an optional implementation, the training device 600 of the intelligent customer service model further includes a first dialogue screening module, and the first dialogue screening module includes at least one of the following:

[0140] A quality screening unit, configured to determine the quality of historical customer service conversations using a conversation quality inspection model, and screen the historical customer service conversations according to the quality;

[0141] A conversion screening unit, used to screen the historical customer service conversations according to their conversion effects;

[0142] The industry screening unit is used to screen the historical customer service conversations according to the industries to which the objects to be promoted in the historical customer service conversations belong.

[0143] In an optional implementation, the training device 600 of the intelligent customer service model further includes a second dialogue screening module, and the second dialogue screening module includes at least one of the following:

[0144] A deduplication unit, used to determine the similarity between different sub-response sentences in the conversation context, and to deduplicate each sub-response sentence according to the similarity;

[0145] The rule filtering unit is used to select sub-reply statements that do not conform to the reply rule from the dialogue context based on the reply rule of the large language model, and filter out the sub-reply statement.

[0146] In an optional implementation, the customer service model module 640 includes:

[0147] A prediction response unit, configured to input the sample dialogue sentence into a second generative language model to obtain at least two prediction response sentences;

[0148] A training set construction unit, configured to construct a training set for a first feedback model according to the sorting information between at least two predicted reply sentences; the first feedback model is obtained by adding a fully connected layer to the second generative language model;

[0149] A feedback fine-tuning unit, configured to use the training set to perform fine-tuning training on the first feedback model to obtain a second feedback model;

[0150] The customer service model unit is used to use the second generative language model and the second feedback model as sub-models in an intelligent customer service model.

[0151] In an optional embodiment, the training set includes sample pairs, the sample pairs include positive samples and negative samples, the positive samples include sample dialogue sentences and first predicted response sentences, the negative samples include sample dialogue sentences and second predicted response sentences, and the first predicted response sentence is ranked better than the second predicted response sentence.

[0152] In an optional implementation, the training set includes a sample triplet, and the sample triplet includes a first positive sample, a second positive sample, and a negative sample;

[0153] The first positive sample includes a sample dialogue sentence and a first predicted reply sentence, the second positive sample includes a second sample dialogue sentence and a second predicted reply sentence, and the negative sample includes a sample dialogue sentence and a third predicted sentence;

[0154] The ranking of the first predicted reply sentence is the same as the ranking of the second predicted reply sentence, and is better than the ranking of the third predicted reply sentence.

[0155] The technical solution of the disclosed embodiment introduces scene placeholders into the sample according to the conversation scene, so that the dialogue model can learn the conversation logic of each conversation scene, and frees the conversation context from the passive setting of strict alternating rounds, thereby improving the flexibility of the conversation context; by introducing knowledge slots based on the owner-side information of the object to be promoted and the traffic-side information corresponding to the conversation context, the generation model's cognition of different owners and different industries is strengthened, so that the dialogue model can learn the information differences in different personalized scenarios; in addition, sample triplets can be introduced in the fine-tuning process of the first feedback model to further improve the accuracy of the quality score.

[0156] Figure 71 is a schematic diagram of a dialog device based on an intelligent customer service model according to an embodiment of the present disclosure. This embodiment is applicable to the case where intelligent customer service is provided based on a generative language model. The device can be implemented in software and / or hardware. Figure 7 As shown, the dialogue device 700 based on the intelligent customer service model of this embodiment may include:

[0157] A placeholder module 710, configured to add a scene placeholder to the current dialogue context based on the dialogue scene to which the current dialogue context belongs;

[0158] A reply generation module 720 is used to generate a current dialogue sentence according to the current dialogue context containing the scene placeholder, and input the current dialogue sentence into the second generative language model in the intelligent customer service model to obtain at least two current reply sentences;

[0159] The reply quality module 730 is used to input the current dialogue sentence and the current reply sentence into the second feedback model in the intelligent customer service model for each current reply sentence, and obtain the quality of the current reply sentence;

[0160] The reply selection module 740 is used to select a target reply sentence from each current reply sentence according to the quality of each current reply sentence.

[0161] In an optional implementation, the scene placeholder is used to organize the current dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

[0162] In an optional implementation, the dialogue scene includes at least one of the following: an opening scene, a guide scene, or a follow-up scene, and the scene placeholders of each dialogue scene are different.

[0163] In an optional implementation, the placeholder module 710 includes:

[0164] A dialogue scene unit, used to determine the dialogue scene to which a sub-dialogue sentence belongs according to the pause duration between different sub-dialogue sentences in the current dialogue context;

[0165] A placeholder unit is used to add a scene placeholder to the sub-dialogue sentence according to the dialogue scene.

[0166] In an optional implementation, the reply generation module 720 is specifically configured to:

[0167] Determining current conversation knowledge for the current conversation context based on owner-side information of the object to be promoted and traffic-side information corresponding to the current conversation context;

[0168] The current dialogue context containing scene placeholders and the current dialogue knowledge are used as the current dialogue sentence.

[0169] In an optional implementation, the owner-side information of the object to be promoted includes at least one of the following: the name, portrait, business scope or private domain information of the owner of the object to be promoted; the traffic-side information corresponding to the current conversation context includes at least one of the following: a query statement on the traffic side or a business point to be queried.

[0170] The technical solution of the disclosed embodiment introduces scene placeholders into the sample according to the conversation scene, so that the dialogue model can learn the conversation logic of each conversation scene, and frees the conversation context from the passive setting of strict alternating rounds, thereby improving the flexibility of the conversation context; by introducing knowledge slots based on the owner-side information of the object to be promoted and the traffic-side information corresponding to the conversation context, the generation model's cognition of different owners and different industries is strengthened, so that the dialogue model can learn the information differences in different personalized scenarios; in addition, sample triplets can be introduced in the fine-tuning process of the first feedback model to further improve the accuracy of the quality score.

[0171] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0172] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0173] Figure 8 It is a block diagram of an electronic device used to implement the training method of the intelligent customer service model or the conversation method based on the intelligent customer service model of the embodiment of the present disclosure.

[0174] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0175] like Figure 8As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0176] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0177] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), an image processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a training method for an intelligent customer service model or a conversation method based on an intelligent customer service model. For example, in some embodiments, the training method for an intelligent customer service model or a conversation method based on an intelligent customer service model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the training method for the intelligent customer service model or the conversation method based on the intelligent customer service model described above may be executed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute a training method for an intelligent customer service model or a conversation method based on an intelligent customer service model in any other appropriate manner (for example, by means of firmware).

[0178] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0179] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0180] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0181] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0182] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0183] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0184] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.

[0185] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.

[0186] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0187] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for an intelligent customer service model, comprising: Based on the conversation scene to which the conversation context belongs in the historical customer service conversation, a scene placeholder is added to the conversation context; wherein the conversation scene to which the conversation context belongs is used to characterize the conversation stage to which the conversation context belongs; Generate sample dialogue sentences according to the dialogue context containing scene placeholders, and use the reply sentences corresponding to the dialogue context as the annotated reply sentences; Inputting the sample dialogue sentence into a pre-trained first generative language model to obtain a predicted reply sentence output by the first generative language model, and fine-tuning the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model; Determining an intelligent customer service model based on the second generative language model; The scene placeholder is used to organize the dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

2. The method according to claim 1, wherein: The dialogue scene includes at least one of the following: an opening scene, a guiding scene or a follow-up scene, and the scene placeholders of each dialogue scene are different, and the scene placeholders are used to enable the first generative language model to learn the words of the dialogue scene.

3. The method according to any one of claims 1 to 2, wherein: The adding a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation includes: Acquire a conversation context based on historical customer service conversations; the conversation context includes a sub-question statement and a sub-reply statement; For a sub-reply statement in the dialogue context, determine the dialogue scene to which the sub-reply statement belongs. If there is no corresponding sub-question statement before the sub-reply statement, add a scene placeholder for the sub-reply statement according to the dialogue scene.

4. The method according to claim 1, wherein: The step of generating a sample dialogue sentence according to the dialogue context containing the scene placeholder and using the reply sentence corresponding to the dialogue context as the marked reply sentence includes: Determining conversation knowledge for the conversation context based on owner-side information of the object to be promoted and traffic-side information corresponding to the conversation context; The dialogue context and dialogue knowledge containing scene placeholders are used as sample dialogue sentences, and the reply sentences corresponding to the dialogue context are used as labeled reply sentences.

5. The method according to claim 4, wherein: The owner-side information of the object to be promoted includes at least one of the following: the name, portrait, business scope or private domain information of the owner of the object to be promoted; the traffic-side information corresponding to the conversation context includes at least one of the following: the query statement on the traffic side or the business point to be queried.

6. The method according to claim 1, further comprising at least one of the following: Using a conversation quality inspection model to determine the quality of historical customer service conversations, and screening the historical customer service conversations based on the quality; Filter the historical customer service conversations according to their conversion effects; The historical customer service conversations are screened according to the industries to which the to-be-promoted objects in the historical customer service conversations belong.

7. The method according to claim 1, further comprising at least one of the following: Determine the similarity between different sub-response statements in the conversation context, and remove duplicates from each sub-response statement based on the similarity; Based on the response rules of the large language model, sub-response sentences that do not conform to the response rules are selected from the dialogue context and filtered out.

8. The method according to claim 1, wherein: The determining of the intelligent customer service model based on the second generative language model includes: Inputting the sample dialogue sentence into the second generative language model to obtain at least two predicted response sentences; constructing a training set for a first feedback model according to the ranking information between at least two predicted reply sentences; the first feedback model is obtained by adding a fully connected layer to the second generative language model; Using the training set to fine-tune the first feedback model to obtain a second feedback model; The second generative language model and the second feedback model are used as sub-models in the intelligent customer service model.

9. The method according to claim 8, wherein: The training set includes sample pairs, the sample pairs include positive samples and negative samples, the positive samples include sample dialogue sentences and first predicted response sentences, the negative samples include sample dialogue sentences and second predicted response sentences, and the first predicted response sentence is ranked better than the second predicted response sentence.

10. The method according to claim 8, wherein: The training set includes a sample triplet, and the sample triplet includes a first positive sample, a second positive sample, and a negative sample; The first positive sample includes a sample dialogue sentence and a first predicted reply sentence, the second positive sample includes a second sample dialogue sentence and a second predicted reply sentence, and the negative sample includes a sample dialogue sentence and a third predicted sentence; The ranking of the first predicted reply sentence is the same as the ranking of the second predicted reply sentence, and is better than the ranking of the third predicted reply sentence.

11. Dialogue methods based on intelligent customer service models, including: Based on the dialogue scene to which the current dialogue context belongs, adding a scene placeholder to the current dialogue context; wherein the dialogue scene to which the dialogue context belongs is used to characterize the conversation stage to which the dialogue context belongs; Generate a current dialogue sentence according to the current dialogue context containing the scene placeholder, and input the current dialogue sentence into the second generative language model in the intelligent customer service model to obtain at least two current reply sentences; For each current reply sentence, input the current dialogue sentence and the current reply sentence into the second feedback model in the intelligent customer service model to obtain the quality of the current reply sentence; Selecting a target response sentence from each current response sentence according to the quality of each current response sentence; The scene placeholder is used to organize the current dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

12. The method according to claim 11, wherein: The dialogue scene includes at least one of the following: an opening scene, a guiding scene or a questioning scene, and the scene placeholders of each dialogue scene are different.

13. The method according to any one of claims 11 to 12, wherein: The adding a scene placeholder to the current dialogue context based on the dialogue scene to which the current dialogue context belongs includes: Determine the dialogue scene to which the sub-dialogue sentence belongs according to the pause duration between different sub-dialogue sentences in the current dialogue context; A scene placeholder is added to the sub-dialogue statement according to the dialogue scene.

14. The method according to claim 11, wherein: The generating of the current dialogue sentence according to the current dialogue context containing the scene placeholder comprises: Determining current conversation knowledge for the current conversation context based on owner-side information of the object to be promoted and traffic-side information corresponding to the current conversation context; The current dialogue context containing scene placeholders and the current dialogue knowledge are used as the current dialogue sentence.

15. The method according to claim 14, wherein: The owner-side information of the object to be promoted includes at least one of the following: the name, portrait, business scope or private domain information of the owner of the object to be promoted; the traffic-side information corresponding to the current conversation context includes at least one of the following: the query statement on the traffic side or the business point to be queried.

16. A training device for an intelligent customer service model, comprising: A placeholder module, used to add a scene placeholder to the conversation context based on the conversation scene to which the conversation context belongs in the historical customer service conversation; wherein the conversation scene to which the conversation context belongs is used to characterize the conversation stage to which the conversation context belongs; A sample generation module, used to generate sample dialogue sentences according to the dialogue context containing scene placeholders, and use the reply sentence corresponding to the dialogue context as the annotated reply sentence; A generative model training module, configured to input the sample dialogue sentence into a pre-trained first generative language model, obtain a predicted reply sentence output by the first generative language model, and fine-tune the pre-trained first generative language model using the labeled reply sentence and the predicted reply sentence to obtain a second generative language model; A customer service model module, used to determine an intelligent customer service model based on the second generative language model; The scene placeholder is used to organize the current dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

17. A conversation device based on an intelligent customer service model, comprising: A placeholder module, used to add a scene placeholder to the current dialogue context based on the dialogue scene to which the current dialogue context belongs; wherein the dialogue scene to which the dialogue context belongs is used to characterize the conversation stage to which the dialogue context belongs; A reply generation module, used to generate a current dialogue sentence according to the current dialogue context containing the scene placeholder, and input the current dialogue sentence into the second generative language model in the intelligent customer service model to obtain at least two current reply sentences; A reply quality module, for each current reply statement, inputting the current dialogue statement and the current reply statement into a second feedback model in the intelligent customer service model to obtain the quality of the current reply statement; A reply selection module, used for selecting a target reply sentence from each current reply sentence according to the quality of each current reply sentence; The scene placeholder is used to organize the current dialogue context into a dialogue that alternates between sub-question statements and sub-response statements.

18. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 15.

19. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-15.

20. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 15.

Images