Intelligent customer service model optimization method, guess what you want to ask method, device and electronic equipment

Abstract

The present invention relates to the field of artificial intelligence technology, and provides an intelligent customer service model optimization method, a guessing what you want to ask method, a device, and an electronic device. First, multiple user features and their corresponding historical conversations, historical consultation forms, and historical intent sets are obtained; then, each user feature is used as a pending user feature, and based on the pending historical conversations and pending historical consultation forms corresponding to the pending user feature, a candidate intent set corresponding to the pending user feature is obtained; and based on the historical intent set and the candidate intent set corresponding to the pending user feature, the actual intent set corresponding to the pending user feature is determined, and the actual intent set corresponding to each user feature is obtained; finally, based on each user feature and its corresponding actual intent set, the intelligent customer service model to be optimized is trained and optimized to obtain an intelligent customer service model. This can reduce the interference of erroneous intents on the model, improve the performance of the intelligent customer service model, and improve the accuracy of model prediction.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent customer service model optimization method, a guessing what you want to ask method, a device, and an electronic device. Background Art

[0002] Currently, intelligent customer service models are typically optimized based on the intent of user clicks or input questions. However, this approach can mislead the intelligent customer service model by capturing incorrect intent due to user errors. Furthermore, some users skip the intelligent customer service and go directly to human customer service. This prevents the model from capturing the user's true intent, resulting in poor performance and reduced prediction accuracy. Summary of the Invention

[0003] In view of this, the object of the present invention is to provide an intelligent customer service model optimization method, a guessing what you want to ask method, device and electronic device.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] In a first aspect, the present invention provides a method for optimizing an intelligent customer service model, the method comprising:

[0006] Obtain multiple user features and their corresponding historical conversations, historical consultation tickets, and historical intent sets; wherein the historical conversations represent the text content entered by the user to the manual customer service; the historical consultation tickets represent the user's consultation information recorded by the manual customer service; and the historical intent set represents the collection of intents that the user has encountered on the intelligent customer service page;

[0007] Taking each of the user features as a pending user feature, and obtaining a candidate intent set corresponding to the pending user feature based on the pending historical conversations and pending historical consultation orders corresponding to the pending user feature;

[0008] Determine the actual intent set corresponding to the pending user feature based on the historical intent set and the candidate intent set corresponding to the pending user feature, and obtain the actual intent set corresponding to each of the user features;

[0009] Based on each of the user features and its corresponding actual intention set, the intelligent customer service model to be optimized is trained and optimized to obtain an intelligent customer service model.

[0010] In an optional embodiment, the historical consultation form includes user request information and consultation category;

[0011] The step of obtaining a candidate intent set corresponding to the pending user feature based on the pending historical conversations and pending historical consultation orders corresponding to the pending user feature includes:

[0012] Using a pre-trained business sentence verification model and a preset recognition model, extract each first intent from the pending historical conversation to obtain a first intent set corresponding to the pending user feature;

[0013] Utilizing a pre-trained business sentence verification model and a preset recognition model, extracting each second intent from the user demand information of the pending historical consultation form, and obtaining a second intent set corresponding to the pending user feature;

[0014] Among the multiple intentions associated with the intelligent customer service model to be optimized, each third intention that matches the consultation category in the pending historical consultation form is obtained, the third intention set corresponding to the pending user feature is obtained, and the candidate intention set corresponding to the pending user feature is obtained.

[0015] In an optional embodiment, the step of extracting each first intent from the pending historical conversation using a pre-trained business sentence verification model and a preset recognition model includes:

[0016] Taking each single sentence in the pending historical conversation as a sentence to be processed, and determining the order of each single sentence in the pending historical conversation;

[0017] Performing a business integrity check on the sentence to be processed using the business sentence verification model;

[0018] If the sentence to be processed passes the verification, the sentence to be processed is used as the first business sentence;

[0019] If the sentence to be processed fails the verification, the sentence to be processed is concatenated with a subsequent single sentence to obtain a target compound sentence, and after the target compound sentence is used as the sentence to be processed, the step of performing the business integrity verification on the sentence to be processed using the business sentence verification model is repeatedly performed until the total number of single sentences in the target compound sentence reaches a preset number;

[0020] The preset recognition model is used to identify the intent corresponding to each of the first business sentences to obtain each of the first intents.

[0021] In an optional embodiment, the step of extracting each second intent from the user demand information of the pending historical consultation form using a pre-trained business sentence verification model and a preset recognition model includes:

[0022] Using the business sentence verification model, perform business integrity verification on each single sentence in the user request information of the pending historical consultation form to obtain each second business sentence that passes the verification;

[0023] The preset recognition model is used to identify the intent corresponding to each second business sentence to obtain each second intent.

[0024] In an optional embodiment, the step of determining the actual intent set corresponding to the pending user feature based on the historical intent set and the candidate intent set corresponding to the pending user feature includes:

[0025] Obtaining the intersection of the historical intent set and the third intent set corresponding to the pending user feature;

[0026] Obtaining a union of a first intent set and a second intent set corresponding to the pending user feature;

[0027] The intersection and the union are combined to obtain the actual intent set corresponding to the pending user feature.

[0028] In an optional implementation manner, the business sentence verification model is obtained in the following manner:

[0029] Acquire multiple single sentence samples and their intent tags and multiple compound sentence samples and their intent tags, wherein the intent tags are business intent tags or non-business intent tags;

[0030] Each single sentence sample and each compound sentence sample in which the intent is marked as a business intent is regarded as a complete business sentence sample;

[0031] Each single sentence sample and each compound sentence sample in which the intent is marked as a non-business intent is regarded as a non-complete business sentence sample;

[0032] For each to-be-processed complex sentence sample whose intent is marked as a business intent, each single sentence in the to-be-processed complex sentence sample is used as an incomplete business sentence sample;

[0033] For each single sentence sample to be processed whose intent is marked as a business intent, the sentence obtained by splicing any two single sentence samples to be processed is regarded as an incomplete business sentence sample;

[0034] Based on the multiple complete business sentence samples and the multiple incomplete business sentence samples, the basic business sentence verification model is trained to obtain the business sentence verification model.

[0035] In a second aspect, the present invention provides a method for guessing what you want to ask, the method comprising:

[0036] Obtain target user characteristics based on target user's basic information and behavior information;

[0037] Use the intelligent customer service model to predict the intention of the target user characteristics and obtain the probability value corresponding to each intention;

[0038] Sort all probability values ​​in descending order, and obtain the intentions corresponding to the first N probability values ​​to obtain N target intentions, where N is a positive integer;

[0039] Recommending questions corresponding to each target intent to the target user;

[0040] Among them, the intelligent customer service model is obtained according to the intelligent customer service model optimization method described in any of the aforementioned implementation methods.

[0041] In a third aspect, the present invention provides an intelligent customer service model optimization device, comprising:

[0042] An acquisition module is used to obtain multiple user features and their corresponding historical conversations, historical consultation tickets, and historical intent sets; wherein the historical conversations represent the text content entered by the user to the manual customer service; the historical consultation tickets represent the user's consultation information recorded by the manual customer service; and the historical intent set represents the collection of intents that the user has encountered on the intelligent customer service page;

[0043] a determination module, configured to take each of the user features as a pending user feature, and obtain a candidate intent set corresponding to the pending user feature based on the pending historical conversations and pending historical consultation orders corresponding to the pending user feature;

[0044] Determine the actual intent set corresponding to the pending user feature based on the historical intent set and the candidate intent set corresponding to the pending user feature, and obtain the actual intent set corresponding to each of the user features;

[0045] The optimization module is used to train and optimize the intelligent customer service model to be optimized based on each of the user features and its corresponding actual intention set to obtain an intelligent customer service model.

[0046] In a fourth aspect, the present invention provides a device for guessing what you want to ask, the device comprising:

[0047] The prediction module is used to obtain target user characteristics based on the target user's basic information and behavior information;

[0048] Use the intelligent customer service model to predict the intention of the target user characteristics and obtain the probability value corresponding to each intention;

[0049] The recommendation module is used to sort all probability values ​​in descending order and obtain the intentions corresponding to the first N probability values ​​to obtain N target intentions, where N is a positive integer;

[0050] Recommending questions corresponding to each target intent to the target user;

[0051] Among them, the intelligent customer service model is obtained according to the intelligent customer service model optimization method described in any of the aforementioned implementation methods.

[0052] In a fifth aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the intelligent customer service model optimization method described in any one of the aforementioned embodiments, and / or the guessing what you want to ask method described in the aforementioned embodiments.

[0053] The intelligent customer service model optimization method, guess what you want to ask method, device and electronic device provided by the present invention first obtain multiple user features and their corresponding historical conversations, historical consultation forms and historical intent sets; then, each user feature is used as a pending user feature, and based on the pending historical conversations and pending historical consultation forms corresponding to the pending user feature, a candidate intent set corresponding to the pending user feature is obtained; and based on the historical intent set and candidate intent set corresponding to the pending user feature, the actual intent set corresponding to the pending user feature is determined, and the actual intent set corresponding to each user feature is obtained; finally, based on each user feature and its corresponding actual intent set, the intelligent customer service model to be optimized is trained and optimized to obtain an intelligent customer service model. By obtaining user consultation information collected by manual customer service through historical conversations and historical consultation forms, and guiding the optimization of the intelligent customer service model, the interference of erroneous intents on the model can be reduced, thereby improving the performance of the intelligent customer service model and improving the accuracy of model prediction.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown;

[0057] Figure 2 A schematic diagram of a process of optimizing an intelligent customer service model according to an embodiment of the present invention is shown;

[0058] Figure 3 An example diagram of an intelligent customer service model optimization method provided by an embodiment of the present invention is shown;

[0059] Figure 4Another flow chart of the intelligent customer service model optimization method provided by an embodiment of the present invention is shown;

[0060] Figure 5 A schematic diagram of a process of the guessing question method provided by an embodiment of the present invention is shown;

[0061] Figure 6 A functional module diagram of an intelligent customer service model optimization device provided by an embodiment of the present invention is shown;

[0062] Figure 7 A functional module diagram of a device for guessing what you want to ask provided by an embodiment of the present invention is shown.

[0063] Icons: 100-electronic device; 110-bus; 120-processor; 130-memory; 150-I / O module; 170-communication interface; 300-intelligent customer service model optimization device; 310-acquisition module; 330-determination module; 350-optimization module; 370-training module; 400-guess what you want to ask device; 410-prediction module; 430-recommendation module. DETAILED DESCRIPTION

[0064] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely intended to represent selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0066] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0067] Please refer to Figure 1 , is a block diagram of an electronic device 100 provided by an embodiment of the present invention. The electronic device 100 includes a bus 110 , a processor 120 , a memory 130 , an I / O module 150 , and a communication interface 170 .

[0068] The bus 110 may be a circuit that connects the above elements to each other and transfers communications (eg, control messages) between the above elements.

[0069] The processor 120 may receive commands from the other components (eg, the memory 130 , the I / O module 150 , the communication interface 170 , etc.) via the bus 110 , may interpret the received commands, and may perform calculations or data processing according to the interpreted commands.

[0070] The processor 120 can be an integrated circuit chip with signal processing capabilities. The processor 120 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0071] The memory 130 may store commands or data received from the processor 120 or other elements (eg, the I / O module 150 , the communication interface 170 , etc.) or commands or data generated by the processor 120 or other elements.

[0072] The memory 130 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM).

[0073] The I / O module 150 can receive commands or data input from the user via input-output means (e.g., sensors, keyboards, touch screens, etc.), and can transmit the received commands or data to the processor 120 or the memory 130 via the bus 110. It is also used to display various information (e.g., multimedia data, text data) received, stored, and processed by the above components, and can display videos, images, data, etc. to the user.

[0074] The communication interface 170 may be used to communicate signaling or data with other node devices.

[0075] It is understandable that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0076] The electronic device provided in the embodiment of the present invention may be a smart phone, a personal computer, a tablet computer, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiment of the present invention does not impose any limitation on this.

[0077] The electronic device 100 described above will be used as an execution subject to execute the steps of the methods provided in the embodiments of the present invention and achieve corresponding technical effects.

[0078] See also Figure 2 , Figure 2 This is a flow chart of an intelligent customer service model optimization method provided by an embodiment of the present invention.

[0079] Step S202: obtaining multiple user features and their corresponding historical conversations, historical consultation forms, and historical intent sets;

[0080] Among them, historical conversations represent the text content entered by users to manual customer service; historical consultation forms represent the user's consultation information recorded by manual customer service; and historical intent sets represent the collection of intents that users have encountered on the intelligent customer service page.

[0081] In this embodiment, based on multiple user samples, user features for each user sample, as well as historical conversations, historical consultation forms, and historical intent sets corresponding to the user features, can be obtained. Historical conversations can be understood as the text content entered by users when communicating with customer service personnel; historical consultation forms can be understood as consultation forms filled out by customer service personnel based on the user's consultation content; and historical intent sets can be understood as the collection of intents that users have clicked or entered on the intelligent customer service page.

[0082] It can be understood that historical conversations and historical consultation forms are based on user consultation information obtained by manual customer service, that is, manual customer service business information; historical intent sets are based on user consultation information obtained by intelligent customer service, that is, intelligent customer service business information.

[0083] It can be seen that compared with the existing technology, the embodiment of the present invention also uses manual customer service business information as training samples. That is, in addition to using the business information collected by intelligent customer service, it will also guide the optimization of the intelligent customer service model based on the business information collected by manual customer service, thereby improving the performance of the intelligent customer service model.

[0084] Step S204: taking each user feature as a pending user feature, and obtaining a candidate intent set corresponding to the pending user feature based on the pending historical conversations and pending historical consultation orders corresponding to the pending user feature;

[0085] Step S206, based on the historical intent set and candidate intent set corresponding to the pending user feature, determining the actual intent set corresponding to the pending user feature, and obtaining the actual intent set corresponding to each user feature;

[0086] In this embodiment, historical conversations and historical consultation forms contain the text content of the user's consultation, so based on historical conversations and historical consultation forms, the user's consultation intention can be obtained, that is, a candidate intention set can be obtained, and then the historical intention set can be filtered and supplemented based on the candidate intention set, that is, the wrong intentions are filtered out and the intentions that are not obtained are supplemented, that is, the user's actual intention set is obtained.

[0087] It can be understood that the method of obtaining the actual intention set corresponding to each user feature is similar. The following will take any user feature as the pending user feature, and the historical conversations and historical consultation forms corresponding to the pending user feature as the pending historical conversations and pending historical consultation forms for example.

[0088] First, based on the pending historical conversations and pending historical inquiries corresponding to the pending user feature, we obtain the candidate intent set corresponding to the pending user feature. Then, combined with the historical intent set corresponding to the pending user feature, we determine the corresponding actual intent set. By processing each user feature in a similar manner, we can obtain the actual intent set corresponding to each user feature.

[0089] Step S208: Based on each user feature and its corresponding actual intent set, the intelligent customer service model to be optimized is trained and optimized to obtain an intelligent customer service model;

[0090] It is understandable that the intelligent customer service model to be optimized can be a deep learning model with multi-label output using Deepfm or the like, or a multi-label classification model using a combination of multiple Xgboosts. This is not limited in the embodiments of the present invention.

[0091] In this embodiment, after obtaining each user feature and its corresponding actual intention set, each user feature can be input into the intelligent customer service model to be optimized, and the optimized intelligent customer service model can be used to predict the intention of each user feature to obtain the predicted intention set corresponding to each user feature; then, based on the actual intention set and predicted intention set corresponding to each user feature, the intelligent customer service model to be optimized is trained and optimized to obtain the intelligent customer service model.

[0092] It can be understood that the embodiment of the present invention obtains user consultation information collected by manual customer service through historical conversations and historical consultation forms, and guides the optimization of the intelligent customer service model, which can reduce the interference of erroneous intentions on the model, thereby improving the performance of the intelligent customer service model and improving the accuracy of model predictions.

[0093] Based on the above steps, we can see that we first obtain multiple user features and their corresponding historical conversations, historical consultation forms, and historical intent sets. We then use each user feature as a pending user feature, and based on the pending historical conversations and pending historical consultation forms corresponding to the pending user feature, we obtain the candidate intent set corresponding to the pending user feature. Furthermore, based on the historical intent set and candidate intent set corresponding to the pending user feature, we determine the actual intent set corresponding to the pending user feature, obtaining the actual intent set corresponding to each user feature. Finally, based on each user feature and its corresponding actual intent set, we train and optimize the intelligent customer service model to obtain an intelligent customer service model. By using historical conversations and historical consultation forms to obtain user consultation information collected by human customer service and guide the optimization of the intelligent customer service model, we can reduce the interference of erroneous intents on the model, thereby improving the performance of the intelligent customer service model and the accuracy of the model's predictions.

[0094] Optionally, for the above step S204, an embodiment of the present invention provides a possible implementation method.

[0095] Step S204-1: Using a pre-trained business sentence verification model and a preset recognition model, extract each first intent from the pending historical conversation to obtain a first intent set corresponding to the pending user feature;

[0096] It is understandable that, since historical conversations contain textual content consulted by users, the embodiment of the present invention pre-trains a business sentence verification model and pre-sets a recognition model to process the textual information of historical conversations to obtain intent.

[0097] The business sentence verification model is used to verify whether a sentence is a complete business sentence. A complete business sentence can be understood as a sentence that can express a specific intention. The preset recognition model is used to recognize the intention based on the sentence.

[0098] In this embodiment, for the pending historical conversations corresponding to the pending user features, the pre-trained business sentence verification model and the preset recognition model can be used to process them to extract each intent therefrom, that is, to obtain each first intent. The set of all first intents is the first intent set corresponding to the pending user features.

[0099] Step S204-3: Using the pre-trained business sentence verification model and the preset recognition model, extract each second intent from the user request information of the pending historical consultation form to obtain a second intent set corresponding to the pending user features;

[0100] Step S204-5, among the multiple intentions associated with the intelligent customer service model to be optimized, obtain each third intention that matches the consultation category in the pending historical consultation form, obtain the third intention set corresponding to the pending user characteristics, and obtain the candidate intention set corresponding to the pending user characteristics.

[0101] It is understandable that the historical consultation form includes user demand information and consultation categories. For ease of understanding, the embodiment of the present invention provides an example diagram of a consultation form.

[0102] See also Figure 3 A consultation ticket includes user information such as the caller ID and user ID, the inquiry category, the channel, the availability of a knowledge base, the closing method, inquiry tags, user request, the solution provided and whether it was resolved, and attachments. Based on the user request and inquiry category in the consultation ticket, user request information and inquiry category can be obtained.

[0103] In this embodiment, the user request information in the historical consultation form contains the text content of the user's consultation, so it can be processed using a pre-trained business sentence verification model and a preset recognition model to obtain the intent. Specifically, the user request information in the pending historical consultation form can be processed using a pre-trained business sentence verification model and a preset recognition model to extract each intent, thereby obtaining each second intent. The set of all second intents is the second intent set corresponding to the pending user feature.

[0104] The categories in historical inquiries are selected by customer service personnel based on communication with the user. It's understandable that the selectable categories in inquiries differ from the intents associated with the intelligent customer service model. Therefore, a matching relationship between the categories of inquiries and the intents associated with the intelligent customer service model can be pre-set.

[0105] This matching relationship can be represented by a mapping table. Based on this mapping table, we can obtain each third intent that matches the consultation category in the pending historical consultation tickets from the multiple intents associated with the intelligent customer service model to be optimized. The set of all third intents is the third intent set corresponding to the pending user features. By obtaining the first, second, and third intent sets corresponding to the pending user features, we obtain the candidate intent set corresponding to the pending user features.

[0106] Optionally, for the above step S204-1, an embodiment of the present invention provides a possible implementation method.

[0107] Step S204-1-1, treating each single sentence in the pending historical conversation as a sentence to be processed, and determining the order of each single sentence in the pending historical conversation;

[0108] Step S204-1-3, using the business sentence verification model to perform business integrity verification on the sentence to be processed;

[0109] Step S204-1-5: If the sentence to be processed passes the verification, the sentence to be processed is used as the first business sentence;

[0110] Step S204-1-7: If the sentence to be processed fails the verification, the sentence to be processed is concatenated with the next single sentence to obtain a target compound sentence. The target compound sentence is used as the sentence to be processed, and the step of performing business integrity verification on the sentence to be processed using the business sentence verification model is repeated until the total number of single sentences in the target compound sentence reaches a preset number.

[0111] In this embodiment, a simple sentence can be understood as a sentence without punctuation marks; a compound sentence can be understood as a sentence containing punctuation marks, that is, multiple simple sentences can be spliced ​​into a compound sentence.

[0112] For ease of understanding, the present invention provides an example to illustrate steps S204-1-1 to S204-1-7. For example, the sentences in the pending historical conversation are: sentence 1, sentence 2, sentence 3, and sentence 4.

[0113] Sentence 1 is treated as the sentence to be processed. The business sentence verification model is used to perform a business integrity check on sentence 1. Specifically, the business sentence verification model is used to predict the probability that sentence 1 is a complete business sentence. If the probability is greater than a first threshold, sentence 1 is deemed to have passed the verification, and step S204-1-5 is executed, treating sentence 1 as the first business sentence. If the probability is less than or equal to the first threshold, sentence 1 is deemed to have failed the verification, and step S204-1-7 is executed.

[0114] Assuming that the probability value of single sentence 1 is less than the first threshold, that is, single sentence 1 fails the verification, single sentence 1 is concatenated with the following sentence, single sentence 2, to obtain the target compound sentence, such as compound sentence A (single sentence 1 + single sentence 2); then compound sentence A is used as the sentence to be processed, and the business integrity verification model is used to verify the business integrity of compound sentence A.

[0115] Assuming that compound sentence A fails verification, a determination is made as to whether the total number of individual sentences in compound sentence A reaches a preset number. For example, if the preset number is 3, and the total number of individual sentences in compound sentence A does not reach the preset number, compound sentence A is concatenated with the next individual sentence, namely, sentence 3, to obtain a target compound sentence, such as compound sentence B (sentence 1 + sentence 2 + sentence 3). Compound sentence B is then used as the sentence to be processed, and a business integrity check is performed on compound sentence B using the business sentence verification model. Assuming that compound sentence B fails verification, a determination is made as to whether the total number of individual sentences in compound sentence B reaches the preset number. Since the total number of individual sentences in compound sentence B reaches the preset number, namely, 3, the processing flow for sentence 1 is terminated.

[0116] Single sentence 2, single sentence 3 and single sentence 4 are respectively regarded as sentences to be processed, and are subjected to business integrity verification using the business sentence verification model in a similar manner to the above, to obtain each first business sentence.

[0117] Step S204-1-9, using a preset recognition model to identify the intent corresponding to each first business sentence, to obtain each first intent;

[0118] In this embodiment, after each first business sentence is obtained, it is recognized using a preset recognition model to identify the intention corresponding to the first business sentence, that is, each first intention is obtained.

[0119] Optionally, for the above step S204-3, an embodiment of the present invention provides a possible implementation method.

[0120] Step S204-3-1, using the business sentence verification model, perform business integrity verification on each single sentence in the user request information of the pending historical consultation form, and obtain each second business sentence that passes the verification;

[0121] In this embodiment, the user request information in the pending historical consultation form can be split according to a preset rule to obtain each single sentence. It is understandable that the processing method for each single sentence is similar. For the sake of simplicity, the following is an example of a single sentence.

[0122] The business integrity check of the single sentence is performed using the business sentence verification model, that is, the probability value of the single sentence being a complete business sentence is predicted using the business sentence verification model; if the probability value is less than or equal to the second threshold, it is determined that the single sentence has failed the verification; if the probability value is greater than the second threshold, it is determined that the single sentence has passed the verification, and the single sentence is then used as the second business sentence.

[0123] In a similar manner as described above, the business sentence verification model is used to perform business integrity verification on each single sentence in the user demand information of the pending historical consultation form to obtain each second business sentence that passes the verification.

[0124] Step S204-3-3: Use a preset recognition model to identify the intent corresponding to each second business sentence to obtain each second intent.

[0125] In this embodiment, after each second business sentence is obtained, it is recognized using a preset recognition model to identify the intention corresponding to the second business sentence, that is, each second intention is obtained.

[0126] Optionally, for the above step S206, an embodiment of the present invention provides a possible implementation method.

[0127] Step S206-1, obtaining the intersection of the historical intent set corresponding to the pending user feature and the third intent set;

[0128] Step S206-3, obtaining the union of the first intent set and the second intent set corresponding to the pending user feature;

[0129] Step S206-5: merge the intersection and the union to obtain the actual intention set corresponding to the pending user feature.

[0130] It is understandable that the third intent set corresponding to the user's characteristics is obtained based on the consultation categories in the historical consultation tickets. The consultation categories are selected by customer service personnel after communicating with the user. Therefore, it can be considered that the third intent set is relatively close to the user's true intent. Therefore, in this embodiment, the historical intent set can be filtered based on the third intent set, and the filtered historical intent set can be supplemented based on the first and second intent sets.

[0131] In this embodiment, the intersection of the historical intent set corresponding to the pending user feature and the third intent set is obtained to filter out incorrect intents in the historical intent set, thereby obtaining a filtered historical intent set. For example, if the intersection of the two is the first set S1, then S1 represents the filtered historical intent set.

[0132] Obtain the union of the first intent set and the second intent set corresponding to the pending user features, for example, the union of the two is the second set S2; then synthesize the first set S1 and the second set S2 to supplement the filtered historical intent set, that is, obtain the actual intent set corresponding to the pending user features.

[0133] It can be understood that the embodiment of the present invention filters the historical intent set through the third intent set, which can reduce the interference of erroneous intent on the model; and supplements the filtered historical intent set based on the first intent set and the second intent set, which can improve the accuracy of model prediction, thereby improving the performance of the intelligent customer service model.

[0134] Optionally, for the above-mentioned business sentence verification model, the embodiment of the present invention provides an implementation method for obtaining the business sentence verification model, please refer to Figure 4 .

[0135] Step S210: Acquire multiple single sentence samples and their intention tags and multiple compound sentence samples and their intention tags, where the intention tags are business intention tags or non-business intention tags;

[0136] Step S212: each single sentence sample and each compound sentence sample whose intent is marked as a business intent mark are regarded as complete business sentence samples;

[0137] Step S214: each single sentence sample and each compound sentence sample marked as a non-business intent sample is treated as an incomplete business sentence sample;

[0138] In this embodiment, a sentence sample set, such as multiple single sentence samples and multiple compound sentence samples, can be obtained from the corpus. To ensure the accuracy of the business sentence verification model, the total number of single sentences included in the compound sentence sample cannot exceed a preset number. For example, if the preset number is 3 in the above embodiment, the compound sentence sample cannot contain more than 3 single sentences.

[0139] Each single sentence and each compound sentence sample has an intent tag, which can be a business intent tag or a non-business intent tag. The business intent tag is used to mark the sentence sample as expressing a specific intent, while the non-business intent tag is used to mark the sentence sample as not expressing a specific intent.

[0140] For all single-sentence samples, each single-sentence sample marked with a business intent tag is considered a complete business sentence sample, and each single-sentence sample marked with a non-business intent tag is considered an incomplete business sentence sample. For example, if single-sentence sample a, "Where can I purchase my membership?", has a business intent tag of "membership purchase," then single-sentence sample a is considered a complete business sentence sample. If single-sentence sample b, "How can I get a refund?", has a non-business intent tag, then single-sentence sample b is considered an incomplete business sentence sample.

[0141] For all complex sentence samples, each complex sentence sample marked with a business intent tag is considered a complete business sentence sample, and each complex sentence sample marked with a non-business intent tag is considered an incomplete business sentence sample. For example, complex sentence sample a, "I just ordered the wrong order and purchased a membership. How can I get a refund?", is marked with a business intent tag. Therefore, complex sentence sample a is considered a complete business sentence sample. Complex sentence sample b, "I'd like to ask what the weather is like today," is marked with a non-business intent tag. Therefore, single sentence sample b is considered an incomplete business sentence sample.

[0142] Step S216: for each complex sentence sample to be processed whose intention is marked as a business intention, each single sentence in the complex sentence sample to be processed is regarded as an incomplete business sentence sample;

[0143] Step S218: For each single sentence sample to be processed whose intent is marked as a business intent, a sentence formed by splicing any two single sentence samples to be processed is used as an incomplete business sentence sample;

[0144] It is understandable that complete business sentence samples are positive samples, while incomplete business sentence samples are negative samples. To improve the accuracy of the business sentence verification model, a large number of negative samples can also be constructed based on the sentence sample set to simulate various forms of sentences in actual business scenarios.

[0145] In this embodiment, for each complex sentence sample to be processed that is marked with a business intent tag, the complex sentence sample to be processed can be split into multiple single sentences, and each of these can be used as an incomplete business sentence sample. For example, the complex sentence sample a described above, namely, "I just clicked the wrong button. I paid for my membership. How can I get a refund?", is a complex sentence sample to be processed that is marked with a business intent tag. In this case, the complex sentence sample a can be split into single sentences a1, namely, "I just clicked the wrong button." Single sentence a2, namely, "I paid for my membership." Single sentence a3, namely, "How can I get a refund?", and each single sentence can be used as an incomplete business sentence sample.

[0146] For each single sentence sample to be processed whose intent is marked as a business intent, any two single sentence samples to be processed can be spliced ​​together, and the spliced ​​sentences can be used as incomplete business sentence samples. For example, the above-mentioned single sentence sample a, namely "Where to buy a membership", and single sentence sample c, namely "How to listen to paid albums", are both single sentence samples to be processed whose intent is marked as a business intent. Splicing single sentence sample a and single sentence sample c will result in the sentence "Where to buy a membership, how to listen to paid albums", which will be used as an incomplete business sentence sample.

[0147] Step S220: training a basic business sentence verification model based on multiple complete business sentence samples and multiple incomplete business sentence samples to obtain a business sentence verification model;

[0148] It is understandable that the basic service sentence verification model is a binary classification model, which may be a Bert model, and the embodiment of the present invention does not limit this.

[0149] In this embodiment, after obtaining a plurality of complete business sentence samples and a plurality of incomplete business sentence samples, they can be labeled to obtain a label for each complete business sentence sample and a label for each incomplete business sentence sample.

[0150] The label of the complete business sentence sample is a first label such as 1, and the label of the incomplete business sentence sample is a second label such as 0. Based on multiple complete business sentence samples and their labels and multiple incomplete business sentence samples and their labels, the basic business sentence verification model is trained to obtain a business sentence verification model.

[0151] Optionally, based on the above optimized intelligent customer service model, the embodiment of the present invention also provides a method for using the intelligent customer service model to guess what you want to ask. Figure 5 , Figure 5 The present invention provides a flowchart of a method for guessing what you want to ask.

[0152] Step S222, obtaining target user characteristics based on the target user's basic information and behavior information;

[0153] Step S224: Use the intelligent customer service model to predict the target user's intentions and obtain the probability value corresponding to each intention;

[0154] Step S226: sort all probability values ​​in descending order, and obtain the intentions corresponding to the first N probability values ​​to obtain N target intentions, where N is a positive integer;

[0155] Step S228: recommend questions corresponding to each target intent to the target user.

[0156] In this embodiment, after the target user enters the smart customer page, basic information of the target user, such as gender and age, and user behavior information, such as historical audio listening and historical browsing paths, can be obtained. Based on the basic information and behavior information of the target user, the target user characteristics are obtained.

[0157] Then, the intelligent customer service model is used to predict the intentions of the target user characteristics and obtain the probability value corresponding to each intent; then all probability values ​​are sorted in order from large to small to obtain the intentions corresponding to the first N probability values, that is, the N intentions with the largest probability values ​​are obtained to obtain N target intentions; then the questions corresponding to each target intent are recommended to the target user, thereby realizing the guessing of what you want to ask.

[0158] It can be understood that the intelligent customer service model in the embodiment of the present invention is obtained after optimization according to the above-mentioned intelligent customer service model optimization method, which can reduce the interference of erroneous intentions on the model and improve the performance of the model. Therefore, the intelligent customer service model can be used to recommend more accurate questions to users, thereby improving the user experience.

[0159] In order to execute the various steps in the intelligent customer service model optimization method provided by the embodiment of the present invention, an implementation method of an intelligent customer service model optimization device is provided below. Figure 6 , Figure 6 This is a functional module diagram of an intelligent customer service model optimization device 300 provided in an embodiment of the present invention. It should be noted that the basic principles and technical effects of the intelligent customer service model optimization device 300 provided in this embodiment are the same as those of the above embodiments. For the sake of simplicity, any matters not mentioned in this embodiment can be referred to the corresponding contents of the above embodiments. The intelligent customer service model optimization device 300 includes:

[0160] Acquisition module 310 is used to obtain multiple user features and their corresponding historical conversations, historical consultation tickets, and historical intent sets. Historical conversations represent textual content entered by users into human customer service; historical consultation tickets represent user consultation information recorded by human customer service; and historical intent sets represent the collection of intents that users have encountered on the intelligent customer service page.

[0161] A determination module 330 is configured to take each user feature as a pending user feature and obtain a candidate intent set corresponding to the pending user feature based on the pending historical conversations and pending historical consultation orders corresponding to the pending user feature;

[0162] Based on the historical intent set and candidate intent set corresponding to the pending user feature, the actual intent set corresponding to the pending user feature is determined, and the actual intent set corresponding to each user feature is obtained;

[0163] The optimization module 350 is used to train and optimize the intelligent customer service model to be optimized based on each user feature and its corresponding actual intention set to obtain an intelligent customer service model.

[0164] Optionally, the determination module 330 is also used to: use a pre-trained business sentence verification model and a preset recognition model to extract each first intent from the pending historical conversation, and obtain a first intent set corresponding to the pending user characteristics; use a pre-trained business sentence verification model and a preset recognition model to extract each second intent from the user demand information of the pending historical consultation form, and obtain a second intent set corresponding to the pending user characteristics; obtain each third intent that matches the consultation category in the pending historical consultation form from the multiple intentions associated with the intelligent customer service model to be optimized, obtain a third intent set corresponding to the pending user characteristics, and obtain a candidate intent set corresponding to the pending user characteristics.

[0165] Optionally, the determination module 330 is also used to: treat each single sentence in the pending historical conversation as a sentence to be processed, and determine the order of each single sentence in the pending historical conversation; use the business sentence verification model to perform business integrity verification on the sentence to be processed; if the sentence to be processed passes the verification, the sentence to be processed is used as the first business sentence; if the sentence to be processed fails the verification, the sentence to be processed is concatenated with the subsequent single sentence to obtain a target compound sentence, and after using the target compound sentence as the sentence to be processed, the business integrity verification of the sentence to be processed using the business sentence verification model is repeatedly performed until the total number of single sentences in the target compound sentence reaches a preset number; and the preset recognition model is used to identify the intention corresponding to each first business sentence to obtain each first intention.

[0166] Optionally, the determination module 330 is also used to: use the business sentence verification model to perform business integrity verification on each single sentence in the user demand information of the pending historical consultation form, and obtain each second business sentence that passes the verification; use the preset recognition model to identify the intention corresponding to each second business sentence, and obtain each second intention.

[0167] Optionally, the determination module 330 is also used to: obtain the intersection of the historical intent set and the third intent set corresponding to the pending user feature; obtain the union of the first intent set and the second intent set corresponding to the pending user feature; merge the intersection and the union to obtain the actual intent set corresponding to the pending user feature.

[0168] Optionally, the intelligent customer service model optimization device 300 also includes a training module 370, which is used to: obtain multiple single sentence samples and their intention tags and multiple compound sentence samples and their intention tags, with the intention tags being business intention tags or non-business intention tags; use each single sentence sample and each compound sentence sample with the intention tag being a business intention tag as a complete business sentence sample; use each single sentence sample and each compound sentence sample with the intention tag being a non-business intention tag as an incomplete business sentence sample; for each to-be-processed compound sentence sample with the intention tag being a business intention tag, use each single sentence in the to-be-processed compound sentence sample as an incomplete business sentence sample; for each to-be-processed single sentence sample with the intention tag being a business intention tag, use the sentence obtained by splicing any two to-be-processed single sentence samples as an incomplete business sentence sample; train the basic business sentence verification model based on multiple complete business sentence samples and multiple incomplete business sentence samples to obtain a business sentence verification model.

[0169] In order to execute the various steps of the guessing question method provided in the embodiment of the present invention, a method for implementing the guessing question device is provided below. Figure 7 , Figure 7 This is a functional module diagram of a device 400 for guessing what you want to ask, provided in an embodiment of the present invention. It should be noted that the basic principles and technical effects of the device 400 provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of brevity, any matters not mentioned in this embodiment may be referred to the corresponding contents of the above-mentioned embodiments. The device 400 for guessing what you want to ask includes:

[0170] Prediction module 410 is used to obtain target user characteristics based on the target user's basic information and behavior information; use the intelligent customer service model to predict the target user's intentions and obtain the probability value corresponding to each intention;

[0171] The recommendation module 430 is used to sort all probability values ​​in descending order, obtain the intentions corresponding to the first N probability values, and obtain N target intentions, where N is a positive integer; and recommend questions corresponding to each target intention to the target user.

[0172] An embodiment of the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program. When the processor executes the computer program, the intelligent customer service model optimization method disclosed in the embodiment of the present invention and / or the guessing you want to ask method are implemented.

[0173] An embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the intelligent customer service model optimization method and / or the guessing you want to ask method disclosed in an embodiment of the present invention are implemented.

[0174] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0175] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0176] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0177] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for optimizing an intelligent customer service model, characterized in that: The method comprises: Obtain multiple user features and their corresponding historical conversations, historical consultation tickets, and historical intent sets; wherein the historical conversations represent the text content entered by the user to the manual customer service; the historical consultation tickets represent the user's consultation information recorded by the manual customer service; and the historical intent set represents the collection of intents that the user has encountered on the intelligent customer service page; Taking each of the user features as a pending user feature, respectively, and using a pre-trained business sentence verification model and a preset recognition model, extracting each first intent from the pending historical conversation to obtain a first intent set corresponding to the pending user feature; Utilizing a pre-trained business sentence verification model and a preset recognition model, extracting each second intent from the user demand information of the pending historical consultation form, and obtaining a second intent set corresponding to the pending user feature; Determining each third intent that matches the consultation category in the pending historical consultation form from a preset mapping table to obtain a third intent set corresponding to the pending user feature; wherein the mapping table represents the matching relationship between the consultation category and the intent associated with the model; Obtaining the intersection of the historical intent set and the third intent set corresponding to the pending user feature, and simultaneously obtaining the union of the first intent set and the second intent set corresponding to the pending user feature, and merging the intersection and the union to obtain the actual intent set corresponding to the pending user feature; Based on each of the user features and its corresponding actual intention set, the intelligent customer service model to be optimized is trained and optimized to obtain an intelligent customer service model.

2. The intelligent customer service model optimization method according to claim 1, characterized in that: The step of extracting each first intent from the pending historical conversation using the pre-trained business sentence verification model and the preset recognition model includes: Taking each single sentence in the pending historical conversation as a sentence to be processed, and determining the order of each single sentence in the pending historical conversation; Performing a business integrity check on the sentence to be processed using the business sentence verification model; If the sentence to be processed passes the verification, the sentence to be processed is used as the first business sentence; If the sentence to be processed fails the verification, the sentence to be processed is concatenated with a subsequent single sentence to obtain a target compound sentence, and after the target compound sentence is used as the sentence to be processed, the step of performing the business integrity verification on the sentence to be processed using the business sentence verification model is repeatedly performed until the total number of single sentences in the target compound sentence reaches a preset number; The preset recognition model is used to identify the intent corresponding to each of the first business sentences to obtain each of the first intents.

3. The intelligent customer service model optimization method according to claim 1, characterized in that: The step of extracting each second intent from the user demand information of the pending historical consultation form by using the pre-trained business sentence verification model and the preset recognition model includes: Using the business sentence verification model, perform business integrity verification on each single sentence in the user request information of the pending historical consultation form to obtain each second business sentence that passes the verification; The preset recognition model is used to identify the intent corresponding to each second business sentence to obtain each second intent.

4. The intelligent customer service model optimization method according to claim 1, characterized in that: The business sentence verification model is obtained in the following manner: Acquire multiple single sentence samples and their intent tags and multiple compound sentence samples and their intent tags, wherein the intent tags are business intent tags or non-business intent tags; Each single sentence sample and each compound sentence sample in which the intent is marked as a business intent is regarded as a complete business sentence sample; Each single sentence sample and each compound sentence sample in which the intent is marked as a non-business intent is regarded as a non-complete business sentence sample; For each to-be-processed complex sentence sample whose intent is marked as a business intent, each single sentence in the to-be-processed complex sentence sample is used as an incomplete business sentence sample; For each single sentence sample to be processed whose intent is marked as a business intent, the sentence obtained by splicing any two single sentence samples to be processed is regarded as an incomplete business sentence sample; Based on the multiple complete business sentence samples and the multiple incomplete business sentence samples, the basic business sentence verification model is trained to obtain the business sentence verification model.

5. A method for guessing what you want to ask, characterized in that: The guessing method includes: Obtain target user characteristics based on target user's basic information and behavior information; Use the intelligent customer service model to predict the intention of the target user characteristics and obtain the probability value corresponding to each intention; Sort all probability values ​​in descending order, and obtain the intentions corresponding to the first N probability values ​​to obtain N target intentions, where N is a positive integer; Recommending questions corresponding to each target intent to the target user; Wherein, the intelligent customer service model is obtained according to the intelligent customer service model optimization method according to any one of claims 1 to 4.

6. An intelligent customer service model optimization device, characterized in that: The device comprises: An acquisition module is used to obtain multiple user features and their corresponding historical conversations, historical consultation tickets, and historical intent sets; wherein the historical conversations represent the text content entered by the user to the manual customer service; the historical consultation tickets represent the user's consultation information recorded by the manual customer service; and the historical intent set represents the collection of intents that the user has encountered on the intelligent customer service page; Identify modules for: Taking each of the user features as a pending user feature, respectively, and using a pre-trained business sentence verification model and a preset recognition model, extracting each first intent from the pending historical conversation to obtain a first intent set corresponding to the pending user feature; Utilizing a pre-trained business sentence verification model and a preset recognition model, extracting each second intent from the user demand information of the pending historical consultation form, and obtaining a second intent set corresponding to the pending user feature; Determining each third intent that matches the consultation category in the pending historical consultation form from a preset mapping table to obtain a third intent set corresponding to the pending user feature; wherein the mapping table represents the matching relationship between the consultation category and the intent associated with the model; Obtaining the intersection of the historical intent set and the third intent set corresponding to the pending user feature, and simultaneously obtaining the union of the first intent set and the second intent set corresponding to the pending user feature, and merging the intersection and the union to obtain the actual intent set corresponding to the pending user feature; The optimization module is used to train and optimize the intelligent customer service model to be optimized based on each of the user features and its corresponding actual intention set to obtain an intelligent customer service model.

7. A device for guessing what you want to ask, characterized in that: The guess you want to ask device includes: The prediction module is used to obtain target user characteristics based on the target user's basic information and behavior information; Use the intelligent customer service model to predict the intention of the target user characteristics and obtain the probability value corresponding to each intention; The recommendation module is used to sort all probability values ​​in descending order and obtain the intentions corresponding to the first N probability values ​​to obtain N target intentions, where N is a positive integer; Recommending questions corresponding to each target intent to the target user; Wherein, the intelligent customer service model is obtained according to the intelligent customer service model optimization method according to any one of claims 1 to 4.

8. An electronic device, characterized in that: It includes a processor and a memory, the memory stores a computer program, and when the processor executes the computer program, it implements the intelligent customer service model optimization method described in any one of claims 1 to 4, and / or the guessing what you want to ask method described in claim 5.

Images