Customer service quality detection method and device

Abstract

The invention provides a customer service quality detection method and device. The method comprises the steps of obtaining a target dialogue text set; the target dialogue text set is preprocessed, text attributes are set for dialogues in the target dialogue text set, a preprocessed information set is obtained, and the text attributes comprise time, emotion orientation and behaviors; based on the preprocessed information set, labels are configured for dialogues in the target dialogue text set through preset business rules; extracting question and answer pairs based on the target dialogue text set, setting the obtained question and answer pairs as a quality inspection item, and generating a quality inspection item list; and performing quality inspection on each quality inspection item in the quality inspection item list based on the text attribute and the label, and generating a text quality inspection score. According to the method, quality inspection evaluation of each quality inspection item is completed by reusing the text attributes and the labels, and sequential text analysis does not need to be performed on each evaluation, so that the number of times of text analysis is reduced, the problem of repeated calculation of similar labels is solved, and the detection efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of service quality detection, and in particular to a method and device for detecting customer service quality. Background Art

[0002] In the field of customer service, after the service is completed, evaluating the service quality of customer service personnel is a way to measure the service quality of customer service personnel. In the traditional customer service quality detection method, the customer service call voice is first converted into the corresponding text, and then the conversation text is labeled, and then the corresponding label is matched to the text that meets the requirements. Furthermore, the labels corresponding to all conversation texts are summarized and counted to obtain the statistical information of the labels, and the customer service quality detection results are obtained according to the preset quality inspection rules and the statistical information of the conversation labels corresponding to all conversation text segments.

[0003] The above customer service quality detection has the problem of repeated calculation of similar tags, which wastes calculation time and affects user experience. Summary of the invention

[0004] The embodiments of the present application provide a method and device for detecting customer service quality, which reduces the number of text analyses by reusing text attributes and labels, thereby solving the problem of repeated calculation of similar labels, improving detection efficiency, and further improving user experience.

[0005] In the first aspect, the embodiment of the present application provides a customer service quality detection method, including: obtaining a target dialogue text set; preprocessing the target dialogue text set, setting text attributes for the dialogues in the target dialogue text set, and obtaining a preprocessed information set, wherein the text attributes include time, emotional orientation, and behavior; based on the preprocessed information set, configuring labels for the dialogues in the target dialogue text set through preset business rules; extracting question-answer pairs based on the target dialogue text set, setting each of the obtained question-answer pairs as a quality inspection item, and generating a quality inspection item list; based on the text attributes and the labels, performing quality inspection on the quality inspection items in the quality inspection item list and generating a text quality inspection score. After obtaining the text attributes and labels of the dialogue, when performing text quality inspection evaluation, the text attributes and labels can be reused to complete the quality inspection evaluation of each quality inspection item, without the need to perform text analysis in sequence for each evaluation, thereby reducing the number of text analysis, solving the problem of repeated calculation of similar labels, and improving detection efficiency.

[0006] In a possible implementation, the target dialogue text set is preprocessed, and text attributes are set for the dialogues in the target dialogue text set to obtain the preprocessed information set, including: obtaining the interval between two adjacent dialogues in the target dialogue text set, wherein the time of the latter dialogue between the two adjacent dialogues includes the interval between the former adjacent dialogue; performing natural language analysis on the dialogues in the target dialogue text set to obtain the emotional tendency in the dialogue, wherein the emotional tendency includes positive, negative and neutral; obtaining the behavior of the customer service staff in the dialogue, wherein the behavior includes questioning behavior, answering behavior, comforting behavior, insulting behavior or no behavior. By analyzing the text attributes of the dialogue in the three dimensions of time, emotional tendency and behavior, the attribute information of the dialogue can be accurately obtained, which is conducive to the subsequent quality evaluation.

[0007] In a possible implementation, configuring labels for the dialogues in the target dialogue text set based on the preprocessed information set by using preset business rules includes: determining whether there is a set behavior of the preset business rule in the dialogue based on the text attributes in the preprocessed information set, and setting corresponding labels based on the detection results. In the process of labeling, by reusing text attributes, the number of text analyses can also be reduced, thereby improving the efficiency of quality detection.

[0008] In a possible implementation, the quality inspection of the quality inspection items in the quality inspection item list based on the text attributes and the labels and generating a text quality inspection score includes: detecting whether the first quality inspection item in the quality inspection item list contains a politeness label, and if it does not contain the politeness label, deducting a corresponding score from the initial score of the text; obtaining complex quality inspection items in the quality inspection item list; detecting whether the complex quality inspection items are compliant, and if non-compliant complex quality inspection items occur, deducting a corresponding score and outputting a text quality inspection score.

[0009] In one possible implementation, obtaining the complex quality inspection items in the quality inspection item list includes: determining whether the emotional tendency of the user in the conversation is negative based on the text attributes in the conversation text of the current quality inspection item, and if negative, determining that the current quality inspection item is a complex quality inspection item.

[0010] In one possible implementation, the detecting whether the complex quality inspection item is compliant includes: based on the label in the conversation text of the current complex quality inspection item, determining whether the customer service has soothing behavior in the conversation, wherein if there is a soothing label in the conversation text of the current complex quality inspection item, then it is determined that the customer service has soothing behavior, and it is determined that the current complex quality inspection item is compliant; if there is no soothing label, then it is determined that the customer service has no soothing behavior, and it is determined that the current complex quality inspection item is not compliant.

[0011] In a possible implementation, after extracting question-answer pairs based on the target dialogue text set, it also includes: classifying the extracted question-answer pairs based on preset question-answer types in a preset knowledge base, wherein the preset question-answer types include professional categories and activity categories; based on the type of question-answer pair, retrieving preset standard answers in the corresponding type data of the knowledge base, wherein the preset standard answers include several key information and corresponding scores for the key information; detecting whether the customer service reply content in the question-answer pair contains the several key information, and if it is detected that any of the key information does not exist, deducting the corresponding score for the key information; after traversing the key information detection of each question-answer pair, outputting the question-answer quality inspection score.

[0012] In a possible implementation, before performing key information detection on the question-answer pairs, the method further includes: filtering out question-answer pairs that are not related to the customer service field based on the customer service field. By filtering out irrelevant question-answer pairs, the accuracy of subsequent question-answer quality inspection scores can be further improved.

[0013] In a possible implementation, the method further includes: performing weighted calculation based on the text quality inspection score and the question and answer quality inspection score to obtain a comprehensive score. By performing weighted calculation on the text quality inspection score and the question and answer quality inspection score, the service quality of the customer service can be comprehensively reflected from two aspects: whether there is substantial content in the text and the quality of the answer. The obtained comprehensive score is closer to the facts and improves the detection accuracy.

[0014] In a second aspect, an embodiment of the present application also provides a customer service quality detection device, including: a processor and a memory, wherein the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the customer service quality detection method provided in the first aspect is implemented.

[0015] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the customer service quality detection method provided in the first aspect is implemented.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the customer service quality detection method provided in the first aspect.

[0017] Through the above technical solution, after obtaining the target dialogue text set, the target dialogue text set can be preprocessed through natural language analysis, and text attributes are set for each dialogue in the target dialogue text set to obtain a preprocessed information set. The text attributes include time, emotional orientation, and behavior. The attribute characteristics of each dialogue are accurately reflected through these three dimensions, so that each dialogue can be labeled by using the text attributes to reduce the number of text analysis. When performing text quality inspection and scoring on the text, the text attributes and labels are reused to complete the text quality inspection and scoring of customer service services, thereby improving the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of a customer service quality detection method provided for one embodiment of the present application;

[0020] Figure 2 A schematic diagram of the structure of a customer service quality detection device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] To facilitate understanding of the technical solution of this application, the following explains the corresponding concepts involved in this application:

[0023] 1. Tagging: refers to setting labels according to rules and tagging specific labels for sentences that meet the rules. The specific format can be: {label: rule}, for example, {"politeness": "you|please"}, where when a sentence contains keywords such as "you" or "please", it can be determined that the current sentence meets the rules, and then the sentence that meets the rules can be tagged with "politeness".

[0024] 2. ASR service: Automatic Speech Recognition, automatic speech recognition service, can convert user's voice into editable or searchable text.

[0025] 3. ElasticSearch: Search server, a distributed, highly scalable engine with real-time search and data analysis capabilities.

[0026] 4. Redis: It is a key-value storage system that supports data types including string, list, set, zset (sorted set) and hash.

[0027] Figure 1 A flowchart of a customer service quality detection method provided for one embodiment of the present application.

[0028] Reference Figure 1 As shown, the method includes the following processes:

[0029] S101: Obtain a target dialogue text set.

[0030] In some embodiments, the specific method for obtaining the target conversation text set may include: after obtaining the customer service call voice, converting the customer service call voice into the corresponding target conversation text set through the ASR service. In other embodiments, the target conversation text set converted by an external device may also be obtained. This application does not limit the method for obtaining the target conversation text set.

[0031] S102: Preprocess the target dialogue text set and set text attributes for the dialogues in the target dialogue text set. The text attributes include time, emotional tendency, and behavior.

[0032] In some embodiments, after obtaining the target dialogue text set, the target dialogue text set can be preprocessed. Specifically, text attributes can be set for each sentence in the target dialogue text set, where the text attributes can include time-related attributes, emotional tendency-related attributes, and behavior-related attributes.

[0033] In some embodiments, the time-related attribute may be the interval between two adjacent dialogue sentences.

[0034] Text of the previous conversation: (16:35:40) User asked: "What time does the XXX brand 4S store on XXXX street close?"

[0035] The current dialogue text to be processed: (16:35:42) Customer service reply: "The business hours of the XXX brand on XXXX Street are 9:00-18:00".

[0036] Based on the above example, it can be known that the interval between the two adjacent dialogues is 2 seconds. Then, a time-related attribute "interval: 2" can be set for the current dialogue text to be processed, indicating that the interval between the current dialogue text to be processed and the adjacent previous dialogue is 2 seconds.

[0037] In some embodiments, attributes related to emotional tendencies may include positive, negative, and neutral.

[0038] In some embodiments, the target conversation text set can be analyzed for emotional expression in each conversation through a relevant natural language analysis method, specifically analyzing whether there is emotional expression in the conversation, and if so, identifying whether the specific emotion is positive or negative (negative emotion). This application does not limit the analysis method of emotional expression.

[0039] In one implementation, when it is identified that the current dialogue text to be processed expresses positive emotions, an attribute related to the emotion tendency can be set for the current dialogue text to be processed: "emotion: 1", indicating that the attribute related to the emotion tendency of the current dialogue text to be processed is positive; for example, the current dialogue text to be processed is: customer service reply: "Customer service number XXX, happy to serve you". The current dialogue text to be processed contains the keyword "happy" expressing positive emotions, so that it can be identified that the current dialogue text to be processed expresses positive emotions, and the attribute related to the emotion tendency can be set for the current dialogue text to be processed: "emotion: 1".

[0040] In one implementation, when it is identified that the current dialogue text to be processed expresses negative emotions, an attribute related to the emotion tendency can be set for the current dialogue text to be processed: "emotion: 2", indicating that the attribute related to the emotion tendency of the current dialogue text to be processed is negative; for example, the current dialogue text to be processed is: user reply: "You didn't solve the problem I asked at all". Through natural language analysis, it can be identified that the current dialogue text to be processed expresses the user's very dissatisfied negative emotions, and the attribute related to the emotion tendency can be set for the current dialogue text to be processed: "emotion: 2".

[0041] In one embodiment, when it is identified that the current dialogue text to be processed does not express any emotion, an attribute related to the emotion tendency can be set for the current dialogue text to be processed: "emotion: 0", indicating that the attribute related to the emotion tendency of the current dialogue text to be processed is neutral. For example, the current dialogue text to be processed is: Customer service reply: "The business hours of XXX brand on XXXX street are 9:00-18:00". Through natural language analysis, it is identified that the current dialogue text to be processed does not express any emotion, and the attribute related to the emotion tendency can be set for the current dialogue text to be processed: "emotion: 0".

[0042] In some embodiments, the attributes related to behavior may include specific business behaviors of customer service personnel such as questioning behavior, answering behavior, soothing behavior, and insulting behavior, and also include attributes of no behavior. In the process of customer service personnel providing services to users, there may be customer service answering behaviors in which the customer service answers the user's questions, customer service confirming the vague information provided by the user, customer service soothing behaviors in which the customer service soothes the user's negative emotions, and customer service insulting behaviors in which the customer service insults the user. It should be noted that in other embodiments, the attributes related to behavior may also include other behaviors, and this application does not limit the types of behaviors. In the process of customer service personnel providing services to users, conversations in which the customer service does not have any behavior may also be stored.

[0043] In one implementation manner, when it is identified that the current dialogue text to be processed contains a questioning behavior, a behavior-related attribute “behavior: 1” may be set for the current dialogue text to be processed.

[0044] In one implementation manner, when it is identified that the current dialogue text to be processed has a solution behavior, a behavior-related attribute "behavior: 2" may be set for the current dialogue text to be processed.

[0045] In one implementation manner, when it is identified that the current dialogue text to be processed contains a soothing behavior, a behavior-related attribute "behavior: 3" may be set for the current dialogue text to be processed.

[0046] In one implementation manner, when it is identified that the current dialogue text to be processed contains insulting behavior, a behavior-related attribute "behavior: 4" may be set for the current dialogue text to be processed.

[0047] In one implementation manner, when it is identified that the current dialog text to be processed does not have any behavior, a behavior-related attribute “behavior: 0” may be set for the current dialog text to be processed.

[0048] In some embodiments, after setting text attributes for the current dialogue text to be processed, the pre-processed first information can be output. The first information can include the current dialogue text to be processed and the corresponding text attributes. The specific format can be {text text, text attributes}. For example, {text text: Hello, you need to bring your driving license to the station for maintenance, text attributes: "emot ion: 0", "interval: 2", "behavior: 2"}.

[0049] The text attributes of each dialogue in the target dialogue text set may be set in the above manner to complete the preprocessing, and each first information may be integrated according to the dialogue sequence to obtain a preprocessed information set, the specific format of which may be shown in the following Table 1:

[0050] Table 1

[0051] Serial number text property 1 Text-1 "emotion:x", "interval:x", "behavior:x" 2 Text-2 "emotion:x", "interval:x", "behavior:x" … … … n Text-n "emotion:x", "interval:x", "behavior:x"

[0052] S103: Based on the preprocessed information set, configure tags for the dialogues in the target dialogue text set according to preset business rules.

[0053] In some embodiments, tags have an association with text attributes, and tags can be matched based on text attributes. The association between tags and text attributes can include an association between some set behavior-related attributes and tags. For example, the behavior-related attribute "behavior: 4" in the conversation text indicates the presence of abusive behavior. In one embodiment, in the above S102, it can be determined whether there are abusive keywords in the text. When the corresponding keywords are detected, it can be determined that there is abusive behavior. It should be noted that in the process of setting text attributes, the analysis of the text has been completed, and the keywords matching the behavior have been identified. When matching tags, the keywords that have been analyzed and matched with the behavior can be directly called for tag matching, thereby saving the time and computing power for analyzing the text again and improving the efficiency of quality detection.

[0054] In some embodiments, different business rules exist based on different user needs. For example, when the user's need is to only detect whether the customer service has insulted the user, the corresponding business rule is to determine whether the customer service has insulted the user in each sentence of the conversation based only on text attributes. If so, an insulting label is set, otherwise no corresponding label is set.

[0055] In other embodiments, user requirements may also include whether the customer service uses polite language and whether the customer service insults the user. Correspondingly, two business rules may be included, wherein the first business rule is used to determine whether the customer service uses polite language based on text attributes. If so, a polite language label is set; if not, a polite language label is set; the second business rule is used to determine whether the customer service has abusive behavior in each sentence of the conversation based on text attributes. If so, an insulting label is set; if not, no corresponding label is set.

[0056] In some embodiments, when matching tags, a regular matching method may be used to perform tag matching, thereby reducing the probability of repeated generation of similar tags.

[0057] In some embodiments, the attributes related to the behavior may also include the use of polite language, which can be determined by identifying whether polite language keywords appear in the text, such as "you", "please", "thank you", "thank you", etc. After the polite language keywords are identified in the text attribute setting process, if two polite language keywords are identified, including "you" and "please", a "polite language" label is matched for the current sentence through regular matching. Therefore, when there are multiple keywords associated with the same behavior, only one corresponding label is matched for the current sentence, thereby reducing the probability of repeated generation of similar labels.

[0058] In some embodiments, the text data after tag matching is completed can be stored in a first preset format, wherein the first preset format can include a json format.

[0059] S104: Extract question-answer pairs based on the target dialogue text set and generate quality inspection items.

[0060] In some embodiments, after obtaining the target conversation text set, the question-answer pair in the conversation text set, i.e., the question and the corresponding answer content, can also be extracted. For example, the user asks a question: the customer service replies: "The business hours of the XXX brand on XXXX Street are 9:00-18:00". The above question-answer content can be identified as a question-answer pair.

[0061] In some embodiments, all question-answer pairs in the target dialogue text set can be extracted by a preset extraction model. The format of the question-answer pairs output by the extraction model can be preset standard QA data, where Q represents question and A represents answer. For example,

[0062] Q: "What time does the 4S store of XXX brand on XXXX street close?"

[0063] A: "The business hours of XXX brand on XXXX Street are 9:00-18:00."

[0064] In some conversation scenarios, customer service staff may not be able to answer the user's question in one reply. They may also guide the user by asking several questions, and then integrate the user's several replies and output the reply content to complete the answer to the user's main question.

[0065] In this case, since the above-mentioned several question-answer pairs are all intended to solve the user's main question, in some embodiments, the above-mentioned several questions and answers can be combined to form one question-answer pair.

[0066] In some embodiments, a specific implementation of merging multiple question-answer pairs may include:

[0067] The adjacent question and answer pairs are identified for key point similarity, and if the similarity exceeds the similarity threshold, the adjacent questions and answers are merged. In the merging process, the key point similarity of adjacent question and answer pairs can be identified in sequence based on the extraction order of the question and answer pairs, and the adjacent questions and answers that meet the conditions (i.e., the similarity exceeds the similarity threshold) are merged. Exemplarily, question and answer pair a and question and answer pair b are continuous question and answer pairs. Through key point similarity identification, the key point similarity between question and answer pair a and question and answer pair b exceeds the similarity threshold. Therefore, question and answer pair a and question and answer pair b can be merged to form a question and answer pair. Among them, the reply content in a merged question and answer pair can include the key point information in each sub-question and answer pair before the merge, so as to ensure the integrity of the reply content. Exemplarily,

[0068] User asked: "Why is the vehicle's left turn indicator not on?"

[0069] Customer service asked: "Please check if the left turn indicator is damaged."

[0070] The user replied: "The vehicle's left turn indicator is intact and not damaged."

[0071] Customer service question: "Please remove the left turn indicator and check whether the connection line is faulty"

[0072] The user replied: "It lights up! It turns out that the power connector of the indicator light is loose."

[0073] In the above example, the user's main question is "Why is the vehicle indicator light off?" The customer service guides the user to find out the possible reasons for the indicator light not being on through multiple small questions. Through key point similarity recognition, if the similarity between the two question-answer pairs following the main question exceeds the similarity threshold, the two question-answer pairs can be merged into one question-answer pair.

[0074] In some embodiments, each acquired question-answer pair may be set as a quality inspection item, and quality inspection may be performed on each quality inspection item subsequently. In one implementation, each acquired question-answer pair may be stored in a list structure as quality inspection item data, that is, a quality inspection item list may be acquired.

[0075] S105: Based on the text attributes and labels, perform quality inspection on each quality inspection item and generate a text quality inspection score.

[0076] In some embodiments, the corresponding quality inspection items can be checked for compliance based on the text attributes in the acquired pre-processed data and the labels of each conversation. In one method, each quality inspection item can be checked for compliance according to the order of the quality inspection item list. The method of checking whether the quality inspection item is compliant can include the following process:

[0077] S201: Detect whether the first quality inspection item contains a politeness label.

[0078] In some embodiments, the first quality inspection item (i.e., the first question and answer pair) is generally the opening dialogue between the user and the customer service, and it is necessary to detect whether the customer service uses polite language to the user in the conversation. Specifically, it can be based on whether the customer service dialogue in the first question and answer pair contains a polite language tag. If so, it is determined that the first quality inspection item is compliant; if not, it is determined that the first quality inspection item is non-compliant.

[0079] S202: Check whether the current quality inspection item is a complex quality inspection item. If it is a complex quality inspection item, execute S203; if it is not a complex quality inspection item, switch to the next quality inspection item and re-execute S202.

[0080] In some embodiments, the method of detecting whether the current quality inspection item is a complex quality inspection item may include:

[0081] Based on the text attributes in the dialogue text of the current quality inspection item, determine whether the emotional tendency of the user in the dialogue is negative. If it is determined that the text attribute is emot ion: 2, then it is determined that the user's emotional tendency is negative, and then it is determined that the user currently needs to be comforted, that is, the current quality inspection item is determined to be a complex quality inspection item. On the contrary, if the user's emotional tendency is not negative, it is determined that the current quality inspection item is not a complex quality inspection item, then switch to the next quality inspection item in the order of the quality inspection item list, and re-execute S202 to detect whether the next quality inspection item is a complex quality inspection item.

[0082] S203: Check whether the current complex quality inspection item is compliant. After the inspection is completed, switch to the next quality inspection item and re-execute S202.

[0083] In some embodiments, the method of detecting whether the current complex quality inspection item is compliant may include:

[0084] Based on the labels in the conversation text of the current complex quality inspection item, detect whether the customer service has any feedback behavior in the conversation text of the current complex quality inspection item, such as whether there is any behavior to appease the user. Specifically, it can be detected whether the conversation text contains a appeasement label. If the appeasement label is detected, it is determined that the current complex quality inspection item is compliant. If the appeasement label is not detected, it is determined that the current complex quality inspection item is not compliant. After the detection is completed, switch to the next quality inspection item in the order of the quality inspection item list, and re-execute S202 to detect whether the next quality inspection item is a complex quality inspection item.

[0085] After traversing each quality inspection item in the quality inspection item list through the above steps S201 to S203 to see if it is compliant, a text quality inspection result list is output, which contains the compliance test results of each quality inspection item. In the process of quality inspection item compliance detection, by combining the dialogue text and labels in the target text dialogue, the quality inspection item compliance detection can be completed quickly, without the need to perform natural language analysis on the text again, but by calling the previously generated text attributes and labels for comparison processing, the text quality inspection item compliance detection is realized, which improves the detection efficiency and reduces the computing power requirements.

[0086] In some embodiments, after obtaining the text quality inspection result list, a text quality inspection score may be performed based on the text quality inspection list.

[0087] In some embodiments, each quality inspection item in the text quality inspection result list has a corresponding score. Specifically, when the text quality inspection is scored, points can be deducted based on the deduction rules when the current quality inspection item is non-compliant. In one embodiment, the customer service has an initial score before the text quality inspection is scored, for example, 100 points. When performing text quality inspection scoring, points can be deducted for each non-compliant quality inspection item in the order of the text quality inspection result list, and the scoring result is output after the deduction of each non-compliant quality inspection item is completed. Among them, when the customer service score is deducted to 0 points, no deduction is performed, and the scoring result, that is, 0 points, is output. Among them, it should be noted that the score deducted when the quality inspection item is non-compliant can be pre-customized based on user needs, and this application does not limit this.

[0088] The above method can be used to output the text quality inspection score of the customer service.

[0089] S106: Performing a question and answer quality check on the customer service based on the question and answer pairs extracted from the target conversation text set, and outputting a question and answer quality check score.

[0090] In some embodiments, after extracting question-answer pairs based on the target conversation text set, the customer service can also be quality-checked based on each question-answer pair, and a corresponding question-answer quality-check score can be output.

[0091] In some embodiments, after the extraction of question-answer pairs is completed, the multiple question-answer pairs obtained can also be classified, wherein the type information used for classification can be set based on user needs, and this application does not limit the type of classification. In one embodiment, in a customer service scenario related to vehicles, the classification type can include professional categories and activity categories related to automobiles. Exemplarily, problems such as vehicle failures and vehicle performance are professional questions, and problems related to vehicle purchase discounts are activity questions.

[0092] In some embodiments, question-answer pairs can be classified using a preset classification model. Specifically, the acquired question-answer pairs can be input into the classification model. The classification model classifies multiple question-answer pairs based on a preset algorithm and outputs classification results.

[0093] In some embodiments, the classification information of the question and answer pairs can be consistent with the index of a pre-established knowledge base, so that rapid retrieval in the knowledge base based on the classification information can be achieved during question and answer evaluation, and the customer service's response information can be compared in parallel with the retrieved information in the knowledge base to evaluate the quality of the customer service's response.

[0094] In some embodiments, the knowledge base may be pre-generated, and the specific process may include:

[0095] First, you can obtain professional product knowledge information related to your business. For example, you can obtain knowledge related to automobile products. The types of knowledge include: acceptance and guidance of common vehicle faults, common vehicle use problems, three-guarantee related issues, maintenance issues, vehicle identification numbers and on-board tools, and activity FAQs.

[0096] After obtaining relevant professional product knowledge information, mark the key information in the knowledge information. For example, the following question and answer instructions can be used to mark the key information:

[0097] Q: Do I need to make an appointment in advance for maintenance?

[0098] A: Hello, in order to reduce your waiting time at the station, it is recommended that you make a maintenance appointment with the dealer at least one day in advance. You can make an appointment by clicking on the maintenance plan through the APP.

[0099] Based on the above question and answer example, key information can be further annotated and stored in a structured manner, with key information and weight name as key, and key information value and weight value as value storage. For example, {"info":"at least one day in advance","score":"80"}, {"info":"Maintenance evolution when clicking on the APP","score":"20"}.

[0100] In some embodiments, after the key information is annotated and stored, the knowledge and annotated information can be indexed and stored in a search server (El asticsearch) by knowledge type. For example, in the automotive professional field, the knowledge types can include: common car use problems, three guarantees related problems, maintenance problems, etc. In other professional fields, classification can be performed according to the corresponding types, and this application does not limit this.

[0101] In the process of customer service question and answer quality inspection, in some embodiments, different types of questions will have different weights and quality inspection complexity. In some single result questions, only one key information in the customer service needs to be checked for correctness. For example, what is the displacement of a certain brand of car and a certain model of car? When checking the customer service question and answer, it is only necessary to check whether the displacement information in the reply provided by the user is correct. Specifically, based on the classification information of the user's single result question, the answer corresponding to the question can be retrieved in the corresponding classification of the knowledge base. For example, based on the question, it can be determined that the question is a professional question, and it can be specifically divided into the vehicle performance branch, and then the result can be queried in the professional classification of the knowledge base. For example, the displacement of the vehicle asked by the user is 1.6L. According to the answer, it is determined whether the query result (1.6L) is included, and the judgment score is given. Specifically, the customer service question and answer quality inspection scoring can be the same or similar to the text quality inspection scoring method, for example, it can also be scored by deducting points, wherein when the customer service score is deducted to 0 points, no deduction is performed, and the scoring result is output. Similarly, before the customer service question and answer quality inspection score is calculated, an initial score, such as 100 points, can be set for the current customer service. The initial score can be customized based on needs, and this application does not impose any restrictions on this.

[0102] In some embodiments, in the above-mentioned single result question, the score of a key information corresponding to its answer can be set based on demand. For example, the score of a single key information can be 100 points. After comparing the customer service reply information with the retrieval information of the knowledge base, when it is determined that the customer service reply is wrong, for example, the correct answer retrieved from the knowledge base should be 1.6L, but the customer service reply content is 2.0L, then points will be deducted, such as deducting 100 points.

[0103] In some embodiments, some questions with high complexity of answers contain multiple key information, such as "When is the promotion for a certain brand of car?" The reply information may include multiple key information, such as "activity time," "activity location," "discount method," "participation method," etc., wherein, during the knowledge base construction process, scores may be set for the multiple key information in the reply according to the criticality of the information.

[0104] When scoring the quality of questions with complex answers, you can search for the corresponding answers in the corresponding categories of the knowledge base based on the classification information of the questions. The following key information can be obtained based on the search results:

[0105] [{“info”:“Activity time: October 1st to October 20th”,”score”:“100”},{“info”:“Only in certain areas”,”score”:“100”},{“info”:“Discount method: You can enjoy points exchange when purchasing a car”,”score”:“60”}],{“info”:“Participation method: Scan the code in the store to participate”,”score”:“50”}]

[0106] Among them, each score is the deduction value of the corresponding key information. After obtaining the above key information and the corresponding deduction value by searching the knowledge base, it is determined whether the current question and answer pair contains the retrieved key information. If not, the corresponding score is deducted.

[0107] Through the above method, each question and answer pair is traversed to complete the customer service question and answer quality inspection and scoring, and the question and answer quality inspection and scoring results are output. Through the above question and answer quality inspection and evaluation, the preset standard answers can be retrieved in the pre-established knowledge base, and the customer service reply content can be horizontally compared with the retrieved answers to detect whether the customer service reply content is comprehensive and professional. In turn, the comprehensiveness and professionalism of customer service quality inspection can be improved.

[0108] S1: Based on the customer service domain, filter out question-answer pairs that are not related to the customer service domain.

[0109] In some embodiments, before executing S106, the obtained question and answer pairs can be filtered for relevance by executing S1, and after the filtering is completed, the question and answer quality inspection and scoring are performed through S106.

[0110] It should be noted that in some application scenarios, users may ask questions that are not related to the customer service field. For example, the customer service field is related to automobiles. During the customer service process, the user asks a question that is not related to the automobile field, such as "Have you eaten?" When the above questions that are not related to the customer service field appear, the preset standard answer cannot be matched in the knowledge base, and the question and answer quality rating of the current question and answer pair cannot be performed.

[0111] In some embodiments, in order to reduce the possibility that such questions affect the accuracy of the overall quality inspection score, the obtained question and answer pairs can be filtered by a filtering model before the question and answer quality inspection score is performed, so as to filter out the question and answer pairs that are not related to the customer service field. Specifically, the filtering model can traverse each question and answer pair to perform relevance detection. For example, in the automotive field, when a question and answer pair that is not related to the automotive field is detected, the irrelevant question and answer pair is eliminated. After the traversal is completed, the retained question and answer pairs can be scored for question and answer quality inspection.

[0112] S107: A comprehensive score is obtained by performing weighted calculation based on the text quality inspection score and the question and answer quality inspection score.

[0113] In the above quality inspection and scoring, the text quality inspection focuses on the overall hard indicator evaluation to solve the problem of whether there is any problem; the question and answer quality inspection focuses on the customer service quality evaluation to detect whether the answers to the questions are accurate, complete and correct. Through the above two independent quality inspection calculations, the text quality inspection score A and the question and answer quality inspection score B can be obtained respectively, where the scores of A and B can be in percentage. In some embodiments, the weight of the text quality inspection score A can be pre-set to a and the weight of the question and answer quality inspection score B can be pre-set to b, and then the text quality inspection score A and the question and answer quality inspection score B can be weighted and summed to obtain the comprehensive score P. The specific calculation formula is as follows:

[0114] P = a*A+b*B, a+b=2;

[0115] It should be noted that the weight values ​​a and b can be adjusted accordingly according to the actual business scenario.

[0116] Figure 2 A schematic diagram of the structure of a customer service quality detection device provided in one embodiment of the present application.

[0117] Reference Figure 2 As shown, the customer service quality detection device may include a processor 100 and a memory 200, and the memory 200 is used to store at least one instruction, which, when loaded and executed by the processor 100, implements the customer service quality detection method provided in any embodiment of the present application.

[0118] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the customer service quality detection method provided by any embodiment of the present application is implemented.

[0119] An embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the customer service quality detection method provided in any embodiment of the present application.

[0120] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0124] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (Processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0125] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting customer service quality, characterized in that: The method comprises: Obtain the target dialogue text set; Preprocessing the target dialogue text set, setting text attributes for dialogues in the target dialogue text set, and obtaining a preprocessed information set, wherein the text attributes include time, emotional orientation, and behavior; Based on the preprocessed information set, configuring labels for the dialogues in the target dialogue text set by using preset business rules; Extracting question-answer pairs based on the target dialogue text set, setting each of the obtained question-answer pairs as a quality inspection item, and generating a quality inspection item list; Based on the text attributes and the labels, quality inspection is performed on the quality inspection items in the quality inspection item list and a text quality inspection score is generated.

2. The method according to claim 1, characterized in that: The target dialogue text set is preprocessed to set text attributes for dialogues in the target dialogue text set, and the preprocessed information set is obtained, including: Obtaining the interval between two adjacent dialogues in the target dialogue text set, wherein the time of the latter dialogue between the two adjacent dialogues includes the interval between the latter dialogue and the previous adjacent dialogue; Performing natural language analysis on the target dialogue text set to obtain emotional tendencies in the dialogues, where the emotional tendencies include positive, negative, and neutral; The behavior of the customer service personnel in the conversation is obtained, wherein the behavior includes questioning behavior, answering behavior, comforting behavior, insulting behavior, or no behavior.

3. The method according to claim 2, characterized in that Based on the preprocessed information set, configuring labels for the target dialogue text set through preset business rules includes: Based on the text attributes in the preprocessed information set, it is determined whether there are set behaviors of preset business rules in multiple dialogues in the target dialogue text set, and corresponding labels are set based on the detection results.

4. The method according to claim 3, characterized in that The performing quality inspection on the quality inspection items in the quality inspection item list based on the text attributes and the labels and generating text quality inspection scores comprises: Detecting whether the first quality inspection item in the quality inspection item list contains a politeness tag, and if it does not contain the politeness tag, deducting a corresponding score from the initial score of the text; Obtaining complex quality inspection items in the quality inspection item list; Check whether the complex quality inspection items are compliant, and if there are non-compliant complex quality inspection items, deduct the corresponding points and output the text quality inspection score.

5. The method according to claim 4, characterized in that The obtaining of complex quality inspection items in the quality inspection item list includes: Based on the text attributes in the conversation text of the quality inspection item, it is determined whether the emotional tendency of the user in the conversation is negative. If it is negative, the current quality inspection item is determined to be a complex quality inspection item.

6. The method according to claim 4, characterized in that The detection of compliance of the complex quality inspection items includes: Based on the label in the conversation text of the current complex quality inspection item, determine whether the customer service has soothing behavior in the conversation, wherein if there is a soothing label in the conversation text of the current complex quality inspection item, it is determined that the customer service has soothing behavior, and the current complex quality inspection item is determined to be compliant; if there is no soothing label, it is determined that the customer service has no soothing behavior, and the current complex quality inspection item is determined to be non-compliant.

7. The method according to any one of claims 4 to 6, characterized in that: After extracting question-answer pairs based on the target dialogue text set, the method further includes: Based on preset question and answer types in a preset knowledge base, the extracted question and answer pairs are classified, wherein the preset question and answer types include professional categories and activity categories; Based on the type of the question-answer pair, a preset standard answer is retrieved from the corresponding type of data in the knowledge base, wherein the preset standard answer includes a number of key information and scores corresponding to the key information; Detect whether the customer service response in the question and answer session contains the key information. If it is detected that any of the key information does not exist, deduct the corresponding score of the key information; After traversing each question-answer pair and checking the key information, the question-answer quality inspection score is output.

8. The method according to claim 1, characterized in that Before key information detection is performed on the question-answer pair, the following is also included: Based on the customer service field, question-answer pairs that are not relevant to the customer service field are filtered out.

9. The method according to claim 7, characterized in that: The method further comprises: A comprehensive score is obtained by weighted calculation based on the text quality inspection score and the question and answer quality inspection score.

10. A customer service quality detection device, characterized in that: The device comprises: A processor and a memory, wherein the memory is used to store at least one instruction, and when the instruction is loaded and executed by the processor, the customer service quality detection method as described in any one of claims 1-5 is implemented.

Images