Customer service attitude detection method, system and device and medium
By collecting and converting chat files, combining the analysis of the binary classification model and Bert model, the problem of large error in customer service attitude detection in the existing technology is solved, and more accurate emotional judgment and service attitude evaluation are achieved.
Patent Information
- Application Number
- CN202411535762.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-25
AI Technical Summary
In the prior art, due to the inaccuracy of the telephone recording, the quality inspection results of customer service attitude detection are relatively large.
By collecting chat files between customers and customer service, converting them into text data, and using the binary classification model and the pre-trained Bert model, combining single-sentence text and contextual sentiment analysis, the service attitude of customer service is judged.
Detecting whether the emotions of both parties are escalating through contextual emotions combined with single sentence intentions, avoiding incorrect judgments caused by environmental noise and ASR transliteration problems, and improving the correctness of customer service emotional judgments.
Smart Images
Figure CN119513314A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology and relates to a customer service attitude detection method, system, device and medium. Background Art
[0002] In the work of telephone customer service in the express delivery industry, how to serve customers well is the top priority in daily work. How to monitor whether the customer service attitude is abnormal in real time and intervene in real time has always been an important and challenging issue. In the past, the common detection method for traditional customer service attitude issues was to upload the recording to the quality inspection platform after the call ended, and after ASR (speech to text), judge whether there is a service attitude problem based on keywords or the semantics of a single sentence. However, due to the collection of environmental noise in telephone recordings and inaccurate ASR (speech to text) transcription, the quality inspection results will have large errors. Summary of the invention
[0003] The purpose of the present invention is to solve the problem in the prior art that telephone recordings may have large errors in quality inspection results due to the collection of environmental noise and inaccurate ASR transcription, and to provide a customer service attitude detection method, system, device and medium.
[0004] In order to achieve the above object, the present invention adopts the following technical solutions:
[0005] A customer service attitude detection method, comprising:
[0006] Step 1: Collect chat files between customers and customer service, and convert the chat files into text data;
[0007] Step 2: Randomly select a single sentence from the text data and determine whether the selected single sentence contains a banned word; if yes, proceed to step 3; if no, traverse all text data until the traversal is completed;
[0008] Step 3: Select several rounds of dialogues before and after the current single sentence text to analyze the sentiment values of the customer and the customer service respectively, and determine whether the sentiment values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is determined to be an uncivilized text;
[0009] Step 4: Repeat steps 2 to 3, traverse all text data, obtain the number of uncivilized texts in the text data, and then judge the service attitude of the customer service in the conversation between the customer and the customer service.
[0010] A further improvement of the present invention is:
[0011] Furthermore, it is determined whether the selected single sentence text contains banned words, specifically: the selected single sentence text is input into the binary classification model for single sentence quality inspection, and the probability of whether the single sentence text contains banned words is output. If it exceeds the preset threshold, it is considered that the selected single sentence text contains banned words.
[0012] Furthermore, several rounds of conversations above and below the current single sentence text are selected to analyze the emotion values of the customer and the customer service respectively, and to determine whether the emotion values between the customer and the customer service exceed their respective thresholds. Specifically, the conversation between the customer and the customer service is input into the pre-trained Bert model, and the emotion of each sentence of the customer and the customer service is identified, and the proportion of the emotion of each sentence of the customer in this round of conversation exceeds the first threshold, and the proportion of the emotion of each sentence of the customer service in this round of conversation exceeds the second threshold. If both conditions are met at the same time, and the first-order difference of the emotion values of the customer and the customer service exceeds the set third threshold, then it is determined that the conversation triggers the label of uncivilized language.
[0013] Furthermore, the emotion types identified in a single sentence text include fear, anger, disgust, fear, liking, happiness and surprise, and the probability value of each type is within 1%-100%.
[0014] Furthermore, the pre-trained Bert model is as follows:
[0015] Select the historical text data of the original conversations between customers and customer service, use the large language generation model T5 model as the teacher model, use prompt to classify and label the historical text data, generate emotion labels corresponding to the text in batches, and use special keywords to correct the recognition results of the large model;
[0016] The word-level token-based BERT model is used as the student model, and the historical text data is divided into a training set and a test set;
[0017] The teacher model and the student model are trained and tested based on the training set and the test set, and a composite loss function including soft target loss and hard target loss is defined to fine-tune the hyperparameters of the Bert model. When the model effect of the Bert model is equivalent to the comprehensive classification effect of the large language generation model and the composite loss function is minimized, the optimized Bert model is output.
[0018] Furthermore, the soft target loss enables the student model to learn to imitate the output of the teacher model, and the hard target loss optimizes the prediction accuracy of the student model using the true labels;
[0019] The soft target loss is to generate the output probability distribution of the teacher model and the student model on the test data set respectively, and calculate the distance between the two probability distributions through KL divergence to form the soft target loss;
[0020] The calculation formula of KL divergence is:
[0021]
[0022] Among them, p(y) is the output probability of the teacher model, and q(y) is the output probability of the student model; KL divergence is the soft target loss
[0023] The hard target loss uses a cross entropy loss function to ensure that the student model accurately predicts the true label; for the true label y and the output p of the student model S (y|x), the hard target loss is:
[0024]
[0025] in, is an indicator function, y is the model prediction label; y = true is the true label; when y is equal to the true label, the model prediction label y takes the value of 1, otherwise it is 0. X represents the set of samples; p S (y|x) represents the probability that the current sample feature is predicted to be y;
[0026] The composite loss function adjusts the contribution of soft target loss and hard target loss through the weight factor λ, specifically:
[0027] L Composite =λL soft +(1-λ)L hard
[0028] Among them, the weight factor λ is an adjustable hyperparameter.
[0029] A customer service attitude detection system, comprising:
[0030] A text conversion module, which collects chat files between customers and customer service and converts the chat files into text data;
[0031] A first judgment module, wherein the first judgment module randomly selects a single sentence text from the text data, and judges whether the selected single sentence text contains a prohibited word; if not, traverses all text data until the traversal is completed;
[0032] A second judgment module, wherein the second judgment module selects several rounds of dialogues before and after the current single sentence text to analyze the emotion values of the customer and the customer service respectively, and judges whether the emotion values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is judged to be an uncivilized text;
[0033] The acquisition module traverses all text data, obtains the amount of uncivilized text in the text data, and then determines the service attitude of the customer service in the conversation between the customer and the customer service.
[0034] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0035] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] The present invention determines whether the selected single sentence text contains banned words, and selects several rounds of conversations before and after the current single sentence text to analyze the emotional values of the customer and the customer service respectively. When the emotional values between the customer and the customer service exceed their respective thresholds, the current single sentence text is determined to be an uncivilized text; the number of uncivilized texts in the text data is obtained, and then the service attitude of the customer service in the conversation between the customer and the customer service is determined. The present invention detects whether the emotions of both parties have escalated by combining contextual emotions with the intention of a single sentence, thereby avoiding erroneous judgments caused by environmental noise and ASR transcription problems; at the same time, the present invention detects whether the emotions of both parties have escalated by the difference value of the anger value ratio of several rounds of conversations between the customer and the customer service and the emotional value in the time series, which can improve the accuracy of the judgment of the customer service emotion. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0039] Figure 1 It is a flow chart of the customer service attitude detection method of the present invention;
[0040] Figure 2 It is a structural schematic diagram of the customer service attitude detection system of the present invention;
[0041] Figure 3 A topological diagram of the customer service attitude detection method of the present invention;
[0042] Figure 4 It is a system structure diagram of the telephone customer service client of the present invention. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0044] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0045] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0046] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.
[0047] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.
[0048] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0049] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0050] See also Figure 1 The present invention discloses a customer service attitude detection method, comprising:
[0051] S101: Collecting chat files between customers and customer service, and converting the chat files into text data;
[0052] S102: randomly select a single sentence text from the text data, and determine whether the selected single sentence text contains a prohibited word; if yes, proceed to S103; if no, traverse all text data until the traversal is completed;
[0053] The selected single sentence text is input into the binary classification model for single sentence quality inspection, and the probability of whether the single sentence text contains banned words is output. If it exceeds the preset threshold, the selected single sentence text is considered to contain banned words.
[0054] S103: Selecting several rounds of dialogues before and after the current single sentence text to analyze the emotion values of the customer and the customer service respectively, and judging whether the emotion values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is judged to be an uncivilized text;
[0055] The conversation between the customer and the customer service is input into the pre-trained Bert model to identify the emotions of each sentence of the customer and the customer service. The proportion of the emotion of each sentence of the customer in this round of conversation exceeds the first threshold, and the proportion of the emotion of each sentence of the customer service in this round of conversation exceeds the second threshold. If both conditions are met at the same time, and the first-order difference of the emotion values of the customer and the customer service exceeds the set third threshold, then it is determined that the conversation triggers the label of uncivilized language.
[0056] The emotion types identified in a single sentence text include fear, anger, disgust, fear, liking, happiness and surprise, and the probability value of each type is within 1%-100%.
[0057] The pre-trained Bert model is as follows:
[0058] Select the historical text data of the original conversations between customers and customer service, use the large language generation model T5 model as the teacher model, use prompt to classify and label the historical text data, generate emotion labels corresponding to the text in batches, and use special keywords to correct the recognition results of the large model;
[0059] The word-level token-based BERT model is used as the student model, and the historical text data is divided into a training set and a test set;
[0060] The teacher model and the student model are trained and tested based on the training set and the test set, and a composite loss function including soft target loss and hard target loss is defined to fine-tune the hyperparameters of the Bert model. When the model effect of the Bert model is equivalent to the comprehensive classification effect of the large language generation model, that is, the error of the F1-score is within 3%; and when the composite loss function is minimized, the optimized Bert model is output.
[0061] The soft target loss enables the student model to learn to imitate the output of the teacher model, and the hard target loss optimizes the prediction accuracy of the student model using the true label.
[0062] The soft target loss is to generate the output probability distribution of the teacher model and the student model on the test data set respectively, and calculate the distance between the two probability distributions through KL divergence to form the soft target loss;
[0063] The calculation formula of KL divergence is:
[0064]
[0065] Among them, p(y) is the output probability of the teacher model, and q(y) is the output probability of the student model; KL divergence is the soft target loss
[0066] The hard target loss uses a cross entropy loss function to ensure that the student model accurately predicts the true label; for the true label y and the output p of the student model S (y|x), the hard target loss is:
[0067]
[0068] in, is an indicator function, y is the model prediction label; y = true is the true label; when y is equal to the true label, the model prediction label y takes the value of 1, otherwise it is 0. X represents the set of samples; p S (y|x) represents the probability that the current sample feature is predicted to be y.
[0069] The composite loss function adjusts the contribution of soft target loss and hard target loss through the weight factor λ, specifically:
[0070] L Composite =λL soft +(1-λ)L hard
[0071] Among them, the weight factor λ is an adjustable hyperparameter.
[0072] S104: Repeat S102 to S103, traverse all text data, obtain the number of uncivilized texts in the text data, and then judge the service attitude of the customer service in the conversation between the customer and the customer service.
[0073] Based on the number of uncivilized texts in the text data, the percentage of the uncivilized texts in the text data is obtained, and according to the size of the percentage, the service attitude of the customer service in the conversation between the customer and the customer service is judged.
[0074] See also Figure 2 The present invention discloses a customer service attitude detection system, comprising:
[0075] A text conversion module, which collects chat files between customers and customer service and converts the chat files into text data;
[0076] A first judgment module, wherein the first judgment module randomly selects a single sentence text from the text data, and judges whether the selected single sentence text contains a prohibited word; if not, traverses all text data until the traversal is completed;
[0077] A second judgment module, wherein the second judgment module selects several rounds of dialogues before and after the current single sentence text to analyze the emotion values of the customer and the customer service respectively, and judges whether the emotion values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is judged to be an uncivilized text;
[0078] The acquisition module traverses all text data, obtains the amount of uncivilized text in the text data, and then determines the service attitude of the customer service in the conversation between the customer and the customer service.
[0079] Example:
[0080] The present invention constructs a set of data models based on post-ASR conversation chat records. The model is divided into three levels: single-sentence text semantics, customer emotion, and customer service emotion. For example: after each chat content is converted into text, the intention of the sentence will be identified, and the intention is called single-sentence text semantics. After generating the single-sentence text semantics, emotion recognition will be performed and an emotion value (1%-100%) will be obtained; finally, it will be determined by who said the sentence to distinguish the customer emotion value from the customer service emotion value. In a complete conversation content, there will be all single-sentence emotion values of the customer and the customer service.
[0081] Specifically include:
[0082] Collect chat files between customers and customer service, and convert chat files into text data;
[0083] Randomly select a single sentence from the text data and determine whether the selected single sentence contains a banned word; if not, traverse all the text data until the traversal is completed;
[0084] Select several rounds of dialogues before and after the current single sentence text to analyze the sentiment values of the customer and the customer service respectively, and judge whether the sentiment values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is judged to be an uncivilized text;
[0085] Traverse all text data, obtain the number of uncivilized texts in the text data, and then judge the service attitude of the customer service in the conversation between the customer and the customer service.
[0086] See also Figure 3 , the present invention includes the following parts:
[0087] (1) Single sentence quality inspection: Single sentence quality inspection is a text classification model based on BERT. Customer service manually annotates single sentence texts to build a data set of problematic service attitudes. Then, a binary classification model is trained to obtain a text classification model for single sentence quality inspection. This model is used to obtain the probability value of whether each sentence during the call contains civilized language.
[0088] (2) Emotion Recognition
[0089] Based on the characteristics of the colloquial and fine-grained emotional granularity of customer service conversations in the express delivery industry, this technical solution selects word-level tokens to better represent colloquial sentence vectors. At the same time, the customer service conversation text is labeled through a large model, and keyword filtering is performed on industry-specific intents such as complaints, post office upgrades, and news media exposure to construct a data set. The sentence vector representation of the CLS (beginning of a single sentence) position is obtained for multi-classification fine-tuning. Finally, the seven-category emotional probability values (1%-100%) are obtained for the text classification. The specific steps are as follows:
[0090] Step 1: Select a large language generation model as the teacher model, build a prompt for emotion classification and labeling, batch generate emotion labels corresponding to the text, and use special keywords to correct the recognition results of the large model.
[0091] Step 2: Build a BERT model based on word-level tokens as the student model. The dataset format generated in step 1 is adjusted to the training dataset of BERT.
[0092] Step 3: By defining a composite loss function that includes soft target loss and hard target loss, the soft target loss enables the student model to learn to imitate the output of the teacher model, and the hard target loss uses the true label to optimize the prediction accuracy of the student model;
[0093] The soft target loss is to generate the output probability distribution of the teacher model and the student model on the test data set respectively, and calculate the distance between the two probability distributions through KL divergence to form the soft target loss;
[0094] The calculation formula of KL divergence is:
[0095]
[0096] Among them, p(y) is the output probability of the teacher model, and q(y) is the output probability of the student model; KL divergence is the soft target loss
[0097] The hard target loss uses a cross entropy loss function to ensure that the student model accurately predicts the true label; for the true label y and the output p of the student model S (y|x), the hard target loss is:
[0098]
[0099] in, is an indicator function, y is the model prediction label; y = true is the true label; when y is equal to the true label, the model prediction label y takes the value of 1, otherwise it is 0. X represents the set of samples; p S (y|x) represents the probability that the current sample feature is predicted to be y;
[0100] The composite loss function adjusts the contribution of soft target loss and hard target loss through the weight factor λ, specifically:
[0101] L Composite =λL soft +(1-λ)L hard
[0102] Among them, the weight factor λ is an adjustable hyperparameter, and its value can be adjusted to optimize the performance of the model according to different training requirements.
[0103] Step 4: Fine-tune the Bert model and adjust hyperparameters such as learning rate and optimizer to optimize model performance;
[0104] Step 5: Evaluate the hit rate and recall rate of the Bert model. When the model effect is comparable to the combined effect of the large model and special keywords, end model fine-tuning.
[0105] (3) Design judgment criteria
[0106] After a single sentence is identified as an uncivilized customer service language, the sentiment value of the conversation between the two parties in the previous and next 5 rounds (10 sentences) will be obtained for analysis. The threshold of emotional anger is set to 50%, and when the sentiment value is greater than 50%, it is judged as emotional anger. When the proportion of sentences with angry customer emotions in the sliding window is greater than or equal to 50%, the proportion of sentences with angry customer emotions is greater than 10% (there is one sentence), and the first-order difference of the sentiment value is not too large, it is determined that the conversation triggers the label of uncivilized language. Among them, the sentiment threshold, the previous and next rounds, the percentage of angry emotions, and the first-order difference value of emotions are configurable parameters.
[0107] (4) Parameter Tuning
[0108] By regularly adjusting the sentiment threshold, conversation rounds, anger percentage, and sentiment first-order difference value through GridSearch on incremental online samples, optimization is achieved.
[0109] See also Figure 4 , the present invention also discloses a telephone customer service client, specifically comprising: a call reception module, a call quality inspection module and an intelligent auxiliary module;
[0110] The call reception module includes: conversation reception text display, recommended reply display and entity extraction;
[0111] The conversation reception text is displayed on the customer service front-end reception interface, which transcribes the current voice conversation between the customer and the customer service in real time and displays it in the chat box in order by time and chat object.
[0112] The recommended reply language is displayed in the chat box after the intent is recognized based on the content of the customer chat. The corresponding reply language configured through intent query is displayed in the chat box for customer service staff to refer to when replying to the customer.
[0113] Entity extraction is to obtain pre-configured entities such as (waybill number, customer surname, gender, etc.) by identifying the call content, and extract whether the call content contains the above information. If so, extract it and fill it into the customer service registration content.
[0114] The call quality inspection module includes: single-sentence AI quality inspection annotation, complete call AI quality inspection scoring, and manual re-inspection scoring;
[0115] Single-sentence AI quality inspection and annotation uses a single-sentence quality inspection model to inspect and annotate each sentence in the call content to distinguish whether the sentence is civilized language.
[0116] The AI quality inspection score for the entire call is a quality inspection scoring item combined with manual configuration. Each marked uncivilized language will receive a quality inspection score. After the entire conversation is over, the scores are summarized to obtain the final AI score result.
[0117] Manual re-examination and scoring are based on the AI scoring results. Humans can manually modify the AI scoring content to get the final score.
[0118] Intelligent auxiliary modules include: ASR speech-to-text module, single sentence quality inspection model and emotion-assisted quality inspection model;
[0119] The ASR speech-to-text module converts the chat between the customer and the customer service into text data;
[0120] The single sentence quality inspection model is used to determine whether the selected single sentence text contains banned words;
[0121] The emotion-assisted quality inspection model selects several rounds of dialogues before and after the current single sentence text to analyze the emotion values of the customer and the customer service respectively, and determines whether the emotion values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is determined to be an uncivilized text.
[0122] A terminal device is provided in one embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are implemented.
[0123] The computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to accomplish the present invention.
[0124] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.
[0125] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0126] The memory may be used to store the computer program and / or module, and the processor implements various functions of the terminal device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory.
[0127] If the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0128] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A customer service attitude detection method, characterized in that: include: Step 1: Collect chat files between customers and customer service, and convert the chat files into text data; Step 2: Randomly select a single sentence from the text data and determine whether the selected single sentence contains a banned word; if yes, proceed to step 3; if no, traverse all text data until the traversal is completed; Step 3: Select several rounds of dialogues before and after the current single sentence text to analyze the sentiment values of the customer and the customer service respectively, and determine whether the sentiment values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is determined to be an uncivilized text; Step 4: Repeat steps 2 to 3, traverse all text data, obtain the number of uncivilized texts in the text data, and then judge the service attitude of the customer service in the conversation between the customer and the customer service.
2. The customer service attitude detection method according to claim 1, characterized in that: The determining whether the selected single sentence text contains banned words is specifically as follows: the selected single sentence text is input into a binary classification model for single sentence quality inspection, and the probability of whether the single sentence text contains banned words is output. If the probability exceeds a preset threshold, it is considered that the selected single sentence text contains banned words.
3. The customer service attitude detection method according to claim 2, characterized in that: The method selects several rounds of dialogues above and below the current single sentence text to analyze the emotion values of the customer and the customer service respectively, and judges whether the emotion values between the customer and the customer service exceed their respective thresholds. Specifically, the dialogue between the customer and the customer service is input into the pre-trained Bert model, and the emotion of each sentence of the customer and the customer service is identified, and the proportion of the emotion of each sentence of the customer in this round of dialogue exceeds the first threshold, and the proportion of the emotion of each sentence of the customer service in this round of dialogue exceeds the second threshold. If both conditions are met at the same time, and the first-order difference of the emotion values of the customer and the customer service exceeds the set third threshold, then it is determined that the dialogue triggers a label of uncivilized language.
4. The customer service attitude detection method according to claim 3, characterized in that: The emotion types identified in the single sentence text include fear, anger, disgust, fear, liking, happiness and surprise, and the probability value of each type is within 1%-100%.
5. The customer service attitude detection method according to claim 4, characterized in that: The pre-trained Bert model is specifically: Select the historical text data of the original conversations between customers and customer service, use the large language generation model T5 model as the teacher model, use prompt to classify and label the historical text data, generate emotion labels corresponding to the text in batches, and use special keywords to correct the recognition results of the large model; The word-level token-based BERT model is used as the student model, and the historical text data is divided into a training set and a test set; The teacher model and the student model are trained and tested based on the training set and the test set, and a composite loss function including soft target loss and hard target loss is defined to fine-tune the hyperparameters of the Bert model. When the model effect of the Bert model is equivalent to the comprehensive classification effect of the large language generation model and the composite loss function is minimized, the optimized Bert model is output.
6. The customer service attitude detection method according to claim 5, characterized in that: The soft target loss enables the student model to learn to imitate the output of the teacher model, and the hard target loss optimizes the prediction accuracy of the student model using the true label; The soft target loss is to generate the output probability distribution of the teacher model and the student model on the test data set respectively, and calculate the distance between the two probability distributions through KL divergence to form the soft target loss; The calculation formula of KL divergence is: Among them, p(y) is the output probability of the teacher model, and q(y) is the output probability of the student model; KL divergence is the soft target loss L soft ; The hard target loss uses a cross entropy loss function to ensure that the student model accurately predicts the true label; for the true label y and the output p of the student model S (y|x), the hard target loss is: in, is an indicator function, y is the model prediction label; y = true is the true label; when y is equal to the true label, the model prediction label y takes the value of 1, otherwise it is 0. X represents the set of samples; p S (y|x) represents the probability that the current sample feature is predicted to be y; The composite loss function adjusts the contribution of soft target loss and hard target loss through the weight factor λ, specifically: THE Composite =λL soft +(1-λ)L hard Among them, the weight factor λ is an adjustable hyperparameter.
7. A customer service attitude detection system, characterized in that: include: A text conversion module, which collects chat files between customers and customer service and converts the chat files into text data; A first judgment module, wherein the first judgment module randomly selects a single sentence text from the text data, and judges whether the selected single sentence text contains a prohibited word; if not, traverses all text data until the traversal is completed; A second judgment module, wherein the second judgment module selects several rounds of dialogues before and after the current single sentence text to analyze the emotion values of the customer and the customer service respectively, and judges whether the emotion values between the customer and the customer service exceed their respective thresholds. If both exceed, the current single sentence text is judged to be an uncivilized text; The acquisition module traverses all text data, obtains the amount of uncivilized text in the text data, and then determines the service attitude of the customer service in the conversation between the customer and the customer service.
8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.