A service quality evaluation method, device, equipment and storage medium thereof

Abstract

The embodiment of the application belongs to the technical field of financial science and technology, is applied to the scene of service quality evaluation of financial business seat service, and relates to a service quality evaluation method, device and equipment and a storage medium thereof, which comprises the following steps: controlling the segmentation processing length of the context of the target dialogue data and the interval of two paragraphs by a preset segmentation strategy, a first parameter and a second parameter, segmenting the dialogue context with the customer's utterance as the center, classifying the emotion of each segment, thereby more accurately analyzing the emotional changes in the dialogue, and only predicting the customer's emotional classification according to the customer's utterance, avoiding the emotional classification of the seat statement, reducing the calculation complexity and noise interference. The negative emotion priority strategy is used to ensure the accuracy of customer emotion recognition, and for the scene with a high neutral emotion proportion, the positive and negative emotions of the customer can be effectively captured, and the accuracy of the service quality evaluation is improved.

Description

Technical Field

[0001] This application relates to the field of financial technology and is applied to the scenario of evaluating the service quality of financial business agents. In particular, it relates to a service quality evaluation method, device, equipment and its storage medium. Background Technology

[0002] With the rapid development of the financial industry, financial companies are increasingly involved in a wide range of businesses. To improve customer service capabilities and fully meet diverse customer needs, many large enterprises have established corresponding call centers. Customers can access and process their business through these call centers without having to visit a branch counter. Currently, service quality assessment requires customers to manually evaluate the service quality of the call center staff after the service is completed. This allows the company to rate the staff's performance based on customer feedback. This manual rating method is cumbersome and relies solely on subjective customer evaluations, resulting in low accuracy.

[0003] Therefore, a number of AI-based service rating methods have emerged, which mainly predict agent service quality through dialogue data between agents and customers. However, at present, the dialogue data is mostly segmented by speaker, topic, or dialogue time. The above methods are prone to unreasonable segmentation, resulting in too many segmentation results, high computational complexity, and serious noise interference, which is not conducive to evaluating agent service quality. Summary of the Invention

[0004] The purpose of this application is to propose a service quality evaluation method, apparatus, device and storage medium to solve the problem that the segmentation processing of dialogue data in the existing technology for evaluating agent service quality is unreasonable, resulting in too many segmented results, high computational complexity and serious noise interference, which is not conducive to evaluating agent service quality.

[0005] To address the aforementioned technical problems, embodiments of this application provide a service quality evaluation method, employing the following technical solution:

[0006] A service quality evaluation method includes the following steps:

[0007] Collect target dialogue data;

[0008] The target dialogue data is segmented into context according to a preset segmentation strategy, a first parameter, and a second parameter to obtain a segmentation result, wherein the first parameter and the second parameter are both integer values ​​greater than 0.

[0009] The segmented processing results are input into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results for all segments.

[0010] Based on the negative sentiment priority strategy, the sentiment classification results of all segments are comprehensively analyzed to obtain the comprehensive analysis results corresponding to the target dialogue data;

[0011] Based on the comprehensive analysis results and the sentiment classification results of all segments, the service quality of the target agent is evaluated.

[0012] Furthermore, before performing the step of performing context segmentation processing on the target dialogue data according to the preset segmentation strategy, the first parameter, and the second parameter to obtain the segmentation processing result, the method further includes:

[0013] The target dialogue data is processed into text using a preset speech-to-text technology to obtain the target dialogue text.

[0014] Based on the preset speech recognition technology, the dialogue texts corresponding to all dialogue subjects in the target dialogue text are identified. The dialogue subjects include two different dialogue subjects: agents and customers.

[0015] Based on the different subjects in the dialogue, the target dialogue text is distinguished and marked according to the context order to obtain the marked target dialogue text.

[0016] Furthermore, the step of performing contextual segmentation processing on the target dialogue data according to the preset segmentation strategy, the first parameter, and the second parameter to obtain the segmentation processing result specifically includes:

[0017] Step 301: Obtain the distinguishing markers of the target customer as the first marker;

[0018] Step 302: Obtain the distinguishing marker of the target agent as the second marker;

[0019] Step 303: The marked target dialogue text is used as the text to be segmented.

[0020] Step 304: Select the first dialogue statement corresponding to the first mark from the text to be segmented, and use the statement preceding the first dialogue statement as the segmentation start statement;

[0021] Step 305: Identify the dialogue statement marked by the first parameter after the segment start statement according to the context order and the preset segmentation conditions, wherein the preset segmentation conditions are that there is at least one dialogue statement marked by the second parameter between two adjacent dialogue statements marked by the first tag.

[0022] Step 306: Using the statement following the first parameter of the dialogue statement marked by the first marker as the segment end statement, obtain the dialogue text between the segment start statement and the segment end statement.

[0023] Step 307: Update the dialogue text after the end statement of the segment to the text to be segmented, and repeat steps 304 to 307 until the dialogue text marked by the first parameter cannot be identified after the current starting segment according to the context order and the preset segmentation conditions, stop repeating the execution, and obtain the dialogue text between the starting statement and the ending statement of each segment.

[0024] Step 308: Take the dialogue text between the start and end statements of each segment as one segment in the segmentation result, organize all segments, and obtain the segmentation result.

[0025] Furthermore, before performing the step of inputting the segmented processing results into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results for all segments, the method further includes:

[0026] Obtain labeled dialogue data for sentiment classification training from a pre-set sentiment classification corpus;

[0027] The labeled dialogue data used for emotion classification training is input into the emotion classification model to be trained for training to obtain the preset emotion classification model. The emotion classification model to be trained contains a preset number of emotion classifiers. The emotion classifiers are classifiers based on the transformers architecture. The preset number of emotion classifiers includes a first number of negative emotion classifiers, a second number of neutral emotion classifiers, and a third number of positive emotion classifiers.

[0028] Furthermore, the step of inputting the segmented processing results into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results for all segments specifically includes:

[0029] Step 501: Select different segments from all the segments in sequence as the current segment to be tested;

[0030] Step 502: Obtain the dialogue data corresponding to the current segment to be tested;

[0031] Step 503: Input the dialogue data into the sentiment classification model to obtain the sentiment classification result corresponding to the dialogue data;

[0032] Step 504: Referring to a preset sentiment classification form, identify the sentiment classification characteristics corresponding to the dialogue data, wherein the sentiment classification characteristics include positivity, neutrality, and negativity;

[0033] Step 505: Based on the sentiment classification characteristics corresponding to the dialogue data, obtain the sentiment classification result corresponding to the current segment to be tested;

[0034] Step 506: Repeat steps 501 to 505 to obtain the sentiment classification results for all segments.

[0035] Furthermore, the step of comprehensively analyzing the sentiment classification results of all segments according to the negative sentiment priority strategy to obtain the comprehensive analysis result corresponding to the target dialogue data specifically includes:

[0036] Emotional characteristic analysis was performed on the sentiment classification results of all segments to obtain the analysis results;

[0037] If, based on the analysis results, negative classification results are identified in all the emotional classification results of the segments, then the comprehensive analysis result corresponding to the target dialogue data is a negative classification.

[0038] If the analysis results indicate that there are no negative classification results among all the emotional classification results of the segments, then the comprehensive analysis result corresponding to the target dialogue data is a positive classification.

[0039] Furthermore, the step of evaluating the service quality of the target agent by combining the comprehensive analysis results and the sentiment classification results of all segments specifically includes:

[0040] Obtain the pre-set benchmark score;

[0041] If the comprehensive analysis result is positive, then the percentage of positive sentiment classification results is calculated based on the sentiment classification results of all segments, and is used as the first percentage value.

[0042] The first floating score is calculated based on the first percentage value and the preset standard floating score.

[0043] The summation of the baseline score and the first floating score is obtained by summation operation, and is used as the service quality evaluation result of the target agent;

[0044] If the comprehensive analysis result is negative, then the percentage of negative emotional classification results is calculated based on the emotional classification results of all segments, and used as the second percentage value.

[0045] The second floating score is calculated based on the second percentage value and the standard floating score.

[0046] The difference between the baseline score and the second floating score is obtained through difference calculation, and is used as the service quality evaluation result of the target agent.

[0047] To address the aforementioned technical problems, this application also provides a service quality evaluation device, which employs the following technical solution:

[0048] A service quality evaluation device, comprising:

[0049] The target dialogue data acquisition module is used to collect target dialogue data;

[0050] The segmentation processing module is used to perform context segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter and a second parameter, and obtain a segmentation processing result, wherein the first parameter and the second parameter are both integer values ​​greater than 0.

[0051] The sentiment classification module is used to input the segmentation processing results into a preset sentiment classification model for sentiment classification, and obtain the sentiment classification results for all segments;

[0052] The comprehensive analysis module is used to perform a comprehensive analysis of the sentiment classification results of all segments according to the negative sentiment priority strategy, and obtain the comprehensive analysis results corresponding to the target dialogue data.

[0053] The service quality evaluation module is used to evaluate the service quality of the target agent by combining the comprehensive analysis results and the sentiment classification results of all segments.

[0054] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0055] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the service quality evaluation method described above.

[0056] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0057] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the service quality evaluation method as described above.

[0058] Compared with the prior art, the embodiments of this application have the following main advantages:

[0059] The service quality evaluation method described in this application involves collecting target dialogue data; performing contextual segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain segmentation processing results; inputting the segmentation processing results into a preset sentiment classification model for sentiment classification to obtain sentiment classification results for all segments; performing comprehensive analysis on the sentiment classification results of all segments according to a negative sentiment priority strategy to obtain comprehensive analysis results corresponding to the target dialogue data; and combining the comprehensive analysis results and the sentiment classification results of all segments to evaluate the service quality of the target agent. By controlling the segmentation length and the interval between two segments through the preset segmentation strategy, the first parameter, and the second parameter, the dialogue context is segmented with the customer's spoken statements as the center, and sentiment classification is performed on each segment. This allows for more accurate analysis of sentiment changes in the dialogue, and by predicting the customer's sentiment classification only based on the customer's spoken statements, sentiment classification of the agent's statements is avoided, reducing computational complexity and noise interference. Using a negative sentiment-first strategy ensures the accuracy of customer sentiment recognition. In scenarios where neutral sentiment accounts for a high proportion, it can effectively capture both positive and negative customer sentiment, thereby improving the accuracy of customer service evaluation. Attached Figure Description

[0060] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0062] Figure 2 This is a flowchart of an embodiment of the service quality evaluation method according to this application;

[0063] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown;

[0064] Figure 4 yes Figure 2 A segmented processing diagram of step 202 is shown below;

[0065] Figure 5 yes Figure 2 A flowchart of a specific embodiment of step 203 shown;

[0066] Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 204 shown;

[0067] Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 205 shown;

[0068] Figure 8 This is a schematic diagram of a structure of an embodiment of the service quality evaluation device according to this application;

[0069] Figure 9 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0071] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0072] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0073] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0074] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0075] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0076] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0077] It should be noted that the service quality evaluation method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the service quality evaluation device is generally set in the server / terminal device.

[0078] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0079] Continue to refer to Figure 2 A flowchart of an embodiment of the service quality evaluation method according to this application is shown. The service quality evaluation method includes the following steps:

[0080] Step 201: Collect target dialogue data.

[0081] In this embodiment, the target dialogue data includes voice conversations between agents and customers in financial business scenarios. For example, in insurance business, voice conversations between sales agents and potential customers; in credit card business, voice conversations between credit card service agents and credit card users.

[0082] Step 202: Perform context segmentation processing on the target dialogue data according to the preset segmentation strategy, the first parameter and the second parameter to obtain the segmentation processing result, wherein the first parameter and the second parameter are both integer values ​​greater than 0.

[0083] In this embodiment, before performing the step of performing contextual segmentation processing on the target dialogue data according to the preset segmentation strategy, the first parameter, and the second parameter to obtain the segmentation processing result, the method further includes: performing textual processing on the target dialogue data using a preset speech-to-text technology to obtain target dialogue text; identifying the dialogue text corresponding to all dialogue subjects in the target dialogue text based on a preset speech recognition technology, wherein the dialogue subjects include two different dialogue subjects: agent and customer; and distinguishing and marking the target dialogue text according to the different dialogue subjects and in the context order to obtain the marked target dialogue text.

[0084] In this embodiment, the speech-to-text technology and the speech recognition technology can be implemented using ASR speech recognition technology. However, in the step of texturing the target dialogue data using the preset speech-to-text technology to obtain the target dialogue text, the role of the ASR speech recognition technology is to convert the dialogue speech into text data. In the step of identifying the dialogue text corresponding to all dialogue subjects in the target dialogue text based on the preset speech recognition technology, the role of the ASR speech recognition technology is to identify different dialogue subjects and their corresponding dialogue texts based on the different timbre features in the acoustic features.

[0085] Continuing with the above-mentioned dialogue between a customer and an agent in a financial business scenario, firstly, the dialogue is transcribed into text using a pre-set speech-to-text technology to obtain the target dialogue text. Based on a pre-set speech recognition technology, the dialogue texts corresponding to all the main participants in the dialogue are identified, that is, the dialogue texts corresponding to the agent and the customer are identified separately. Then, according to the different participants, the target dialogue texts are distinguished and marked according to the context order to obtain the marked target dialogue text. That is, distinguishing marks are set in advance according to the differences between the agent and the customer, and the dialogue texts corresponding to the agent and the customer are marked separately using the distinguishing marks.

[0086] Continue to refer to Figure 3 , Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown includes:

[0087] Step 301: Obtain the distinguishing markers of the target customer as the first marker;

[0088] Step 302: Obtain the distinguishing marker of the target agent as the second marker;

[0089] Step 303: The marked target dialogue text is used as the text to be segmented.

[0090] Step 304: Select the first dialogue statement corresponding to the first mark from the text to be segmented, and use the statement preceding the first dialogue statement as the segmentation start statement;

[0091] Step 305: Identify the dialogue statement marked by the first tag with the first parameter after the segment start statement according to the context order and the preset segmentation conditions, wherein the preset segmentation conditions are that there is at least one dialogue statement marked by the second tag with the second parameter between two adjacent dialogue statements marked by the first tag.

[0092] Step 306: Using the statement following the first parameter of the dialogue statement marked by the first marker as the segment end statement, obtain the dialogue text between the segment start statement and the segment end statement.

[0093] Step 307: Update the dialogue text after the end statement of the segment to the text to be segmented, and repeat steps 304 to 307 until the dialogue text marked by the first parameter cannot be identified after the current starting segment according to the context order and the preset segmentation conditions, stop repeating the execution, and obtain the dialogue text between the starting statement and the ending statement of each segment.

[0094] Step 308: Take the dialogue text between the start and end statements of each segment as one segment in the segmentation result, organize all segments, and obtain the segmentation result.

[0095] Continue to refer to Figure 4 , Figure 4 yes Figure 2 The diagram shows a segmented processing step 202, wherein... Figure 4 Figure 4a shows a target dialogue text that has been marked. The first mark is c1, which represents the dialogue statement corresponding to the customer, and the second mark is t1, which represents the dialogue statement corresponding to the telephone agent. Figure 4b shows the dialogue text between the segment start statement and the segment end statement assuming the first parameter is 3 and the second parameter is 1. Figure 4c shows the remaining part of the segment when the first parameter is 3 and the second parameter is 1. Figure 4d shows the dialogue text between the segment start statement and the segment end statement assuming the first parameter is 3 and the second parameter is 2. Figure 4e shows the remaining part of the segment when the first parameter is 3 and the second parameter is 2. Obviously, by performing context segmentation processing on the target dialogue data through the preset segmentation strategy, the first parameter, and the second parameter, the segmentation processing result can be obtained. By adjusting the parameter values ​​of the first parameter and the second parameter, the target dialogue data can be flexibly segmented.

[0096] By flexibly segmenting the target dialogue data, the sentiment classification results corresponding to each segment are obtained. A comprehensive analysis of all sentiment classification results is then performed to obtain the overall sentiment analysis results of the target dialogue data. The dialogue context is segmented around the customer's spoken statements, and sentiment classification is applied to each segment. This allows for more accurate analysis of sentiment changes in the dialogue. Furthermore, the sentiment classification of the customer is predicted based solely on their spoken statements, avoiding the cumbersome process of previous methods that involved classifying sentiment for both the customer and the agent. Compared to previous methods, this approach reduces computational complexity and noise interference. By using a preset segmentation strategy and controlling the length of the context and the interval between two segments using a first and second parameter, the sentiment recognition performance and computational complexity are effectively balanced.

[0097] Step 203: Input the segmented processing results into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results for all segments.

[0098] In this embodiment, before performing the step of inputting the segmented processing results into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results of all segments, the method further includes: obtaining labeled dialogue data for sentiment classification training from a preset sentiment classification corpus; inputting the labeled dialogue data for sentiment classification training into the sentiment classification model to be trained for training to obtain the preset sentiment classification model, wherein the sentiment classification model to be trained contains a preset number of sentiment classifiers, the sentiment classifiers are classifiers based on the transformers architecture, and the preset number of sentiment classifiers includes a first number of negative sentiment classifiers, a second number of neutral sentiment classifiers, and a third number of positive sentiment classifiers.

[0099] By acquiring labeled dialogue data for emotion classification training from a pre-defined emotion classification corpus, where "labeled" refers to dialogue data already labeled with emotion categories, such as happy, neutral, and angry, where happy corresponds to a positive emotion classifier, neutral corresponds to a neutral emotion classifier, and angry corresponds to a negative emotion classifier, the labeled dialogue data for emotion classification training is input into the emotion classification model to be trained to obtain the pre-defined emotion classification model, where the pre-defined emotion classification model refers to the already trained emotion classification model. This facilitates subsequent steps in recognizing emotion categories from dialogue speech fragments.

[0100] Continue to refer to Figure 5 , Figure 5 yes Figure 2 A flowchart of a specific embodiment of step 203 shown includes:

[0101] Step 501: Select different segments from all the segments in sequence as the current segment to be tested;

[0102] Step 502: Obtain the dialogue data corresponding to the current segment to be tested;

[0103] Step 503: Input the dialogue data into the sentiment classification model to obtain the sentiment classification result corresponding to the dialogue data;

[0104] Step 504: Referring to a preset sentiment classification form, identify the sentiment classification characteristics corresponding to the dialogue data, wherein the sentiment classification characteristics include positivity, neutrality, and negativity;

[0105] Step 505: Based on the sentiment classification characteristics corresponding to the dialogue data, obtain the sentiment classification result corresponding to the current segment to be tested;

[0106] Step 506: Repeat steps 501 to 505 to obtain the sentiment classification results for all segments.

[0107] By obtaining the sentiment classification results for all segments, it is convenient to combine these results for comprehensive sentiment analysis of the target dialogue data. By flexibly segmenting the target dialogue data and obtaining the sentiment classification results for each segment, a comprehensive sentiment analysis result is obtained based on the sentiment classification results of all segments. The dialogue context is segmented around the customer's spoken statements, and sentiment classification is performed on each segment, thus more accurately analyzing the emotional changes in the dialogue. Furthermore, the customer's sentiment classification is predicted only based on their spoken statements, avoiding the cumbersome process of previous methods that classified sentiment for both the customer and the agent. Compared to previous methods, this approach reduces computational complexity and noise interference. By using a preset segmentation strategy and controlling the length of the context and the interval between two segments using the first and second parameters, a balance is effectively struck between sentiment recognition performance and computational complexity.

[0108] Step 204: Based on the negative sentiment priority strategy, perform a comprehensive analysis on the sentiment classification results of all segments to obtain the comprehensive analysis results corresponding to the target dialogue data.

[0109] Continue to refer to Figure 6 , Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 204 shown includes:

[0110] Step 601: Perform sentiment characteristic analysis on the sentiment classification results of all segments to obtain the analysis results;

[0111] In this embodiment, the step of performing emotional characteristic analysis on the emotional classification results of all segments to obtain the analysis results specifically includes: according to the negative emotion priority strategy, filtering the negative results of the emotional classification results of all segments, identifying whether there are classification results at the first number of negative emotion classifiers, if there are, then the comprehensive analysis result corresponding to the target dialogue data is a negative classification, if there are no, then the comprehensive analysis result corresponding to the target dialogue data is a positive classification.

[0112] Step 602: If, based on the analysis results, negative classification results are identified in all the emotional classification results of the segments, then the comprehensive analysis result corresponding to the target dialogue data is a negative classification.

[0113] Step 603: If, based on the analysis results, it is determined that there are no negative classification results among the sentiment classification results of all segments, then the comprehensive analysis result corresponding to the target dialogue data is a positive classification.

[0114] By employing the negative sentiment priority strategy, the comprehensive analysis result corresponding to the target dialogue data is determined. In essence, as long as there is a negative sentiment classification result for any segment, the comprehensive analysis result corresponding to the target dialogue data is a negative classification. When all segments are neutral or positive, the comprehensive analysis result corresponding to the target dialogue data is a positive classification.

[0115] Using a negative sentiment-first strategy ensures the accuracy of customer sentiment recognition. In scenarios where neutral sentiment accounts for a high proportion, it can effectively capture both positive and negative customer sentiment, thereby improving the accuracy of customer service evaluation.

[0116] Step 205: Based on the comprehensive analysis results and the sentiment classification results of all segments, evaluate the service quality of the target agent.

[0117] Continue to refer to Figure 7 , Figure 7 yes Figure 2 A flowchart of a specific embodiment of step 205 shown includes:

[0118] Step 701: Obtain the pre-set benchmark score;

[0119] Step 702: If the comprehensive analysis result is a positive classification, then the percentage of positive emotional classification results is calculated based on the emotional classification results of all segments, and used as the first percentage value.

[0120] Step 703: Calculate the first floating score based on the first percentage value and the preset standard floating score;

[0121] Step 704: By summing the base score and the first floating score, the sum of the base score and the first floating score is obtained as the service quality evaluation result of the target agent.

[0122] Step 705: If the comprehensive analysis result is negative, then the percentage of negative emotional classification results is calculated based on the emotional classification results of all segments, and used as the second percentage value.

[0123] Step 706: Calculate the second floating score based on the second percentage value and the standard floating score;

[0124] Step 707: Obtain the difference between the benchmark score and the second floating score through difference calculation, and use it as the service quality evaluation result of the target agent.

[0125] Specifically, for example, if the baseline score is 50 and the standard floating score is 50, assuming there are a total of 10 segments, if the comprehensive analysis result is a positive classification, it means that the sentiment classification result corresponding to the 10 segments is neutral or positive. If 7 positive sentiment classification results are found, then the first proportion is 70%, and the first floating score is 35. The service quality evaluation result of the target agent is the sum of the baseline score and the first floating score, which is 85. If the comprehensive analysis result is a negative classification, it means that there are negative sentiment classification results among the sentiment classification results corresponding to the 10 segments. If 7 negative sentiment classification results are found, then the second proportion is 70%, and the second floating score is 35. The service quality evaluation result of the target agent is the difference between the baseline score and the second floating score, which is 15.

[0126] By using a baseline score and a floating score, the service quality evaluation results of the target agent are obtained, reducing the complexity of calculating the service quality evaluation results.

[0127] This application collects target dialogue data; performs contextual segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain segmentation processing results; inputs the segmentation processing results into a preset sentiment classification model for sentiment classification to obtain sentiment classification results for all segments; performs comprehensive analysis on the sentiment classification results of all segments according to a negative sentiment priority strategy to obtain comprehensive analysis results corresponding to the target dialogue data; and combines the comprehensive analysis results and the sentiment classification results of all segments to evaluate the service quality of the target agent. By controlling the segmentation length and the interval between two segments through the preset segmentation strategy, the first parameter, and the second parameter, the dialogue context is segmented with the customer's spoken statements as the center, and sentiment classification is performed on each segment, thereby more accurately analyzing the emotional changes in the dialogue. Furthermore, it predicts the customer's sentiment classification only based on the customer's spoken statements, avoiding sentiment classification of the agent's statements, thus reducing computational complexity and noise interference. The negative sentiment priority strategy ensures the accuracy of customer sentiment recognition. In scenarios with a high proportion of neutral emotions, it can effectively capture both positive and negative customer emotions, improving the accuracy of customer service evaluation.

[0128] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0129] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0130] In this embodiment, the segmentation length and interval between two segments of the target dialogue data context are controlled by a preset segmentation strategy, a first parameter, and a second parameter. The dialogue context is segmented centered on the customer's spoken statements, and each segment is categorized by emotion. This allows for more accurate analysis of emotional changes in the dialogue. Furthermore, the customer's emotion category is predicted solely based on their spoken statements, avoiding emotion categorization of agent statements and reducing computational complexity and noise interference. A negative emotion-first strategy ensures the accuracy of customer emotion recognition. In scenarios where neutral emotions account for a high proportion, it effectively captures both positive and negative customer emotions, improving the accuracy of customer service evaluation.

[0131] Further reference Figure 8 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a service quality evaluation device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0132] like Figure 8 As shown, the service quality evaluation device 800 described in this embodiment includes: a target dialogue data acquisition module 801, a segmentation processing module 802, an emotion classification module 803, a comprehensive analysis module 804, and a service quality evaluation module 805. Wherein:

[0133] The target dialogue data acquisition module 801 is used to acquire target dialogue data.

[0134] The segmentation processing module 802 is used to perform context segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter and a second parameter, and obtain a segmentation processing result, wherein the first parameter and the second parameter are both integer values ​​greater than 0.

[0135] The emotion classification module 803 is used to input the segmentation processing results into a preset emotion classification model for emotion classification, and obtain the emotion classification results of all segments;

[0136] The comprehensive analysis module 804 is used to perform a comprehensive analysis on the sentiment classification results of all segments according to the negative sentiment priority strategy, and obtain the comprehensive analysis result corresponding to the target dialogue data.

[0137] The service quality evaluation module 805 is used to evaluate the service quality of the target agent by combining the comprehensive analysis results and the sentiment classification results of all segments.

[0138] This application collects target dialogue data; performs contextual segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain segmentation results; inputs the segmentation results into a preset sentiment classification model for sentiment classification to obtain sentiment classification results for all segments; performs comprehensive analysis on the sentiment classification results of all segments according to a negative sentiment priority strategy to obtain comprehensive analysis results corresponding to the target dialogue data; and combines the comprehensive analysis results and the sentiment classification results of all segments to evaluate the service quality of the target agent. By controlling the segmentation length and the interval between two segments through the preset segmentation strategy, the first parameter, and the second parameter, the dialogue context is segmented with the customer's spoken statements as the center, and sentiment classification is performed on each segment, thereby more accurately analyzing the emotional changes in the dialogue. Furthermore, it predicts the customer's sentiment classification only based on the customer's spoken statements, avoiding sentiment classification of the agent's statements, thus reducing computational complexity and noise interference. The use of a negative sentiment priority strategy ensures the accuracy of customer sentiment recognition. For scenarios with a high proportion of neutral emotions, it can effectively capture both positive and negative customer emotions, improving the accuracy of service evaluation.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0140] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0141] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.

[0142] The computer device 9 includes a memory 9a, a processor 9b, and a network interface 9c that are interconnected via a system bus. It should be noted that only the computer device 9 with components 9a-9c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0143] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0144] The memory 9a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 9a may be an internal storage unit of the computer device 9, such as the hard disk or memory of the computer device 9. In other embodiments, the memory 9a may also be an external storage device of the computer device 9, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 9. Of course, the memory 9a may include both the internal storage unit and its external storage device of the computer device 9. In this embodiment, the memory 9a is typically used to store the operating system and various application software installed on the computer device 9, such as computer-readable instructions for a service quality evaluation method. In addition, the memory 9a can also be used to temporarily store various types of data that have been output or will be output.

[0145] In some embodiments, the processor 9b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 9b is typically used to control the overall operation of the computer device 9. In this embodiment, the processor 9b is used to execute computer-readable instructions stored in the memory 9a or to process data, such as executing computer-readable instructions for the service quality evaluation method.

[0146] The network interface 9c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 9 and other electronic devices.

[0147] The computer equipment proposed in this embodiment belongs to the field of financial technology and is applied in the scenario of evaluating the service quality of financial business agents. This application collects target dialogue data; performs contextual segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain segmentation processing results; inputs the segmentation processing results into a preset sentiment classification model for sentiment classification to obtain sentiment classification results for all segments; performs comprehensive analysis on the sentiment classification results of all segments according to a negative sentiment priority strategy to obtain comprehensive analysis results corresponding to the target dialogue data; and combines the comprehensive analysis results and the sentiment classification results of all segments to evaluate the service quality of the target agent. By controlling the segmentation length and the interval between two segments through the preset segmentation strategy, the first parameter, and the second parameter, the dialogue context is segmented with the customer's spoken statements as the center, and sentiment classification is performed on each segment, thereby more accurately analyzing the sentiment changes in the dialogue. Furthermore, it predicts the customer's sentiment classification only based on the customer's spoken statements, avoiding sentiment classification of the agent's statements, thus reducing computational complexity and noise interference. Using a negative sentiment-first strategy ensures the accuracy of customer sentiment recognition. In scenarios where neutral sentiment accounts for a high proportion, it can effectively capture both positive and negative customer sentiment, thereby improving the accuracy of customer service evaluation.

[0148] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the service quality evaluation method described above.

[0149] The computer-readable storage medium proposed in this embodiment belongs to the field of financial technology and is applied to the scenario of evaluating the service quality of financial business agents. This application collects target dialogue data; performs contextual segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain segmentation processing results; inputs the segmentation processing results into a preset sentiment classification model for sentiment classification to obtain sentiment classification results for all segments; performs comprehensive analysis on the sentiment classification results of all segments according to a negative sentiment priority strategy to obtain comprehensive analysis results corresponding to the target dialogue data; and combines the comprehensive analysis results and the sentiment classification results of all segments to evaluate the service quality of the target agent. By controlling the segmentation length and the interval between two segments through the preset segmentation strategy, the first parameter, and the second parameter, the dialogue context is segmented with the customer's spoken statements as the center, and sentiment classification is performed on each segment, thereby more accurately analyzing the sentiment changes in the dialogue. Furthermore, it predicts the customer's sentiment classification only based on the customer's spoken statements, avoiding sentiment classification of the agent's statements, reducing computational complexity and noise interference. Using a negative emotion-first strategy ensures the accuracy of customer emotion recognition. In scenarios where neutral emotions account for a high proportion, it can effectively capture both positive and negative customer emotions, thereby improving the accuracy of service evaluations.

[0150] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0151] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A service quality evaluation method, characterized in that, Includes the following steps: Collect target dialogue data; The target dialogue data is segmented into context based on a preset segmentation strategy, a first parameter, and a second parameter to obtain a segmentation result. The first parameter and the second parameter are both integer values ​​greater than 0. Before executing the step of performing context segmentation on the target dialogue data according to the preset segmentation strategy, the first parameter, and the second parameter to obtain a segmentation result, the method further includes: The target dialogue data is processed into text using a preset speech-to-text technology to obtain the target dialogue text. Based on the preset speech recognition technology, the dialogue texts corresponding to all dialogue subjects in the target dialogue text are identified. The dialogue subjects include two different dialogue subjects: agents and customers. Based on the different subjects in the dialogue, the target dialogue text is distinguished and marked according to the context order to obtain the marked target dialogue text; The step of performing context segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain the segmentation processing result specifically includes: Step 301: Obtain the distinguishing markers of the target customer as the first marker; Step 302: Obtain the distinguishing marker of the target agent as the second marker; Step 303: The marked target dialogue text is used as the text to be segmented. Step 304: Select the first dialogue statement corresponding to the first mark from the text to be segmented, and use the statement preceding the first dialogue statement as the segmentation start statement; Step 305: Identify the dialogue statement marked by the first parameter after the segment start statement according to the context order and the preset segmentation conditions, wherein the preset segmentation conditions are that there is at least one dialogue statement marked by the second parameter between two adjacent dialogue statements marked by the first tag. Step 306: Using the statement following the first parameter of the dialogue statement marked by the first marker as the segment end statement, obtain the dialogue text between the segment start statement and the segment end statement. Step 307: Update the dialogue text after the end statement of the segment to the text to be segmented, and repeat steps 304 to 307 until the dialogue text marked by the first parameter cannot be identified after the current starting segment according to the context order and the preset segmentation conditions, stop repeating the execution, and obtain the dialogue text between the starting statement and the ending statement of each segment. Step 308: Take the dialogue text between the start statement and the end statement of each segment as a segment in the segmentation result, organize all segments, and obtain the segmentation result; The segmented processing results are input into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results for all segments. Based on the negative sentiment priority strategy, the sentiment classification results of all segments are comprehensively analyzed to obtain the comprehensive analysis results corresponding to the target dialogue data; Based on the comprehensive analysis results and the sentiment classification results of all segments, the service quality of the target agent is evaluated.

2. The service quality evaluation method according to claim 1, characterized in that, Before performing the step of inputting the segmented processing results into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results for all segments, the method further includes: Obtain labeled dialogue data for sentiment classification training from a pre-set sentiment classification corpus; The labeled dialogue data used for emotion classification training is input into the emotion classification model to be trained for training to obtain the preset emotion classification model. The emotion classification model to be trained contains a preset number of emotion classifiers. The emotion classifiers are classifiers based on the transformers architecture. The preset number of emotion classifiers includes a first number of negative emotion classifiers, a second number of neutral emotion classifiers, and a third number of positive emotion classifiers.

3. The service quality evaluation method according to any one of claims 1 to 2, characterized in that, The step of inputting the segmented processing results into a preset sentiment classification model for sentiment classification to obtain the sentiment classification results for all segments specifically includes: Step 501: Select different segments from all the segments in sequence as the current segment to be tested; Step 502: Obtain the dialogue data corresponding to the current segment to be tested; Step 503: Input the dialogue data into the sentiment classification model to obtain the sentiment classification result corresponding to the dialogue data; Step 504: Referring to a preset sentiment classification form, identify the sentiment classification characteristics corresponding to the dialogue data, wherein the sentiment classification characteristics include positivity, neutrality, and negativity; Step 505: Based on the sentiment classification characteristics corresponding to the dialogue data, obtain the sentiment classification result corresponding to the current segment to be tested; Step 506: Repeat steps 501 to 505 to obtain the sentiment classification results for all segments.

4. The service quality evaluation method according to claim 3, characterized in that, The step of comprehensively analyzing the sentiment classification results of all segments according to the negative sentiment priority strategy to obtain the comprehensive analysis result corresponding to the target dialogue data specifically includes: Emotional characteristic analysis was performed on the sentiment classification results of all segments to obtain the analysis results; If, based on the analysis results, negative classification results are identified in all the emotional classification results of the segments, then the comprehensive analysis result corresponding to the target dialogue data is a negative classification. If the analysis results indicate that there are no negative classification results among all the emotional classification results of the segments, then the comprehensive analysis result corresponding to the target dialogue data is a positive classification.

5. The service quality evaluation method according to claim 4, characterized in that, The step of evaluating the service quality of the target agent by combining the comprehensive analysis results and the sentiment classification results of all segments specifically includes: Obtain the pre-set benchmark score; If the comprehensive analysis result is positive, then the percentage of positive sentiment classification results is calculated based on the sentiment classification results of all segments, and is used as the first percentage value. The first floating score is calculated based on the first percentage value and the preset standard floating score. The summation of the baseline score and the first floating score is obtained by summation operation, and is used as the service quality evaluation result of the target agent; If the comprehensive analysis result is negative, then the percentage of negative emotional classification results is calculated based on the emotional classification results of all segments, and used as the second percentage value. The second floating score is calculated based on the second percentage value and the standard floating score. The difference between the baseline score and the second floating score is obtained through difference calculation, and is used as the service quality evaluation result of the target agent.

6. A service quality evaluation device, characterized in that, include: The target dialogue data acquisition module is used to collect target dialogue data; The segmentation processing module is used to perform context segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain a segmentation processing result. The first parameter and the second parameter are both integer values ​​greater than 0. Before performing context segmentation processing on the target dialogue data according to the preset segmentation strategy, the first parameter, and the second parameter to obtain a segmentation processing result, the module further includes: The target dialogue data is processed into text using a preset speech-to-text technology to obtain the target dialogue text. Based on the preset speech recognition technology, the dialogue texts corresponding to all dialogue subjects in the target dialogue text are identified. The dialogue subjects include two different dialogue subjects: agents and customers. Based on the different subjects in the dialogue, the target dialogue text is distinguished and marked according to the context order to obtain the marked target dialogue text; The step of performing context segmentation processing on the target dialogue data according to a preset segmentation strategy, a first parameter, and a second parameter to obtain a segmentation result specifically includes: Step 301: Obtain the distinguishing markers of the target customer as the first marker; Step 302: Obtain the distinguishing marker of the target agent as the second marker; Step 303: The marked target dialogue text is used as the text to be segmented. Step 304: Select the first dialogue statement corresponding to the first mark from the text to be segmented, and use the statement preceding the first dialogue statement as the segmentation start statement; Step 305: Identify the dialogue statement marked by the first parameter after the segment start statement according to the context order and the preset segmentation conditions, wherein the preset segmentation conditions are that there is at least one dialogue statement marked by the second parameter between two adjacent dialogue statements marked by the first tag. Step 306: Using the statement following the first parameter of the dialogue statement marked by the first marker as the segment end statement, obtain the dialogue text between the segment start statement and the segment end statement. Step 307: Update the dialogue text after the end statement of the segment to the text to be segmented, and repeat steps 304 to 307 until the dialogue text marked by the first parameter cannot be identified after the current starting segment according to the context order and the preset segmentation conditions, stop repeating the execution, and obtain the dialogue text between the starting statement and the ending statement of each segment. Step 308: Take the dialogue text between the start statement and the end statement of each segment as a segment in the segmentation result, organize all segments, and obtain the segmentation result; The sentiment classification module is used to input the segmentation processing results into a preset sentiment classification model for sentiment classification, and obtain the sentiment classification results for all segments; The comprehensive analysis module is used to perform a comprehensive analysis of the sentiment classification results of all segments according to the negative sentiment priority strategy, and obtain the comprehensive analysis results corresponding to the target dialogue data. The service quality evaluation module is used to evaluate the service quality of the target agent by combining the comprehensive analysis results and the sentiment classification results of all segments.

7. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the service quality evaluation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the service quality evaluation method as described in any one of claims 1 to 5.

Images