A Customer Service Quality Inspection System Based on Large Language Model Distillation and Its Deployment Method

Abstract

This application belongs to the field of artificial intelligence and natural language processing technology, and provides a customer service quality inspection system and its deployment method based on large language model distillation. The steps are as follows: First, acquire multiple sets of labeled data, which are generated from multiple historical dialogues between customer service representatives and customers through risk labeling, including dialogue information and corresponding risk category labels; input each set of labeled data into a trained first semantic model, outputting a first probability label (i.e., a first quality inspection result) containing semantic judgment logic and probability distribution information for different quality inspection categories; combine multiple first probability labels with the first labeled data to train a second semantic model with a parameter scale significantly smaller than the first semantic model. This method can improve quality inspection efficiency while ensuring the quality inspection effect and reducing the hardware resource cost of model deployment.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence and natural language processing technology, and in particular relates to a customer service quality inspection system based on large language model distillation and its deployment method. Background Technology

[0002] In the field of customer service marketing, quality inspection of customer service call records is an important step in ensuring service quality and reducing risks.

[0003] Related technologies employ a large language model-based prompt word engineering approach, inputting call logs into large models such as DeepSeek-R1 and Qwen2.5-72B-Instruct, and using prompt word engineering to analyze relevant information. However, this method has significant limitations. The large language model-based prompt word engineering approach relies on large-scale models, requiring high computational resources, consuming large amounts of tokens, and has a slow processing speed, thus failing to meet the needs of rapid quality inspection. Summary of the Invention

[0004] This application provides a customer service quality inspection system and its deployment method based on large language model distillation, which can ensure the quality inspection effect, improve the quality inspection efficiency, and reduce the hardware resource cost of model deployment.

[0005] Firstly, embodiments of this application provide a customer service quality inspection method, including:

[0006] Obtain multiple sets of first-label data; wherein, the first-label data is generated by risk labeling processing of multiple sets of historical dialogues between customer service and customers, and the first-label data includes dialogue information and corresponding risk category labels;

[0007] Each set of first labeled data is input into the trained first semantic model to obtain the first probability label corresponding to each set of first labeled data; wherein, the first probability label is the first quality inspection result output by the first semantic model that has semantic judgment logic for the first labeled data and contains probability distribution information for different quality inspection categories;

[0008] A second semantic model is trained based on multiple first probability labels and first annotation data to obtain the trained second semantic model; wherein, the parameter size of the second semantic model is smaller than that of the first semantic model; wherein, the trained second semantic model is used to perform quality detection on the dialogue between customer service representatives and customers.

[0009] In this embodiment, customer service dialogue information after risk labeling, i.e., the first labeled data, is input into a trained first semantic model, enabling it to output first probability labels containing probability distributions and semantic judgment logic. Then, a second semantic model with a smaller parameter scale is trained based on these first probability labels. Finally, the lightweight second semantic model is used to implement customer service dialogue quality inspection. The soft labels of the first semantic model can convey deep semantic judgment logic, allowing the second semantic model to inherit the quality inspection accuracy of the teacher model while remaining lightweight, ensuring that the quality inspection effect is not compromised. Furthermore, the parameter scale of the second semantic model is much smaller than that of the first semantic model, requiring fewer computational resources at runtime, which can significantly reduce operating costs such as server deployment and computing power consumption, and improve quality inspection efficiency.

[0010] In one possible implementation of the first aspect, multiple sets of first-label data between customer service representatives and customers are obtained, including:

[0011] Obtain multiple sets of original historical dialogues between customer service representatives and customers;

[0012] For each set of original historical dialogues, the original historical dialogues are cleaned to obtain the cleaned second historical dialogues.

[0013] The second historical dialogue is segmented in chronological order to obtain the segmented second historical dialogue;

[0014] The first annotation data corresponding to each group of second historical dialogues is obtained based on the second historical dialogues after multi-group segmentation.

[0015] In this embodiment, the original historical dialogues are transformed into structured, high-quality labeled data materials, providing supervisory data that fits the customer service quality inspection scenario for the subsequent training of the second semantic model. This ensures that the model can learn accurate risk identification and compliance judgment logic from the data source, indirectly supporting the core invention goals of lightweight model and high accuracy.

[0016] In one possible implementation of the first aspect, the first annotation data corresponding to each group of second historical dialogues is obtained based on the multi-group segmented second historical dialogues, including:

[0017] Mark the identity information of customer service representatives and customers in the second historical dialogue after each segment to obtain the marked third historical dialogue;

[0018] Data tags were added to each group of third-historical dialogues involving risky customers and / or non-compliant customer service actions, resulting in multiple groups of first-labeled data.

[0019] In this embodiment, the segmented historical dialogues are labeled with identity information and risk / violation tags. The resulting first labeled data can provide high-quality data with clear semantic subjects and supervision signals for the subsequent training of the second semantic model. This ensures that the model learns accurate quality control logic from the source and supports the model in achieving the core goals of lightweight and high accuracy.

[0020] In one possible implementation of the first aspect, the first labeled data is input into the trained first semantic model to obtain a first probability label corresponding to each set of first labeled data, including:

[0021] Each set of first labeled data is input into the trained first semantic model, so that the trained first semantic model generates an original numerical vector based on the first labeled data and performs a probability transformation on the original numerical vector to obtain a first probability label; wherein, the original numerical vector is used to represent the probability of a risky customer or a non-compliant customer service action in the first labeled data;

[0022] Obtain the first probability label output by the first semantic model after training.

[0023] In this embodiment, the first labeled data is input into the trained first semantic model to generate an original numerical vector containing the probability of risk / violation and convert it into a first probability label. This label can serve as the core supervision signal for the subsequent training of the second semantic model, helping to transfer the semantic understanding capabilities of the large model to the lightweight model and providing key support for the lightweight quality inspection model to maintain high accuracy.

[0024] In one possible implementation of the first aspect, a second semantic model is trained based on multiple first probability labels and first annotation data to obtain the trained second semantic model, including:

[0025] Using each group of first labeled data and the first probability label corresponding to the first labeled data as training sample pairs, multiple training sample pairs are obtained;

[0026] Each training sample is input into the second semantic model to be trained. The second semantic model learns the first probability label and obtains the second label data output by the second semantic model. The second label data is a second quality inspection result containing semantic judgment logic and probability distribution.

[0027] Obtain the first loss of the training samples corresponding to the second label data; wherein, the first loss includes relative entropy loss and cross entropy loss. The relative entropy loss is used to measure the degree of difference between the second label data output by the second semantic model and the first probability label; the cross entropy loss is used to measure the degree of difference between the second quality inspection result corresponding to the second label data and the data labels in the first labeled data that label risky customers and / or illegal customer service actions.

[0028] If the first loss does not converge, the second semantic model is trained on the training samples until the first loss converges, thus obtaining the trained second semantic model.

[0029] In this embodiment, by constructing a training sample pair of "first labeled data + first probability label", the relative entropy loss is used to align the semantic capabilities of the large model, the cross-entropy loss is used to fit the manual labeling rules, and the training is terminated by loss convergence. The resulting second semantic model can inherit high semantic understanding capabilities while maintaining lightweight (the parameter scale is lower than that of the first semantic model), effectively solving the problems of high quality inspection cost and slow speed of traditional large models.

[0030] In one possible implementation of the first aspect, the method further includes:

[0031] Obtain the first real-time conversation between the customer service representative and the customer to be tested;

[0032] The quality of the first real-time dialogue is checked based on the trained second semantic model, and the third quality check result is obtained.

[0033] In this embodiment, a third quality inspection result is obtained by acquiring a first real-time dialogue and inputting the trained second semantic model (lightweight model). This not only enables rapid quality inspection by relying on the lightweight characteristics of the model, meeting the speed requirements of actual business, but also ensures the accuracy of quality inspection by relying on the semantic understanding capabilities inherited by the model, effectively solving the problems of high cost and slow speed of traditional large model quality inspection.

[0034] In one possible implementation of the first aspect, the method further includes:

[0035] Obtain the second labeled data corresponding to the first real-time dialogue; wherein, the second labeled data is the dialogue information after labeling risky customers and / or non-compliant customer service actions in the first real-time dialogue;

[0036] The third quality inspection result is compared with the data tags in the second labeled data that indicate risky customers and / or non-compliant customer service actions to obtain the first comparison result;

[0037] Based on the first comparison result, the trained second semantic model is optimized to obtain the optimized second semantic model.

[0038] In this embodiment of the application, the model quality inspection results (third quality inspection results) are compared with the labeled labels to obtain the difference conclusions (first comparison results), and the second semantic model is optimized accordingly. The core beneficial effect is that the lightweight second semantic model can continuously iterate based on the real labels of real-time dialogues, continuously improve the accuracy of customer service dialogue quality inspection, and better adapt to the dynamically changing quality inspection needs in actual business.

[0039] Secondly, embodiments of this application provide a quality inspection device, comprising:

[0040] The annotation data acquisition module is used to acquire multiple sets of first annotation data; wherein, the first annotation data is generated by risk annotation processing of multiple sets of historical dialogues between customer service and customers, and the first annotation data includes dialogue information and corresponding risk category annotations;

[0041] The probability label acquisition module is used to input each set of first labeled data into the trained first semantic model to obtain the first probability label corresponding to each set of first labeled data; wherein, the first probability label is the first quality inspection result output by the first semantic model, which has semantic judgment logic for the first labeled data and contains probability distribution information for different quality inspection categories;

[0042] The semantic model training module is used to train a second semantic model based on multiple first probability labels and first labeled data to obtain the trained second semantic model; wherein, the parameter size of the second semantic model is smaller than that of the first semantic model; wherein, the trained second semantic model is used to perform quality detection on the dialogue between customer service representatives and customers.

[0043] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the quality inspection method as described in any of the first aspects above.

[0044] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the quality inspection method as described in any of the first aspects above.

[0045] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute any of the quality inspection methods described in the first aspect above.

[0046] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1This is a flowchart illustrating the customer service quality inspection method provided in the embodiments of this application;

[0049] Figure 2 This is a schematic diagram of the process for obtaining labeled data provided in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of the process for obtaining probability tags provided in an embodiment of this application;

[0051] Figure 4 This is a schematic diagram of the process for training a second semantic model provided in an embodiment of this application;

[0052] Figure 5 This is a flowchart illustrating the real-time quality inspection process provided in an embodiment of this application;

[0053] Figure 6 This is a flowchart illustrating the optimized semantic model provided in an embodiment of this application;

[0054] Figure 7 This is a schematic diagram of the system architecture provided in the embodiments of this application;

[0055] Figure 8 This is a structural block diagram of the customer service quality inspection device provided in the embodiments of this application;

[0056] Figure 9 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0057] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0058] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0059] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0060] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0061] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0062] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0063] In the field of customer service and marketing, quality inspection of call records between customer service representatives and customers is a crucial step in ensuring service quality and reducing risks. Related technologies employ large language model prompt word engineering, which inputs call records into large models such as DeepSeek-R1 and Qwen2.5-72B-Instruct, and uses prompt word engineering to analyze relevant information. However, this method has significant limitations. Large language model prompt word engineering relies on large-scale models, requiring high computational resources, consuming large amounts of tokens, and has slow processing speed, making it unsuitable for rapid quality inspection needs.

[0064] To address the problems in the aforementioned related technologies, this application provides a customer service quality inspection method, apparatus, terminal device, and storage medium. It employs a technical solution based on large model prompt word engineering combined with model distillation. Specifically, a relatively small but high-performance large language model can be selected as the base model, and model distillation is used to compress and optimize the large language model. Through model distillation, the knowledge and semantic understanding capabilities of the large model can be transferred to a smaller model, enabling semantic understanding and compliance judgment of customer service and customer dialogue. This significantly reduces model operating costs and token consumption while ensuring the accuracy of quality inspection.

[0065] See Figure 1This is a flowchart illustrating the customer service quality inspection method provided in this application embodiment. It is intended as an example and not a limitation. The method may include the following steps:

[0066] S101, obtain multiple sets of first-label data; wherein, the first-label data is generated by risk labeling processing of multiple sets of historical dialogues between customer service and customers, and the first-label data includes dialogue information and corresponding risk category labels.

[0067] In this embodiment of the application, the first labeled data refers to a complete data unit containing "dialogue information + risk label" formed after "structured processing + risk dimension labeling" of multiple sets of historical dialogues between customer service and customers. It includes dialogue information and corresponding risk category labels and is the core basic data for subsequent model training.

[0068] In one embodiment, see Figure 2 This is a schematic diagram of the process for obtaining labeled data provided in an embodiment of this application, such as... Figure 2 As shown, step S101 includes:

[0069] S201, retrieve multiple sets of original historical dialogues between customer service and customers.

[0070] In this application embodiment, the original historical dialogue refers to the actual interactive dialogue records between customer service and customers in past customer service marketing scenarios, which can be obtained by extracting the voice call archives between customer service and customers (such as historical call recordings stored in the enterprise customer service system) and text call records (such as text chat logs of online customer service, including real-time dialogues on APP / web page).

[0071] The original historical dialogues need to cover different customer service business scenarios of the enterprise (such as financial customer service's financial consultation dialogues, e-commerce customer service's product recommendation dialogues, and education customer service's course sales dialogues) to avoid insufficient generalization ability of the model due to single business data. It needs to include risk-free routine consultations (such as "product usage method consultations"), "low-risk dialogues" (such as "routine product introductions"), and "high-risk dialogues" (such as "high-risk financial product recommendations") to ensure that subsequent risk labeling and model training can cover all risk scenarios.

[0072] S202, For each set of original historical dialogues, perform data cleaning on the original historical dialogues to obtain the cleaned second historical dialogues.

[0073] In this embodiment, to improve the quality of dialogue data and reduce interference with subsequent model training, each set of original historical dialogues needs to be cleaned to obtain a cleaned second historical dialogue. The core is to remove invalid interference information from the original dialogue through standardization processing, retain effective interaction content, and lay the foundation for subsequent "dialogue structuring" and "risk labeling".

[0074] Specifically, data cleaning can transform messy raw materials (original historical dialogues) into high-quality processing intermediates, namely second historical dialogues, by removing invalid characters, punctuation, duplicate content, and standardizing formats.

[0075] S203, divide the second historical dialogue into segments according to time sequence to obtain the segmented second historical dialogue.

[0076] In this embodiment, segmenting the second historical dialogue essentially achieves "dialogue structuring," transforming potentially continuous text blocks into independent interactive units that are "time-traceable and role-distinguished." The final segmented second historical dialogue must meet two key requirements: first, clear chronological order (strictly ordered according to the time of dialogue occurrence, without errors); and second, independent units (each segment corresponds to a complete "customer question / expression - customer service response" or an independent expression from a single party, facilitating subsequent segment-by-segment semantic and risk analysis).

[0077] S204, based on the second historical dialogue after multi-group segmentation, obtain the first annotation data corresponding to each group of second historical dialogues.

[0078] In this embodiment of the application, based on multiple sets of second historical dialogues (including dialogue sequence, roles and content) that have been segmented by time, corresponding risk-related labels are matched for each set of segmented dialogues, and finally the first labeled data exclusive to each set of second historical dialogues is formed. These data will serve as the basic material with risk labels required for subsequent model training.

[0079] The above method transforms the original historical dialogues into structured, high-quality labeled data materials, providing supervisory data that fits the customer service quality inspection scenario for the subsequent training of the second semantic model. This ensures from the data source that the model can learn accurate risk identification and compliance judgment logic, indirectly supporting the core invention goals of lightweight model and high accuracy.

[0080] In one embodiment, step S204 includes:

[0081] The identity information of customer service representatives and customers in the second historical dialogue after each group is segmented is marked to obtain the marked third historical dialogue; data labels are marked for risky customers and / or non-compliant customer service actions in each group of third historical dialogues to obtain multiple groups of first labeled data.

[0082] In this embodiment, for each group of segmented second historical dialogues, the identities of "customer service" and "customer" in the dialogue are supplemented and marked (clarifying the speaker role of each segment), forming a third historical dialogue that includes "time sequence segmentation + identity marking"; then, based on the third historical dialogue, annotations are made for the two core quality inspection dimensions of "whether risky customers are involved" and "whether there are any violations of customer service actions", and finally, the corresponding annotation results are matched for each group of third historical dialogues, forming multiple groups of first annotation data.

[0083] In the above method, the segmented historical dialogues are labeled with identity information and risk / violation tags. The resulting first labeled data can provide high-quality data with clear semantic subjects and supervision signals for the subsequent training of the second semantic model. This ensures that the model learns accurate quality inspection logic from the source, supporting the model to achieve the core goals of lightweight and high accuracy.

[0084] S102, input each set of first labeled data into the trained first semantic model to obtain the first probability label corresponding to each set of first labeled data; wherein, the first probability label is the first quality inspection result output by the first semantic model that has semantic judgment logic for the first labeled data and contains probability distribution information for different quality inspection categories.

[0085] In this embodiment of the application, the first semantic model after training can be a teacher model, such as DeepSeek-R1, or a basic semantic model that has been pre-trained. It needs to be trained in advance using customer service quality inspection related data (such as historical labeled data) to ensure that it has the ability to "understand the semantics of customer service dialogue and judge the quality inspection dimensions".

[0086] Each set of first-labeled data (e.g., a complete data unit containing "segmented dialogue + risk tags") is input into the "trained first semantic model". The input here includes not only the dialogue text but may also contain basic tag information from the first-labeled data (e.g., "preliminary labeling of customer risk attributes"), providing a reference for the model's semantic judgment. Based on its semantic understanding capabilities acquired during training, the first semantic model analyzes the input first-labeled data, not only determining whether the dialogue involves "risky customers" or "violations in customer service actions" (core document quality control dimensions), but also uncovering implicit semantic logic in the dialogue (e.g., "inducing behavior under normal vocabulary," avoiding the one-sidedness of keyword matching).

[0087] The model ultimately outputs a "first probability label," also known as a soft label. This soft label is not a simple hard judgment of "compliant / non-compliant," but rather a quality inspection result that includes the "probability distribution" of different quality inspection categories, i.e., the first quality inspection result (such as "probability of risky customers 0.92, probability of compliant customer service 0.06, probability of non-compliant customer service 0.02"). This result is accompanied by "semantic judgment logic" (i.e., the basis for the model to derive the probability distribution based on the semantics of the dialogue, such as "because the customer repeatedly expressed 'not afraid of risk and pursuing high returns,' the probability of a risky customer is high").

[0088] In one embodiment, see Figure 3 This is a schematic diagram of the process for obtaining probability tags provided in an embodiment of this application, such as... Figure 3 As shown, step S102 includes:

[0089] S301, each set of first labeled data is input into the trained first semantic model so that the trained first semantic model generates an original numerical vector based on the first labeled data and performs a probability transformation on the original numerical vector to obtain a first probability label; wherein, the original numerical vector is used to represent the probability of a risky customer or a non-compliant customer service action in the first labeled data.

[0090] In this embodiment, each set of "first labeled data" (such as segmented dialogues containing "customer inquiries about high-risk financial management + customer service inducement to purchase" and basic risk tags) is input into the "trained first semantic model". The input content not only includes dialogue text, but can also be associated with basic markers in the first labeled data (such as customer service / customer identity, segmented time sequence) to help the model accurately locate the semantic subject (such as judging that the "inducement action" comes from customer service). Based on the "customer service quality inspection semantic judgment logic" learned during training (such as recognizing the semantic features of "inducement language" and "high-risk demand"), the first semantic model analyzes the input first labeled data and outputs a "raw numerical vector" (raw logit value). The core function of this vector is to "quantify probability". Each value in the vector corresponds to a "raw prediction score" of a quality inspection dimension (such as a value of 9.2 for the "risky customer" dimension, 8.5 for the "illegal customer service action" dimension, and 1.3 for the "compliance" dimension), directly reflecting the model's initial judgment strength on "whether the first labeled data involves risky customers / illegal customer service actions".

[0091] The first semantic model processes the "original numerical vector" using a probability transformation technique: it uses a softmax function (or combines this with a temperature parameter T to smooth the distribution) to convert the original numerical vector into a "probability distribution" form, i.e., the "first probability label." For example, the original numerical vector (9.2, 8.5, 1.3) might, after transformation, output "risk customer probability 0.52, illegal customer service action probability 0.48, compliance probability 0.01." This result not only includes a judgment on "whether there is risk / violation," but also reflects the model's semantic judgment logic through the probability distribution (e.g., "the model believes this dialogue is more likely to involve risky customers and illegal customer service actions"). The probability transformation formula is as follows:

[0092] Let the logit output by the teacher model (i.e., the first semantic model after training) be... The generation process of the first probability label, i.e., the soft label, is as follows:

[0093]

[0094] in, It is a temperature parameter used to smooth the probability distribution, making the soft label more informative.

[0095] S302, Obtain the first probability label output by the first semantic model after training.

[0096] In this embodiment of the application, the first labeled data (structured segmented dialogue with risk labels) is input into the first semantic model (such as DeepSeek-R1) after training. After semantic analysis and probability transformation, the model obtains the first probability label output by the model, which contains the probability distribution of quality inspection dimensions such as "risky customer" and "violation of customer service actions".

[0097] In the above method, the first labeled data is input into the trained first semantic model to generate an original numerical vector containing the probability of risk / violation and convert it into a first probability label. This label can serve as the core supervision signal for the subsequent training of the second semantic model, helping to transfer the semantic understanding ability of the large model to the lightweight model and providing key support for the lightweight quality inspection model to maintain high accuracy.

[0098] S103, a second semantic model is trained based on multiple first probability labels and first annotation data to obtain the trained second semantic model; wherein, the parameter size of the second semantic model is smaller than that of the first semantic model; wherein, the trained second semantic model is used to perform quality detection on the dialogue between customer service and customer.

[0099] In this embodiment of the application, the core feature of the second semantic model is "lightweight". Its parameter scale is smaller than that of the first semantic model, which is equivalent to the student model corresponding to the first semantic model. For example, it is based on Qwen3-8B, Qwen3-4B or other lightweight architectures, and is used to imitate the behavior of the teacher model.

[0100] The second semantic model, or student model, is trained using the first probability label and the first labeled data. The purpose is to compress and optimize the large language model (i.e., the first semantic model) using model distillation. Through model distillation, the knowledge and semantic understanding capabilities of the large model can be transferred to a smaller model, enabling semantic understanding and compliance judgment of customer service and customer dialogues. This significantly reduces model operating costs and token consumption while ensuring the accuracy of quality inspection.

[0101] The training process follows the model distillation logic, which uses the first probability label (soft label) as the core supervision signal to guide the second semantic model to learn the semantic understanding ability of the first semantic model. At the same time, it combines hard labels in the first labeled data (such as manually labeled "compliant / non-compliant") to optimize the training. Through the objective function of "cross-entropy loss (fitting hard labels) + KL divergence (aligning soft label distribution)", it ensures that the second semantic model maintains a high quality inspection accuracy with a smaller parameter scale. For specific training steps, refer to steps S401-S404 below.

[0102] In the above method, customer service dialogue information after risk labeling, i.e., the first labeled data, is input into the trained first semantic model, enabling it to output first probability labels containing probability distributions and semantic judgment logic. Then, a second semantic model with a smaller parameter scale is trained based on these first probability labels. Finally, the lightweight second semantic model is used to implement customer service dialogue quality inspection. The soft labels of the first semantic model can convey deep semantic judgment logic, allowing the second semantic model to inherit the quality inspection accuracy of the teacher model while being lightweight, ensuring that the quality inspection effect is not compromised. In addition, the parameter scale of the second semantic model is much smaller than that of the first semantic model, requiring less computing resources at runtime, which can significantly reduce the operating costs such as server deployment and computing power consumption, and improve the quality inspection efficiency.

[0103] In one embodiment, see Figure 4 This is a schematic diagram of the process for training the second semantic model provided in an embodiment of this application, such as... Figure 4 As shown, step S103 includes:

[0104] S401, using each group of first labeled data and the first probability label corresponding to the first labeled data as training sample pairs, multiple training sample pairs are obtained.

[0105] In this embodiment of the application, each training sample pair includes two parts: "input" and "supervision signal". The "input" is the first labeled data (structured segmented dialogue with manual annotation, including customer service / customer identity, dialogue content and basic risk labels), and the "supervision signal" is the first probability label corresponding to the first labeled data (soft label with probability distribution output by the first semantic model). The two correspond one-to-one to ensure that the model can learn both the "explicit rules of manual annotation" and the "semantic judgment logic of the large model" at the same time during training.

[0106] These training sample pairs are the core data for training the second semantic model. By inputting them into the second semantic model to be trained, the model can be guided to learn the manually labeled hard labels in the first labeled data while aligning with the semantic understanding ability of the large model in the first probability label. This ultimately achieves the distillation goal of "small model inheriting the performance of large model" and avoids the problem of insufficient semantic understanding ability caused by traditional single hard label training.

[0107] S402, the training samples are input into the second semantic model to be trained, and the first probability label is learned through the second semantic model to obtain the second label data output by the second semantic model; wherein, the second label data is the second quality inspection result containing semantic judgment logic and probability distribution.

[0108] In this embodiment, one training sample from the constructed "training sample pair" (each sample pair consists of "first labeled data" and "corresponding first probability label," where the first labeled data is a structured customer service dialogue with manually labeled risk / compliance information, and the first probability label is a soft label with probability distribution dimensions such as "risky customer" and "violation of customer service actions" output by the first semantic model) is input into the second semantic model, which has not yet completed training. The core learning of the second semantic model revolves around the "first probability label," specifically:

[0109] The second semantic model first performs semantic analysis on the first labeled input data (such as identifying the risk needs expressed by customers and the compliance logic of customer service responses). Then, using the first probability label as a supervision signal, it adjusts its own parameters to fit the probability distribution characteristics of the label. For example, if the probability of "risky customer" in the first probability label is 0.92, the model will learn to make its predicted probability of "risky customer" in the same dialogue gradually approach that value, while mastering the semantic judgment logic contained in the label (such as the reasoning logic of "customer repeatedly mentions 'not afraid of losses and pursuing high returns' → judged as a high-risk customer"). This process conforms to the distillation logic of "training student models with soft labels as supervision signals".

[0110] After learning, the second semantic model outputs "second label data" for each training sample pair. This type of data has two key characteristics: first, it contains semantic judgment logic, meaning that the model output probability is not random, but based on the understanding of the dialogue semantics (e.g., because it recognizes that the customer service "recommended a high-risk product without warning of the risks," the probability of "customer service violation" is high); second, it presents a probability distribution form, rather than a single hard judgment of "compliant / non-compliant" (e.g., "probability of risky customer 0.99, probability of customer service compliance 0.11").

[0111] S403, obtain the first loss of the training samples corresponding to the second label data; wherein, the first loss includes relative entropy loss and cross entropy loss, the relative entropy loss is used to measure the degree of difference between the second label data output by the second semantic model and the first probability label; the cross entropy loss is used to measure the degree of difference between the second quality inspection result corresponding to the second label data and the label data in the first labeled data that annotates risky customers and / or illegal customer service actions.

[0112] In this embodiment, the core data consists of the second label data output by the second semantic model (soft labels output by the second semantic model), the first probability labels in the training samples (soft labels output by the first semantic model), and the hard labels in the first labeled data (manually labeled "risky customer / violation of customer service actions" explicit labels). The relative entropy loss (KL divergence loss function) and the cross-entropy loss (based on hard labels) are calculated separately, and both constitute the first loss. The formula is as follows:

[0113] KL divergence loss function:

[0114]

[0115] in, It is the KL divergence, used to measure the student model output. With teacher model output The distribution differences.

[0116] Total loss function:

[0117]

[0118] in, It is cross-entropy loss (based on hard labels). This is the loss weight (usually taken as 0.5).

[0119] From the perspective of relative entropy loss (KL divergence loss), it measures the difference in probability distribution between the second-label data and the first probability label to judge the learning effect of the second semantic model on the semantic judgment logic of the first semantic model. If the distributions of the two are closer (such as the second-label data "risky customer probability 0.91" and the first probability label "risky customer probability 0.92"), the smaller the relative entropy loss, it indicates that the model has inherited the semantic understanding ability of the large model well, which is in line with the distillation goal of "aligning the student and teacher model outputs with soft labels as supervision signals".

[0120] From the perspective of cross-entropy loss, it uses the hard labels in the first labeled data (such as manually labeled "this customer is a risky customer" or "customer service has violated regulations") as the benchmark, and measures the difference between the second quality inspection results corresponding to the second labeled data (such as the model predicting "risky customer probability 0.91, customer service violation probability 0.99") and the hard labels. If the model prediction results are more consistent with the hard labels (such as the hard label being "risky customer" and the model's probability for that dimension being close to 1), the cross-entropy loss is smaller, ensuring that the model's learning fits the quality inspection rules in actual business and avoids deviation from the real labels.

[0121] Ultimately, the first loss method, by fusing relative entropy loss and cross-entropy loss, ensures that the second semantic model inherits the semantic understanding ability of the first semantic model, while also making it conform to the real quality inspection standards of manual annotation. This provides a basis for subsequent model parameter optimization (such as adjusting weights through backpropagation), ultimately achieving the core invention objective of "maintaining high quality inspection accuracy with a lightweight model".

[0122] S404. If the first loss does not meet the convergence condition, continue training the second semantic model based on the training samples until the first loss converges, and obtain the trained second semantic model.

[0123] In this embodiment of the application, during the training process of the second semantic model, it is necessary to continuously monitor whether the calculated first loss (composed of relative entropy loss / KL divergence loss and cross-entropy loss) has reached the convergence condition (such as the loss value being stable below the preset threshold for multiple consecutive rounds, or the loss value no longer decreasing with the training rounds).

[0124] If the first loss does not meet the convergence condition, it indicates that the current model parameters still need optimization. Multiple training sample pairs (each pair containing the first labeled data and its corresponding first probability label) need to be input into the second semantic model to be trained, repeating the process of "model learning - outputting second labeled data - calculating the first loss," continuously adjusting the model parameters to reduce the loss. This iterative process needs to continue until the first loss reaches convergence. At this point, the second semantic model has fully learned the semantic judgment logic in the first probability label and the hard label rules in the first labeled data, ultimately obtaining a trained second semantic model. This model has a smaller parameter size than the first semantic model and can meet the lightweight and high-accuracy requirements of customer service dialogue quality inspection.

[0125] In the above method, by constructing training sample pairs of "first labeled data + first probability label", the relative entropy loss is used to align the semantic capabilities of the large model, the cross-entropy loss is used to fit the manual annotation rules, and the training termination condition is the convergence of the loss. The resulting second semantic model can inherit high semantic understanding capabilities while maintaining lightweight (the parameter scale is lower than that of the first semantic model), effectively solving the problems of high quality inspection cost and slow speed of traditional large models.

[0126] In one implementation, see Figure 5 This is a flowchart illustrating the real-time quality inspection process provided in an embodiment of this application, such as... Figure 5 As shown, it includes:

[0127] S501: Obtain the first real-time conversation between the customer service representative and the customer to be tested.

[0128] In this embodiment, the dialogue records between customer service and customers (including text-based dialogue or dialogue data converted from speech to text) are acquired in real time. This real-time dialogue data will be used as the raw material for subsequent quality inspection and will be directly transmitted to the "preprocessing module" for cleaning and structuring, providing basic input for the subsequent training of the second semantic model (lightweight quality inspection model) to perform quality inspection.

[0129] S502, perform quality checks on the first real-time dialogue based on the trained second semantic model, and obtain the third quality check result.

[0130] In this embodiment, the acquired first real-time dialogue (real-time interaction data between customer service and customer, such as text or voice-to-text dialogue) is preprocessed according to the standards in the "Data Acquisition and Preprocessing" section of the document (e.g., removing invalid characters, structuring by time sequence, and labeling customer service and customer identities) to ensure that the data format is consistent with the input format during the training of the second semantic model. Subsequently, the preprocessed first real-time dialogue is input into the trained second semantic model—this model corresponds to the "lightweight student model" in the document (e.g., based on the Qwen3-9B / 4B architecture), which has a smaller parameter scale than the first semantic model and has inherited the semantic understanding capabilities of the larger model through model distillation.

[0131] The second semantic model will perform semantic analysis on the first real-time dialogue based on the quality inspection logic learned during training, such as "risk customer identification" and "customer service compliance judgment", and output the third quality inspection result containing probability distribution (such as "risk customer probability 0.93, customer service violation probability 0.98").

[0132] In the above method, the third quality inspection result is obtained by acquiring the first real-time dialogue and inputting it into the trained second semantic model (lightweight model). This method can achieve rapid quality inspection by relying on the lightweight characteristics of the model, meeting the speed requirements of actual business, and can also ensure the accuracy of quality inspection by relying on the semantic understanding ability inherited by the model. This effectively solves the problems of high cost and slow speed of traditional large model quality inspection.

[0133] In one embodiment, see Figure 6 This is a flowchart illustrating the optimized semantic model provided in an embodiment of this application, such as... Figure 6 As shown, it includes:

[0134] S601, obtain the second labeled data corresponding to the first real-time dialogue; wherein, the second labeled data is the dialogue information after risk labeling of multiple sets of historical dialogues between customer service and customer.

[0135] In this embodiment, the core source of the second labeled data is multiple sets of historical dialogues between customer service representatives and customers (consistent with the "data source is call records between customer service representatives and customers" in the "data collection and preprocessing" section of the document). These historical dialogues need to be processed by the "dialogue structuring" process in the document first (segmented according to time sequence and labeled with the identities of customer service representatives and customers). Then, each set of historical dialogues is labeled with risks manually or semi-automatically (corresponding to the "label generation" method in the document). The labeling dimensions include whether risky customers are involved and whether customer service representatives have violated regulations. Finally, the second labeled data containing "structured dialogue content + risk labeling information" is formed.

[0136] The purpose of obtaining the second labeled data is to provide a reference for the quality inspection of the first real-time dialogue (such as assisting in judging the probability of similar historical risk scenarios in the real-time dialogue). Its data format and labeling logic must be consistent with the "first labeled data" (structured dialogue with risk labels) in the file to ensure that it can subsequently support the quality inspection of the first real-time dialogue by the trained second semantic model.

[0137] S602, compare the third quality inspection result with the label data in the second label data that marks risky customers and / or non-compliant customer service actions to obtain the first comparison result.

[0138] In this embodiment of the application, the core judgments corresponding to the probability distribution in the third quality inspection results (such as "probability of risky customer 0.92 → judged as risky customer" and "probability of illegal customer service action 0.99 → judged as illegal") are matched one by one with the hard labels of the same type of dialogue in the second labeled data to determine whether the two are consistent (such as the model's judgment that "risky customer" is consistent with the hard label "risky customer", or the model's judgment that "compliance" is inconsistent with the hard label "illegal"), and finally a first comparison result containing information such as "consistent / inconsistent" and "difference dimension" is formed.

[0139] S603, optimize the trained second semantic model based on the first comparison result to obtain the optimized second semantic model.

[0140] In this embodiment of the application, for the scenario where "the model judgment is inconsistent with the hard label" in the first comparison result, the corresponding first real-time dialogue data and the deviation dimension (such as the model misjudging "high-risk customer" as "normal customer") are extracted and added to the model training dataset.

[0141] Subsequently, using these biased labeled data as new supervision signals, the parameters of the trained second semantic model are fine-tuned using the loss function (cross-entropy loss + KL divergence) provided in this application. The semantic judgment weights of the model in the bias dimension are adjusted through backpropagation (such as enhancing the ability to recognize semantic features such as "customers repeatedly mention high-risk needs"), thereby gradually reducing the model's misjudgment rate.

[0142] This optimization process needs to be repeated until the "consistency rate between model judgment and hard label" in the first comparison result reaches a preset threshold, ultimately yielding the optimized second semantic model. This ensures that the model continuously improves semantic understanding accuracy and risk identification capabilities in actual quality inspection scenarios, consistently meeting the lightweight and high-accuracy requirements of customer service quality inspection.

[0143] In the above method, the model quality inspection results (third quality inspection results) are compared with the labeled labels to obtain the difference conclusions (first comparison results), and the second semantic model is optimized accordingly. The core beneficial effect is that the lightweight second semantic model can continuously iterate based on the real labels of real-time dialogues, continuously improve the accuracy of customer service dialogue quality inspection, and better adapt to the dynamically changing quality inspection needs in actual business.

[0144] See Figure 7 This is a schematic diagram of the customer service quality inspection system architecture provided in the embodiments of this application, such as... Figure 7 Specifically, it includes:

[0145] Data acquisition module: responsible for acquiring the conversation records between customer service and customers.

[0146] Preprocessing module: Cleans and structures the raw dialogue data.

[0147] Model Inference Module: Deploys lightweight models to perform quality checks on dialogue content.

[0148] Results output module: Outputs quality inspection results, including risk customer identification and customer service compliance assessment.

[0149] Feedback and optimization module: Continuously optimize model performance by comparing actual quality inspection results with manual annotations.

[0150] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0151] Corresponding to the customer service quality inspection method in the above embodiment, Figure 8 This is a structural block diagram of the customer service quality inspection device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0152] Reference Figure 8 The device includes:

[0153] The annotation data acquisition module 81 is used to acquire multiple sets of first annotation data; wherein, the first annotation data is generated by risk annotation processing of multiple sets of historical dialogues between customer service and customers, and the first annotation data includes dialogue information and corresponding risk category annotations;

[0154] The probability label acquisition module 82 is used to input each group of first labeled data into the trained first semantic model to obtain the first probability label corresponding to each group of first labeled data; wherein, the first probability label is the first quality inspection result output by the first semantic model that has semantic judgment logic for the first labeled data and contains probability distribution information for different quality inspection categories;

[0155] The semantic model training module 83 is used to train a second semantic model based on multiple first probability labels and first labeled data to obtain the trained second semantic model; wherein, the parameter size of the second semantic model is smaller than that of the first semantic model; wherein, the trained second semantic model is used to perform quality detection on the dialogue between customer service and customers.

[0156] Optionally, the labeled data acquisition module 81 is also used for:

[0157] Obtain multiple sets of original historical dialogues between customer service representatives and customers;

[0158] For each set of original historical dialogues, the original historical dialogues are cleaned to obtain the cleaned second historical dialogues.

[0159] The second historical dialogue is segmented in chronological order to obtain the segmented second historical dialogue;

[0160] The first annotation data corresponding to each group of second historical dialogues is obtained based on the second historical dialogues after multi-group segmentation.

[0161] Optionally, the labeled data acquisition module 81 is also used for:

[0162] Mark the identity information of customer service representatives and customers in the second historical dialogue after each segment to obtain the marked third historical dialogue;

[0163] Data tags were added to each group of third-historical dialogues involving risky customers and / or non-compliant customer service actions, resulting in multiple groups of first-labeled data.

[0164] Optionally, the probability tag acquisition module 82 is also used for:

[0165] Each set of first labeled data is input into the trained first semantic model, so that the trained first semantic model generates an original numerical vector based on the first labeled data and performs a probability transformation on the original numerical vector to obtain a first probability label; wherein, the original numerical vector is used to represent the probability of a risky customer or a non-compliant customer service action in the first labeled data;

[0166] Obtain the first probability label output by the first semantic model after training.

[0167] Optionally, the semantic model training module 83 is also used for:

[0168] Using each group of first labeled data and the first probability label corresponding to the first labeled data as training sample pairs, multiple training sample pairs are obtained;

[0169] Each training sample is input into the second semantic model to be trained. The second semantic model learns the first probability label and obtains the second label data output by the second semantic model. The second label data is a second quality inspection result containing semantic judgment logic and probability distribution.

[0170] Obtain the first loss of the training samples corresponding to the second label data; wherein, the first loss includes relative entropy loss and cross entropy loss. The relative entropy loss is used to measure the degree of difference between the second label data output by the second semantic model and the first probability label; the cross entropy loss is used to measure the degree of difference between the second quality inspection result corresponding to the second label data and the data labels in the first labeled data that label risky customers and / or illegal customer service actions.

[0171] If the first loss does not converge, the second semantic model is trained on the training samples until the first loss converges, thus obtaining the trained second semantic model.

[0172] In one embodiment, the quality inspection device 8 further includes a training module application module 84, used for:

[0173] Obtain the first real-time conversation between the customer service representative and the customer to be tested;

[0174] The quality of the first real-time dialogue is checked based on the trained second semantic model, and the third quality check result is obtained.

[0175] In one embodiment, the quality inspection device 8 further includes a training model optimization module 85, used for:

[0176] Obtain the second labeled data corresponding to the first real-time dialogue; wherein, the second labeled data is the dialogue information after labeling risky customers and / or non-compliant customer service actions in the first real-time dialogue;

[0177] The third quality inspection result is compared with the data tags in the second labeled data that indicate risky customers and / or non-compliant customer service actions to obtain the first comparison result;

[0178] Based on the first comparison result, the trained second semantic model is optimized to obtain the optimized second semantic model.

[0179] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0180] in addition, Figure 8The customer service quality inspection device shown can be a software unit, hardware unit, or a combination of software and hardware built into existing terminal equipment, or it can be integrated into the terminal equipment as an independent component, or it can exist as an independent terminal equipment.

[0181] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0182] Figure 9 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Figure 9 (Only one is shown in the image) a processor, a memory 91, and a computer program 92 stored in the memory 91 and executable on at least one processor 90, wherein the processor 90 executes the computer program 92 to implement the steps in any of the above customer service quality inspection method embodiments.

[0183] The terminal device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 9 This is merely an example of terminal device 9 and does not constitute a limitation on terminal device 9. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0184] The processor 90 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0185] In some embodiments, memory 91 may be an internal storage unit of terminal device 9, such as a hard disk or memory of terminal device 9. In other embodiments, memory 91 may be an external storage device of terminal device 9, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, etc., equipped on terminal device 9. Furthermore, memory 91 may include both internal storage units and external storage devices of terminal device 9. Memory 91 is used to store operating system, application programs, boot loader, data, and other programs, such as program code of computer programs. Memory 91 may also be used to temporarily store data that has been output or will be output.

[0186] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0187] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0188] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0189] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0190] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0191] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0192] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0193] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A customer service quality inspection method, characterized in that, The method includes: Obtain multiple sets of first-label data; wherein, the first-label data is generated by risk labeling processing of multiple sets of historical dialogues between customer service and customers, and the first-label data includes dialogue information and corresponding risk category labels; Each set of the first labeled data is input into the trained first semantic model to obtain the first probability label corresponding to each set of the first labeled data; wherein, the first probability label is the first quality inspection result output by the first semantic model that has semantic judgment logic for the first labeled data and contains probability distribution information for different quality inspection categories. A second semantic model is trained based on multiple first probability labels and first labeled data to obtain a trained second semantic model; wherein the parameter size of the second semantic model is smaller than that of the first semantic model; wherein the trained second semantic model is used to perform quality detection on the dialogue between customer service and customer. The step of training a second semantic model based on multiple first probability labels and first annotation data to obtain the trained second semantic model includes: Using each group of first labeled data and the first probability label corresponding to the first labeled data as training sample pairs, multiple training sample pairs are obtained; The training sample pairs are input into the second semantic model, and the first probability label is learned by the second semantic model to obtain the second label data output by the second semantic model; wherein, the second label data is a second quality inspection result containing semantic judgment logic and probability distribution; Obtain the first loss of the training samples corresponding to the second label data; wherein, the first loss includes relative entropy loss and cross entropy loss, the relative entropy loss is used to measure the degree of difference between the second label data output by the second semantic model and the first probability label; the cross entropy loss is used to measure the degree of difference between the second quality inspection result corresponding to the second label data and the data labels in the first labeled data that label risky customers and / or illegal customer service actions; If the first loss does not reach the convergence condition, the second semantic model is trained again based on the training samples until the first loss converges, thus obtaining the trained second semantic model.

2. The customer service quality inspection method as described in claim 1, characterized in that, The acquisition of multiple sets of first-labeled data includes: Obtain multiple sets of original historical dialogues between the customer service representative and the customer; For each group of original historical dialogues, the original historical dialogues are cleaned to obtain the cleaned second historical dialogues. The second historical dialogue is segmented in chronological order to obtain the segmented second historical dialogue; The first annotation data corresponding to each group of the second historical dialogue is obtained based on the second historical dialogue after it has been segmented into multiple groups.

3. The customer service quality inspection method as described in claim 2, characterized in that, The step of obtaining the first labeled data corresponding to each group of the second historical dialogue based on the multi-group segmented second historical dialogue includes: Mark the identity information of the customer service representative and the customer in the second historical dialogue after each segment to obtain the marked third historical dialogue; Data tags are added to each group of the third historical dialogues involving risky customers and / or non-compliant customer service actions, resulting in multiple groups of the first labeled data.

4. The customer service quality inspection method as described in claim 3, characterized in that, The step of inputting each group of the first labeled data into the trained first semantic model to obtain the first probability label corresponding to each group of the first labeled data includes: Each set of the first labeled data is input into the trained first semantic model, so that the trained first semantic model generates an original numerical vector based on the first labeled data and performs a probability transformation on the original numerical vector to obtain the first probability label; wherein, the original numerical vector is used to represent the probability of a risky customer or a non-compliant customer service action in the first labeled data. Obtain the first probability label output by the first semantic model after training.

5. The customer service quality inspection method as described in claim 1, characterized in that, The method further includes: Obtain the first real-time conversation between the customer service representative and the customer to be tested; The first real-time dialogue is subjected to quality detection based on the trained second semantic model, and a third quality inspection result is obtained.

6. The customer service quality inspection method as described in claim 5, characterized in that, The method further includes: Obtain the second annotation data corresponding to the first real-time dialogue; wherein, the second annotation data is the dialogue information after annotating risky customers and / or non-compliant customer service actions in the first real-time dialogue; The third quality inspection result is compared with the tag data in the second labeling data that marks risky customers and / or non-compliant customer service actions to obtain the first comparison result; Based on the first comparison result, the trained second semantic model is optimized to obtain the optimized second semantic model.

7. A customer service quality inspection device, characterized in that, include: The annotation data acquisition module is used to acquire multiple sets of first annotation data; wherein, the first annotation data is generated by risk annotation processing of multiple sets of historical dialogues between customer service and customers, and the first annotation data includes dialogue information and corresponding risk category annotations; The probability label acquisition module is used to input each group of the first labeled data into the trained first semantic model to obtain the first probability label corresponding to each group of the first labeled data; wherein, the first probability label is the first quality inspection result output by the first semantic model that has semantic judgment logic for the first labeled data and contains probability distribution information for different quality inspection categories. The semantic model training module is used to train a second semantic model based on multiple first probability labels and first labeled data to obtain a trained second semantic model; wherein the parameter size of the second semantic model is smaller than that of the first semantic model; wherein the trained second semantic model is used to perform quality detection on the dialogue between customer service and customer. The semantic model training module is also used for: Using each group of first labeled data and the first probability label corresponding to the first labeled data as training sample pairs, multiple training sample pairs are obtained; The training sample pairs are input into the second semantic model, and the first probability label is learned by the second semantic model to obtain the second label data output by the second semantic model; wherein, the second label data is a second quality inspection result containing semantic judgment logic and probability distribution; Obtain the first loss of the training samples corresponding to the second label data; wherein, the first loss includes relative entropy loss and cross entropy loss, the relative entropy loss is used to measure the degree of difference between the second label data output by the second semantic model and the first probability label; the cross entropy loss is used to measure the degree of difference between the second quality inspection result corresponding to the second label data and the data labels in the first labeled data that label risky customers and / or illegal customer service actions; If the first loss does not reach the convergence condition, the second semantic model is trained again based on the training samples until the first loss converges, thus obtaining the trained second semantic model.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images