A dialogue quality inspection method and an electronic device
By combining model clustering and quality inspection strategies, the problem of insufficient generalization ability of deep learning models is solved, enabling dialogue quality inspection in different business scenarios, improving efficiency and accuracy, and reducing computing resource consumption.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN XIAOYING INFORMATION TECH CO LTD
- Filing Date
- 2026-06-30
- Publication Date
- 2026-07-31
AI Technical Summary
In existing technologies, deep learning models lack generalization ability when predicting the compliance of call behavior during dialogues, making them unsuitable for different business scenarios, and manual quality inspection is inefficient.
A combination of model clustering and quality inspection strategies is adopted. Multiple pre-trained first models output different types of behavioral data, and combined with quality inspection strategies based on business scenarios and lightweight second models and large language models, dialogue quality inspection is achieved.
It enables accurate quality inspection in different business scenarios, reduces manual labeling costs, improves quality inspection efficiency and diversity, and saves computing resources.
Smart Images
Figure CN122491256A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a dialogue quality inspection method and electronic device. Background Technology
[0002] In some business scenarios involving dialogue, the dialogue text between the speakers can be quality inspected to check the compliance of their behavior during the call. Currently, quality inspectors can judge the compliance of the speakers' behavior during the call, but this is subject to their subjective judgment and manual quality inspection is slow.
[0003] Based on this, deep learning models can be used in related technologies to predict the compliance of call behavior during dialogue. However, deep learning models have insufficient generalization ability. A single deep learning model can only accurately predict the compliance of call behavior in a single business scenario, but cannot accurately predict the compliance of call behavior in other different business scenarios. Summary of the Invention
[0004] This application provides a dialogue quality inspection method and electronic device, which can accurately inspect the behavioral data of dialogue participants in different business scenarios.
[0005] In a first aspect, embodiments of this application provide a dialogue quality inspection method, comprising: inputting dialogue text into a model cluster to obtain multiple types of behavioral data in a business scenario; the model cluster includes multiple first models, and different types of behavioral data are output by different first models, wherein the dialogue text is at least text of a dialogue between two interlocutors; obtaining multiple quality inspection strategies in the business scenario; each quality inspection strategy is associated with at least one type of behavioral data in the business scenario; performing quality inspection on the associated at least one type of behavioral data using the quality inspection strategy in the business scenario to obtain a first quality inspection result; the first quality inspection result is used to characterize that the behavior of a target interlocutor among the two interlocutors conforms to the quality inspection strategy in the business scenario; Obtain structured data output by the second model; the structured data includes target dialogue text and user sentiment corresponding to the target dialogue text; the target dialogue text includes a first dialogue text and a second dialogue text, the first dialogue text being the text in the dialogue text that conforms to the quality inspection strategy, and the second dialogue text being the context of the first dialogue text; The structured data and the first quality inspection result are input into the large language model to obtain the second quality inspection result.
[0006] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the dialogue quality inspection method provided in the first aspect.
[0007] The beneficial effects of this application's embodiments compared to existing technologies are as follows: Firstly, dialogue text can be input into multiple first models in a model cluster to obtain various types of behavioral data for the required business scenario. Then, a quality inspection strategy specific to that business scenario is applied to the behavioral data to obtain the inspection results. During this process, when the business scenario dynamically changes, the behavioral data required for the changed business scenario can be selected from the behavioral data output by the multiple first models, and the quality inspection strategy for the changed business scenario can be applied to the behavioral data, thereby achieving accurate quality inspection of behavioral data under different business scenarios. Furthermore, the different first models in the model cluster are pre-trained and do not require additional training. Therefore, utilizing multiple existing first models to obtain the behavioral data required for the business scenario reduces the manual annotation costs associated with training different first models.
[0008] Secondly, within the same business scenario, there are multiple quality inspection strategies. Different dimensions of quality inspection strategies can be used to inspect at least one type of behavioral data generated by the interlocutor during the dialogue process to obtain the first quality inspection result. This makes the process of inspecting behavioral data more diversified and refined, and provides more diversified and refined quality inspection results.
[0009] Thirdly, the structured data output from the second model and the first quality inspection result are then input into the large language model to obtain the second quality inspection result. This involves combining a lightweight first model and the second model with a heavyweight large language model. In this process, although the lightweight first model has poor generalization ability, the combination of multiple pre-trained first models can flexibly output behavioral data from different business scenarios. Furthermore, quality inspection strategies associated with these business scenarios are used to inspect the behavioral data, yielding preliminary quality inspection results. This addresses the issue of poor model generalization leading to an inability to adapt to multiple business scenarios. While the heavyweight large language model consumes significant computational resources, the lightweight first model already provides the preliminary first quality inspection result, and the lightweight second model provides the user's emotions and the corresponding target dialogue text. Therefore, the large language model does not need to perform step-by-step analysis and quality inspection of the dialogue text and emotions generated during the dialogue process, reducing resource consumption associated with quality inspection, saving computational resources of the large model, and thus enabling faster generation of the second quality inspection result. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 This is a schematic flowchart of a dialogue quality inspection method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the behavioral data output by multiple first models provided in an embodiment of this application; Figure 3 This is a schematic diagram of multiple quality inspection strategies provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the association of multiple quality inspection strategies with at least one type of behavioral data according to an embodiment of this application; Figure 5 This is a flowchart of the steps of a dialogue quality inspection method provided in an embodiment of this application; Figure 6 This is a schematic diagram of a first model that associates different business scenarios with different groups, provided in an embodiment of this application; Figure 7 This is a flowchart of the steps of a dialogue quality inspection method provided in an embodiment of this application; Figure 8 This is a schematic diagram illustrating the relationship between the first model, the second model, the third model, and the large language model provided in an embodiment of this application; Figure 9 This is a flowchart of the steps of a dialogue quality inspection method provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structured data output by the second model provided in an embodiment of this application; Figure 11 This is a block diagram of a dialogue quality inspection device provided in an embodiment of this application. Detailed Implementation
[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0013] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0014] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0015] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."
[0016] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0017] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0018] In related technologies, deep learning models can be used to predict the compliance of call behavior during dialogue. However, deep learning models have insufficient generalization ability. A single deep learning model can only accurately predict the compliance of call behavior in a single business scenario, but cannot accurately predict the compliance of call behavior in other different business scenarios.
[0019] Therefore, it is possible to train a single deep learning model using training data from different business scenarios, so that the deep learning model can learn call behavior in different business scenarios and predict the compliance of call behavior in different business scenarios. However, this training cost is too high because the amount of training data fed to the deep learning model in a single business scenario is huge, which requires staff to label a large number of training data. Moreover, when facing different business scenarios, even more training data is required, and the number of labels required for the training data will also be greater, which undoubtedly brings a lot of manual labeling costs.
[0020] Based on this, this disclosure proposes a dialogue-based quality inspection method. Figure 1 A schematic flowchart of the dialogue quality inspection method provided in this application is shown. It is for illustrative purposes only and not as a limitation. The dialogue quality inspection method includes the following steps: S101 inputs the dialogue text into the model cluster to obtain various types of behavioral data in the business scenario.
[0021] The dialogue text consists of at least two people engaging in a conversation.
[0022] For example, the dialogue text can be the text obtained by converting the voice during a call between a customer service representative and a customer, or it can be the text obtained by converting the voice during a call between multiple people.
[0023] The model cluster is a collection of multiple first models. Different types of behavioral data are output by different first models.
[0024] For example, please see Figure 2 As shown, the model cluster includes eight first models, namely Model 1 to Model 8. These ten first models output eight behavioral data points, namely, severely escalating conflict-related remarks, mildly escalating conflict-related remarks, customers making negative personal remarks, customers reporting to the police and filing complaints, customer service reassurance, asking for customer service employee ID, customer service responding with employee ID, and customers expressing dissatisfaction with customer service.
[0025] In this scenario, at least two participants will generate different dialogue scenarios throughout the conversation. The business scenario refers to the dialogue scenario under the dimension that needs to be inspected during the entire conversation. For example, the conversation may generate different dialogue scenarios such as violation scenarios, appeasement scenarios, and payment reminder scenarios. The business scenario can be the violation scenario that needs to be inspected.
[0026] In the business scenario of customer service and customer dialogue, both customer service representatives and customers generate different behavioral data. This behavioral data includes actions and corresponding behavioral tags. Behavioral tags are either 0 or 1, where 0 indicates the action does not exist and 1 indicates the action exists. This behavioral data can be generated by different parties in the dialogue. For example, in a dialogue between two parties, customer service and a customer, the behavioral data could be generated by either the customer service representative or the customer. For instance, behavioral data such as statements that escalate conflict, statements that mildly escalate conflict, and customer service representatives answering with their employee ID numbers are generated by the customer service representative, while behavioral data such as customers making negative personal statements, asking for customer service employee ID numbers, expressing dissatisfaction with customer service, and filing police reports are generated by the customer.
[0027] For example, after inputting the dialogue text between the speakers into Model 1, the behavioral data output by Model 1 includes severely escalating conflict-related verbal behaviors, with a behavioral label of 0 (indicating that there are no severely escalating conflict-related verbal behaviors); the behavioral data output by Model 2 includes mildly escalating conflict-related verbal behaviors, with a behavioral label of 1 (indicating that there are mildly escalating conflict-related verbal behaviors); the behavioral data output by Model 3 includes customers making negative personal statements, with a behavioral label of 1 (indicating that there are customers making negative personal statements), and so on. Subsequent Models 4 to 8 are similar and will not be listed here.
[0028] The first model outputs at least one type of behavioral data of the same type.
[0029] For example, the behavioral data output by Model 1 includes one instance of severely escalating conflict through verbal communication, with each instance labeled as 1, indicating that the customer service representative engaged in one instance of severely escalating conflict through verbal communication. The behavioral data output by Model 2 includes two instances of mildly escalating conflict through verbal communication, with each instance labeled as 1, indicating that the customer service representative engaged in two instances of mildly escalating conflict through verbal communication.
[0030] S102, obtain multiple quality inspection strategies under the business scenario.
[0031] In each business scenario, there are multiple quality inspection strategies. These strategies are different and have varying degrees of stringency. The quality inspection strategies vary depending on the business scenario.
[0032] For example, if the business scenario is a violation scenario, please refer to [link / reference]. Figure 3 As shown, the violation scenarios include quality inspection strategies 1 to 3. Quality inspection strategy 1 is hitting "severely escalating conflict-related remarks". Quality inspection strategy 2 includes hitting "mildly escalating conflict-related remarks" more than twice and not hitting "customer making negative personal remarks". Quality inspection strategy 3 includes hitting "mildly escalating conflict-related remarks" and the behavior of "mildly escalating conflict-related remarks" occurs before "customer reports to the police".
[0033] Each quality inspection strategy is associated with at least one type of behavioral data in a business scenario. The behavioral data associated with different quality inspection strategies in the same business scenario may be the same or different. Based on the behavioral types contained in the quality inspection strategy, the behavioral data of at least one type can be merged according to different quality inspection strategies. Then, a mapping relationship is established between the merged behavioral data and the adopted quality inspection strategy to obtain the behavioral data of at least one type associated with different quality inspection strategies.
[0034] For example, please see Figure 4 As shown, Model 1 outputs severely escalated conflict-related comments, Model 2 outputs mildly escalated conflict-related comments, Model 3 outputs negative personal comments made by customers, and Model 4 outputs customer complaints and police reports. Based on Quality Inspection Strategy 1, the behavioral data of severely escalated conflict-related comments can be used as the first data point; based on Quality Inspection Strategy 2, mildly escalated conflict-related comments and negative personal comments made by customers can be used as the second data point; and based on Quality Inspection Strategy 3, mildly escalated conflict-related comments and customer police reports and complaints can be used as the third data point. A mapping relationship is then established between the first data point and Quality Inspection Strategy 1, between the second data point and Quality Inspection Strategy 2, and between the third data point and Quality Inspection Strategy 3. This ensures that Quality Inspection Strategy 1 is associated with severely escalated conflict-related comments, Quality Inspection Strategy 2 is associated with mildly escalated conflict-related comments and negative personal comments made by customers, and Quality Inspection Strategy 3 is associated with mildly escalated conflict-related comments and customer police reports and complaints.
[0035] It is understandable that different models are trained using different types of training data and corresponding labels. For example, Model 1 can be trained using different texts of highly intensified conflict and corresponding behavioral labels, while Model 2 can be trained using different texts of mildly intensified conflict and corresponding textual labels, and so on.
[0036] S103, the quality inspection strategy under the business scenario is used to perform quality inspection on at least one type of related behavioral data to obtain a first quality inspection result.
[0037] The first quality inspection result is used to characterize the behavior of the target dialogue partner among the two dialogue partners, indicating that the behavior conforms to the quality inspection strategy under the business scenario. If the behavior data conforms to the quality inspection strategy under the violation scenario, it means that the dialogue partner has violated the rules; if the behavior data conforms to the quality inspection strategy under the appeasement scenario, it means that the dialogue partner has not violated the rules. The target dialogue partner includes at least one of the two dialogue partners. For example, in a dialogue between a customer service representative and a customer, the target dialogue partner can be at least one of the customer service representative and the customer.
[0038] The first quality inspection result includes at least one of the following: behavioral data that conforms to the quality inspection strategy, the quality inspection strategy, and the first dialogue text in which the behavioral data that conforms to the quality inspection strategy appears.
[0039] For example, the first quality inspection results include: Behavioral data includes the behavior of "customer service representatives making statements that escalate conflict" and the behavioral tag "mildly escalating conflict." Mildly escalating conflict is defined as customer service representatives making contemptuous, questioning, or urging statements towards customers, and customers making negative personal statements, defined as customers using uncivil or personally abusive language towards customer service representatives.
[0040] The quality control strategy includes situations where customer service representatives make two or more minor, escalating arguments, but the customer has not made any personal, negative comments.
[0041] The first dialogue text includes the original text of statements that slightly escalated the conflict and the original text of negative personal statements made by the customer. The original text of statements that slightly escalated the conflict is "Customer Service: If that's how you understand it, then there's nothing I can do" and "Customer Service: I've explained this problem three times already, don't you understand?" The original text of negative personal statements made by the customer is "Customer: XXX content".
[0042] Among them, the quality inspection strategy is used to inspect at least one type of related behavioral data to determine whether the behavioral data conforms to the quality inspection strategy. If it conforms to the quality inspection strategy, it means that the behavior indicated by the behavioral data is either illegal or not illegal.
[0043] Table 1. At least one type of behavioral data associated with different quality inspection strategies in violation scenarios.
[0044] For example, continuing the example in step S102, referring to the at least one type of behavioral data associated with different quality inspection strategies under the violation scenarios shown in Table 1, in the merged behavioral data, the behavioral label for severely escalating conflict is 0, which does not match quality inspection strategy 1; the behavioral label for mildly escalating conflict is 1 and appears twice, indicating that there are two instances of mildly escalating conflict; the behavioral label for customers making negative personal comments is 0, indicating that there are no instances of customers making negative personal comments, thus matching quality inspection strategy 2; the behavioral label for mildly escalating conflict is 1 and appears twice; the behavioral label for police reports and complaints is 0, indicating that there are no police reports and complaints, thus not matching quality inspection strategy 3. Therefore, under the violation scenarios shown in Table 1, after quality inspection of at least one type of associated behavioral data using different quality inspection strategies, since quality inspection strategy 2 is met, while quality inspection strategies 1 and 3 are not met, the first quality inspection result is the result obtained after quality inspection strategy 2, that is, the first quality inspection result matches "mildly escalating conflict" more than twice and does not match "customers making negative personal comments".
[0045] Through the above technical solution, firstly, dialogue text can be input into multiple primary models in the model cluster to obtain various types of behavioral data for the required business scenario. Then, a quality inspection strategy specific to that business scenario is applied to the behavioral data to obtain the quality inspection results. During this process, when the business scenario dynamically changes, the behavioral data required for the changed business scenario can be selected from the behavioral data output by the multiple primary models, and the quality inspection strategy for the changed business scenario can be applied to the behavioral data, thereby achieving accurate quality inspection of behavioral data under different business scenarios. Furthermore, the different primary models in the model cluster are pre-trained and do not require additional training, thus reducing the manual annotation costs associated with training different primary models.
[0046] Secondly, within the same business scenario, multiple quality inspection strategies can be adopted. Different dimensions of quality inspection strategies can be used to inspect at least one type of behavioral data generated by the interlocutor during the dialogue process to obtain the first quality inspection result. This makes the process of inspecting behavioral data more diversified and refined, and provides more diversified and refined quality inspection results.
[0047] Figure 5 This is an exemplary embodiment involving step S103 above, which is used to interpret an exemplary scheme for associating different business scenarios with different groups of first models, including the following steps: S103-1, Input the dialogue text into a set of first models associated with the business scenario to obtain various types of behavioral data under the business scenario.
[0048] The model cluster includes multiple sets of first models, and each set of first models includes multiple first models. The first models in different sets are associated with different business scenarios.
[0049] For example, see Figure 6 As shown, the model cluster includes models 1 to 8. Models 1, 2, 3 and 4 form a first model group, and the business scenario associated with this first model group is a violation scenario; Models 5 and 8 form a first model group, and the business scenario associated with this first model group is a reassurance scenario; Models 6 and 7 form a first model group, and the business scenario associated with this first model group is a payment reminder notification scenario.
[0050] Optionally, a set of first models associated with the business scenario can be selected first, and then the dialogue text can be input into the set of first models to obtain different types of behavioral data output by different first models in the set of first models; then, multiple quality inspection strategies associated with the business scenario can be obtained, and multiple quality inspection strategies can be used to inspect at least one type of associated behavioral data to obtain the first quality inspection result.
[0051] For example, taking a violation scenario as an example, the first model associated with the violation scenario includes Models 1 to 4, and the quality inspection strategies under the violation scenario are 1 to 3. We can first filter out Models 1 to 4 associated with the violation scenario, and obtain behavioral data on severely escalating conflict-related statements output by Model 1, behavioral data on mildly escalating conflict-related statements output by Model 2, behavioral data on customers making negative personal statements output by Model 3, and behavioral data on customers filing police reports and complaints output by Model 4. Then, we obtain the quality inspection strategies 1 to 3 associated with this business scenario. We use quality inspection strategy 1 to inspect the associated severely escalating conflict-related behavioral data, quality inspection strategy 2 to inspect the associated mildly escalating conflict-related and customer negative personal statements behavioral data, and quality inspection strategy 3 to inspect the associated mildly escalating conflict-related and customer complaint behavioral data, thus obtaining the first quality inspection result.
[0052] Optionally, the dialogue text can be input into the model cluster first to obtain different types of behavioral data output by different first models. Then, from the different types of behavioral data, a set of behavioral data output by the first model related to the business scenario can be selected. Then, multiple quality inspection strategies associated with the business scenario can be obtained, and multiple quality inspection strategies can be used to perform quality inspection on at least one type of associated behavioral data to obtain the first quality inspection result.
[0053] For example, taking a violation scenario as an example, the first model associated with the violation scenario includes models 1 to 4, and quality inspection strategies 1 to 3 are available under the violation scenario. First, different types of behavioral data output by models 1 to 8 can be obtained. Then, from these different types of behavioral data, the behavioral data output by models 1 to 4 associated with the violation scenario can be filtered out. Next, the quality inspection strategies 1 to 3 associated with this business scenario can be obtained. Quality inspection strategy 1 is used to inspect the associated behavioral data of severely escalating conflict, quality inspection strategy 2 is used to inspect the associated behavioral data of mildly escalating conflict and negative personal statements made by customers, and quality inspection strategy 3 is used to inspect the associated behavioral data of mildly escalating conflict and customer complaints, thus obtaining the first quality inspection result.
[0054] Through the above technical solution, when the business scenario changes, a set of first models associated with the changed business scenario outputs the behavioral data of the interlocutor in that business scenario. Then, a quality inspection strategy for the changed business scenario is used to inspect the behavioral data of the interlocutor in that business scenario, thus obtaining the first quality inspection result for that business scenario. In this process, even if the business scenario changes dynamically, a quality inspection strategy for the changed business scenario will still be used to inspect the behavioral data output by the first model associated with that business scenario to obtain the first quality inspection result. This transforms the solution of training a single first model for multiple scenarios into a solution of combining the first models required for each business scenario. This eliminates the need for repeated training of a single first model across multiple scenarios and training samples, reducing manual annotation costs while dynamically meeting the quality inspection needs of different business scenarios.
[0055] S104, Obtain the structured data output by the second model.
[0056] The structured data includes the target dialogue text and the user emotion corresponding to the target dialogue text. The user emotion corresponding to the target dialogue text is the emotion of the speaker when the speaker speaks the dialogue text.
[0057] The target dialogue text includes the first dialogue text and the second dialogue text. The first dialogue text is the text in the dialogue text that conforms to the quality inspection policy. It can also be understood as the dialogue text to which the behavioral data belongs when the behavioral data conforms to the quality inspection policy. The second dialogue text is the context of the first dialogue text. The context can be the text within a preset number of paragraphs above and below the first dialogue text, such as the text within 4 sentences above and below.
[0058] The second model can be a lightweight student model, such as Qwen3-4B or Qwen3-8B. Qwen3-4B is a lightweight language model with 4 billion parameters, and Qwen3-8B is also a lightweight model with 4 billion parameters. Both can be used as student models. Knowledge from the larger language model, which serves as the teacher model, is learned through distillation of the lightweight model, achieving lightweight quality control and obtaining the first quality control result.
[0059] S105, input the structured data and the first quality inspection result into the large language model to obtain the second quality inspection result.
[0060] For example, the first quality inspection result could be as follows: Behavioral data includes the behavior of "customer service representatives making statements that escalate conflict" and the behavioral tag "mildly escalating conflict." Mildly escalating conflict is defined as customer service representatives making contemptuous, questioning, or urging statements towards customers, and customers making negative personal statements, defined as customers using profanity or personal attacks against customer service representatives.
[0061] The quality control strategy includes situations where customer service representatives make two or more minor, escalating arguments, but the customers do not make any negative personal statements. The first dialogue text includes the original text of statements that slightly escalated the conflict and the original text of negative personal statements made by the customer. The original text of statements that slightly escalated the conflict is "Customer Service: If that's how you understand it, then there's nothing I can do" and "Customer Service: I've explained this problem three times already, don't you understand?" The original text of negative personal statements made by the customer is "Customer: XXX content".
[0062] For example, structured data can be as follows: Sentence _id 6: Customer (calmly): "I must resolve this today."
[0063] Sentence _id 7: Customer service (calmly) "I'm verifying for you."
[0064] Sentence _id 8: Customer service (anxiously): "If that's how you interpret it, then there's nothing I can do."
[0065] Sentence _id 9: Customer (angry): "What kind of attitude is this?"
[0066] … Sentence _id 14: Customer service (calmly) "Please confirm again."
[0067] Sentence _id 15: Customer service (anxiously): "I've explained this problem three times already, don't you understand?"
[0068] Sentence _id 16: Customer (angry): "I'm going to complain about you."
[0069] Sentence _id 17: Customer service (calmly) "Goodbye".
[0070] In this example, sentence id 8 and sentence id 15 are the first dialogue text, and the rest of the dialogue text is the context of the first dialogue text, that is, the second dialogue text. Each dialogue text contains the specific text content of the dialogue text, as well as the emotions of the speakers when they say that dialogue text.
[0071] Of course, in addition to mapping user emotions to target dialogue text, acoustic features such as user speech rate can also be mapped to target dialogue text to establish structured data among target dialogue text, user emotions, and user speech rate.
[0072] It's understandable that users speak at different paces depending on their emotions; for example, they speak faster when anxious and slower when calm. Therefore, speech rate can be used to help identify user emotions, leading to more accurate emotion assessments. Consequently, behavioral data identified based on more accurate emotion data will also be more accurate. Furthermore, speech rate can be used to generate quality control results. If customer service representatives speak too quickly, it can negatively impact customer experience. Therefore, large language models can be used to perform quality control on customer service speech rate, resulting in a second quality control result that includes the assessment of speech rate.
[0073] Optionally, the first quality inspection result, structured data, and prompt words can be input into the large language model, and the large language model can output the second quality inspection result.
[0074] Among them, prompt words are the task descriptions of the large language model, which are used to guide the large language model to generate a second quality inspection result that meets the user's needs. They can also be understood as prompt words used to guide the large language model to analyze dialogue text.
[0075] For example, the prompt could be, "You are a financial payment reminder and notification expert. Please conduct a secondary assessment of the behavioral data based on the initial assessment results and structured data."
[0076] The first quality inspection result is used to provide a basis for judgment for the large language model, and the large language model is used to make a second judgment on the accuracy of the first quality inspection result.
[0077] In this context, the structured data, which includes the second dialogue text within the context of the first dialogue text, serves to eliminate the different meanings of the same word. The same word may have different meanings in different dialogues; by combining the first dialogue text with the second dialogue text, these differing meanings can be eliminated.
[0078] For example, "millet" could refer to grains or mobile phones. By inputting the second dialogue text, which is the context of the first dialogue text, into the large language model, the large language model can understand the meaning of the words appearing in the first dialogue text, thus eliminating ambiguity.
[0079] The structured data includes user emotions within the target dialogue text, which can further assist the large language model in determining the accuracy of behavioral data appearing in the current dialogue text. Different behavioral data correspond to different user emotions. For example, if the behavioral data is highly inflammatory and confrontational, the corresponding user emotion is anger; if the behavioral data is mildly inflammatory and confrontational, the corresponding user emotion is anxiety.
[0080] The second quality inspection result output by the large language model includes the cause analysis of the first quality inspection result and the quality inspection conclusion obtained after the analysis.
[0081] For example, the cause analysis is as follows: (1) The first quality inspection result shows that the customer service statements id=8 and id=15 were marked as “mildly escalating conflict” by the second model. The number of hits was 2, and the customer made negative personal remarks = false (no customer made negative personal remarks). The form meets the violation scenario: the customer service made mildly escalating conflict statements 2 or more times and the customer did not make negative personal remarks.
[0082] (2) User emotion verification: id=8 emotion "anxious" and id=15 emotion "anxious", both are consistent with the semantics of "mildly intensified conflict-related remarks"; the customer only showed the emotion of "angry" but did not make any negative personal remarks, so the negative personal remarks made by the customer are still false.
[0083] For example, the quality inspection conclusion is as follows: Based on the comprehensive assessment, the violation scenario is established, and there are no indications that the initial assessment was a false alarm. Therefore, the violation conclusion in the first quality inspection result is upheld.
[0084] Among them, large language models can be pre-trained large language models such as GPT, Claude, and DeepSeek. The DeepSeek model is a large language model with strong semantic understanding and generation capabilities, with a parameter scale of approximately 671 billion. It is used as a teacher model to generate high-quality soft tags.
[0085] The large language model can be trained by fine-tuning (LORA) the professional vocabulary in the field of quality inspection. The fine-tuning formulas include the following formulas (1) and (2): (1) In formula (1), and It is a low-rank decomposition matrix. These are the updated parameters of LoRa, and also the weight parameters of the updated large language model.
[0086] (2) In formula (2), Y is the output of the second quality inspection result. This is the first quality check result, and x is the prompt word input to the large language model.
[0087] Through the above technical solution, a quality inspection strategy based on the business scenario can be used to inspect the behavioral data output by the first model associated with the business scenario, obtaining a first quality inspection result. Then, the structured data output by the second model and the first quality inspection result are input into the large language model to obtain a second quality inspection result. Firstly, it combines a lightweight first model and a second model with a heavyweight large language model. In this process, although the lightweight first model has poor generalization ability, the combination of multiple pre-trained first models can flexibly output behavioral data under different business scenarios, and use the quality inspection strategy associated with the business scenario to inspect the behavioral data, obtaining preliminary quality inspection results, thus solving the problem of poor model generalization ability leading to inability to adapt to multiple business scenarios. Although the heavyweight large language model consumes huge computational resources, the lightweight first model has already provided preliminary first quality inspection results, and the lightweight second model has already provided user emotions and corresponding target dialogue text. Therefore, the large language model does not need to perform step-by-step analysis and quality inspection of the dialogue text and emotions generated during the dialogue process, reducing the resource consumption of quality inspection, saving the computational resources of the large model, and thus enabling faster generation of the second quality inspection result.
[0088] Figure 7 This is an exemplary embodiment of the present disclosure, which, after obtaining a first quality inspection result through interpretation, combines the first quality inspection result, structured data, and entity keywords into a large language model to obtain a second quality inspection result, including the following steps: S106, Obtain the entity keywords output by the third model.
[0089] The entity keyword is at least one keyword in the dialogue text, and the at least one keyword is used together to represent a summary of the dialogue text. The entity keyword includes basic information in the dialogue text.
[0090] For example, entity keywords can be keywords that appear in the chat between customer service and customers, such as "the person who was notified of the payment reminder", "the bill amount is 5800 yuan", "the number of overdue days is 37 days", "employee number 112233", and "the payment reminder notification stage is M2".
[0091] The third model can be a BiLSTM-CRF (Bidirectional Long Short-Term Memory Network-Conditional Random Field) model, which is used to extract entity keywords specified by the third model from the input dialogue text. The expression of the third model is as follows: (3) In formula (3), is the output of the hidden layer of the BiLSTM (Bidirectional Long Short-Term Memory Network) in the third model; is the emission probability, that is, the score of the entity label , and there is a score corresponding to the entity label for each subword; is the transition probability, which is the transition matrix of the CRF (Conditional Random Field) in the third model, and it records the score of transitioning from the previous entity label to the current entity label; Z is the normalization factor; is the conditional probability of the normalized entity keyword output by the third model, with a range of 0 to 1. The larger the value, the greater the probability of the output entity keyword; is the subword sequence in the input dialogue text; is a single entity label, corresponding one-to-one with the subwords of .
[0092] For example, the subword sequence = [x1, x2,..., x of each subword in the dialogue text, such as "工", "号", "1234", "还", "款", is input into the third model, and the corresponding labels of these subwords are "B-WORK_ID", "I-WORK_ID", "B-REPAY_MONEY", "I-REPAY_MONEY", O... respectively. When "B-WORK_ID" and "I-WORK_ID" appear continuously, the decoder in the third model combines the corresponding of "B-WORK_ID" and "I-WORK_ID" into the entity keyword "工号"; when "B-REPAY_MONEY" and "I-REPAY_MONEY" appear continuously, the decoder in the third model combines the corresponding of "B-REPAY_MONEY" and "I-REPAY_MONEY" into the entity keyword "还款". Among them, O represents the non-entity keyword in the dialogue text.
[0093] Among them, the conditional probability is obtained by adding the emission probabilities and transition probabilities of multiple entity labels and then performing normalization processing. The calculation formula is as follows: (4) It can be seen from formula (4) that the scores of all possible paths can be summed through the normalization factor, it searches for the maximum path of the scores, obtains the entity labels under the path with the maximum score, and then combines them in the BIO splicing method to obtain the entity keyword.
[0094] S107, input the structured data, the first quality inspection result and the entity keywords into the large language model to obtain the second quality inspection result.
[0095] For example, see Figure 8 As shown, the first quality inspection result of the first model in step S105, the structured data output by the second model, and the entity keywords extracted by the third model in step S106 can be input into the large language model to obtain the second quality inspection result.
[0096] By inputting entity keywords into the large language model, the large language model can understand the basic information and background of the entire dialogue process.
[0097] In addition to the preliminary judgment result of the first quality inspection result and the user emotion review result mentioned in step S105, the cause analysis in the second quality inspection result may also include the behavior review result judged by entity keywords.
[0098] For example, the cause analysis also includes the absence of positive facts in the entity keyword list such as "repaid" or "complaint established" that could offset slightly escalating arguments, and the context still belongs to the scenario of payment reminder and pressure.
[0099] Through the above technical solution, in addition to inputting structured data and the first quality inspection result into the large language model, entity keywords can also be input into the large language model to obtain the second quality inspection result. Since entity keywords include keywords involved in the customer service-customer dialogue, the large language model can better understand the dialogue context, enabling it to more accurately review the first quality inspection result and obtain a more accurate second quality inspection result. Furthermore, because the input to the large language model is not the entire dialogue text, but rather key summary / basic information from the dialogue text filtered by the third model, the processing load of the large language model is significantly reduced, resulting in faster output of the second quality inspection result.
[0100] Figure 9 and Figure 10 This is an exemplary embodiment of the present disclosure, which is an exemplary scheme for interpreting the structured data output by the second model. Please refer to [link / reference]. Figure 10 The second model can be an ASR (Automatic Speech Recognition) engine, which includes a text recognition model and an emotion recognition model, and includes the following steps: S108, input the dialogue voice into the text recognition model to obtain the dialogue text.
[0101] The dialogue text includes the target dialogue text, which includes the first dialogue text and the second dialogue text mentioned above.
[0102] Among them, the text recognition model can be Paraformer-UNASR (a non-autoregressive speech recognition model) or SenseVoice (a Chinese speech recognition model), and the text recognition model can convert speech into dialogue text.
[0103] Optionally, input the dialogue speech into the text recognition model to obtain an initial text; the initial text contains subwords; screen candidate words matching the subwords from the hot word database; the hot word database includes hot words related to the business scenario; obtain the dialogue text according to the dialogue speech, the initial text, and the candidate words.
[0104] Among them, the hot word database includes words frequently appearing in the business scenario between the customer service and the customer, and can also include dialect words in various regions.
[0105] Among them, the candidate words matching the subwords can be candidate words with an edit distance less than a preset edit distance from the subwords, or candidate words with a cosine similarity greater than a preset similarity from the subwords.
[0106] For example, first input the dialogue speech "non-performing loan" into the text recognition model, and decode to obtain the initial text "dai zhang", which is divided into two subwords "dai" and "zhang", and of course can also be divided into four subwords "d", "ai", "zh", "ang", and of course can also be divided into phonemes "dai 1" "zhang 4", where 1 and 4 are the tones of the phonemes; then screen the candidate word "non-performing loan" matching the subwords from the hot word database; finally, input the candidate word, the original dialogue speech, and the initially decoded initial text into the text recognition model, and the text recognition model decides whether to use the candidate word to replace the subwords in the initial text. For example, if the candidate word is "non-performing loan" and the subword is "dai zhang", in the current dialogue scenario, the candidate word "non-performing loan" should be used. By matching the subwords with the candidate hot words, the business scenario of the current dialogue can be combined to improve the accuracy of transcribing the dialogue speech into the dialogue text.
[0107] Optionally, before inputting the dialogue speech into the text recognition model, refer to Figure 10 , and the input dialogue speech can also be input into a noise reduction algorithm for noise reduction processing, so as to eliminate environmental noise interference and improve the accuracy of the subsequent text recognition model in recognizing speech.
[0108] S109, input the dialogue speech into the emotion recognition model to obtain the target user emotion.
[0109] Among them, the target user emotion includes the user emotion when the interlocutor outputs the target dialogue text.
[0110] Among them, the emotion recognition model can be a CNN-BiLSTM-ATt model or a wav2vec model, which can output the user's emotions by recognizing the speech in the dialogue.
[0111] S110, the target dialogue text is associated with the target user's emotion to obtain the structured data.
[0112] It is possible to associate the target dialogue text with the target user's emotions at the same point in time, and obtain the target dialogue text and target user emotions with timestamp alignment.
[0113] Optionally, the corresponding speech of the target dialogue text can be filtered out from the overall dialogue text, and the target user's emotions under the target dialogue text can be identified, thereby associating the target dialogue text with the target user's emotions to obtain structured data.
[0114] Alternatively, one can first obtain the user sentiment of each dialogue text under the overall dialogue text, and then filter out the target dialogue text and the user sentiment corresponding to the target dialogue text from the dialogue text, thereby forming structured data.
[0115] While the large language model can parse text content, such as the first quality inspection result and entity keywords, it has limited ability to capture non-textual information in dialogue speech, such as tone, pauses, and changes in user emotions. Therefore, the emotion recognition model in the second model can be used to identify user emotions and output user emotions in text format to assist the large language model in understanding the emotions and tone of the speaker when uttering the current dialogue text. This allows the large language model to more accurately identify whether there are behaviors in the dialogue text that conform to the quality inspection strategy.
[0116] The following are exemplary embodiments related to step S103 of this disclosure, which are used to explain the use of different quality inspection methods to obtain the first quality inspection result, including the following two exemplary embodiments.
[0117] In the first exemplary embodiment, multiple quality inspection strategies under the business scenario are adopted to perform quality inspection on at least one type of behavioral data associated with each of the multiple quality inspection strategies, and multiple quality inspection results are obtained; from the multiple quality inspection results, the quality inspection result that meets the quality inspection is selected as the first quality inspection result.
[0118] One of these is a quality inspection strategy that inspects at least one type of behavioral data associated with it.
[0119] For example, referring to Table 1, taking a business scenario of violation, including quality inspection strategies 1-3, the violation scenario is associated with models 1-4. Therefore, it will obtain behavioral data of severely escalating conflict-related remarks output by model 1, behavioral data of mildly escalating conflict-related remarks output by model 2, behavioral data of customers making negative personal statements output by model 3, and behavioral data of customer complaints output by model 4. Then, severely escalating conflict-related remarks are taken as the first data point, mildly escalating conflict-related remarks and negative personal statements by customers are taken as the second data point, and mildly escalating conflict-related remarks and customer complaints are taken as the third data point. Then, it will be divided into quality inspection strategies 1-3. Do not perform quality checks on the first to third data points simultaneously. If multiple quality checks are performed at the same time, the final result will be that the first behavioral data point does not meet quality check strategy 1 (the customer service representative did not engage in any severely escalating conflict-related verbal or physical behavior), the second behavioral data point meets quality check strategy 2 (the customer service representative engaged in two instances of mildly escalating conflict-related verbal or physical behavior, but the customer did not make any negative personal statements), and the third behavioral data point does not meet quality check strategy 3 (the customer service representative engaged in two instances of mildly escalating conflict-related verbal or physical behavior, but the customer did not file a police report or complaint). Therefore, the first quality check result output is the result that meets quality check strategy 2: "The customer service representative engaged in two instances of mildly escalating conflict-related verbal or physical behavior, but the customer did not make any negative personal statements."
[0120] In the second exemplary embodiment, multiple quality inspection strategies arranged in descending order of priority are traversed sequentially, and for each traversed quality inspection strategy, the quality inspection strategy is used to perform quality inspection on at least one type of behavior label associated with the quality inspection strategy to obtain a quality inspection result; in response to a quality inspection result that does not meet the quality inspection requirements, the next quality inspection strategy is traversed until the quality inspection result obtained by the traversed quality inspection strategy is compliant; the compliant quality inspection result is taken as the first quality inspection result.
[0121] The more serious the violation indicated in the quality inspection strategy, the higher the priority of the quality inspection strategy.
[0122] For example, referring to Table 1, we can first use quality inspection strategy 1 to inspect the first behavioral data. If the first behavioral data does not meet the quality inspection strategy 1, it means that the customer service representative does not engage in any highly inflammatory or confrontational verbal behavior. Then, we can use quality inspection strategy 2, which has a lower priority, to inspect the second behavioral data. Since the second behavioral data meets the quality inspection strategy 2, it means that the customer service representative engaged in two instances of mildly inflammatory or confrontational verbal behavior, and the customer did not make any negative personal statements. At this point, we can stop the quality inspection and no longer use quality inspection strategy 3 to inspect the third behavioral data.
[0123] Through the first exemplary embodiment, multiple quality inspection strategies can be used to simultaneously inspect the behavioral data associated with each other, thereby obtaining multiple quality inspection results. The quality inspection result that passes among the multiple quality inspection results is output as the first quality inspection result. This can show the different degrees of violations exhibited by customer service representatives and customers during the communication process, which is convenient for subsequent refined management.
[0124] In the second embodiment, a high-priority quality inspection strategy can be used to inspect the associated behavioral data first. If the behavioral data fails the high-priority quality inspection strategy, a low-priority quality inspection strategy can be used to inspect the associated behavioral data. During this process, the first quality inspection result with high priority and high violation level can be output, without having to display the quality inspection result with low priority and low violation level, thereby reducing computing power and saving computing resources.
[0125] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0126] Figure 11 A structural block diagram of the dialogue quality inspection device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0127] Reference Figure 11 The dialogue quality inspection device 1200 includes: The first input module 1210 is configured to input dialogue text into a model cluster to obtain various types of behavioral data in a business scenario; the model cluster includes multiple first models, and different types of behavioral data are output by different first models, and the dialogue text is at least the text of a dialogue between two people; The first acquisition module 1220 is configured to acquire multiple quality inspection strategies under the business scenario; each quality inspection strategy is associated with at least one type of behavioral data under the business scenario. The quality inspection module 1230 is configured to perform quality inspection on at least one type of associated behavioral data using the quality inspection strategy under the business scenario, and obtain a first quality inspection result; the first quality inspection result is used to characterize that the behavior of the target dialoguer among the two dialoguers conforms to the quality inspection strategy under the business scenario. The second acquisition module 1240 is configured to acquire structured data output by the second model; the structured data includes target dialogue text and user sentiment corresponding to the target dialogue text; the target dialogue text includes a first dialogue text and a second dialogue text, the first dialogue text being the text in the dialogue text that conforms to the quality inspection strategy, and the second dialogue text being the context of the first dialogue text. The second input module 1250 is configured to input the structured data and the first quality inspection result into the large language model to obtain the second quality inspection result.
[0128] Optionally, the model cluster includes multiple groups of first models, each group of first models includes multiple first models, and different groups of first models are associated with different business scenarios; the input module 1210 is also configured to input the dialogue text into a group of first models associated with the business scenario to obtain multiple types of behavioral data under the business scenario.
[0129] Optionally, the quality inspection module 1230 is further configured to merge at least one type of behavioral data according to different quality inspection strategies to obtain at least one type of behavioral data associated with different quality inspection strategies; and to perform quality inspection on the associated at least one type of behavioral data using the quality inspection strategies to obtain the first quality inspection result.
[0130] Optionally, the multiple quality inspection strategies have priorities; the quality inspection module 1230 is further configured to sequentially traverse the multiple quality inspection strategies arranged from highest to lowest priority, and for each traversed quality inspection strategy, perform quality inspection on at least one type of behavior tag associated with the quality inspection strategy to obtain a quality inspection result; in response to the quality inspection result being non-compliant, traverse the next quality inspection strategy until the quality inspection result obtained by the traversed quality inspection strategy is compliant; and use the compliant quality inspection result as the first quality inspection result.
[0131] Optionally, the quality inspection module 1230 is further configured to employ multiple quality inspection strategies under the business scenario to perform quality inspection on at least one type of behavioral data associated with each of the multiple quality inspection strategies, thereby obtaining multiple quality inspection results; one quality inspection strategy performs quality inspection on at least one type of behavioral data associated with it; and from the multiple quality inspection results, select the quality inspection result that meets the quality inspection criteria as the first quality inspection result.
[0132] Optionally, the second input module 1250 is further configured to obtain entity keywords output by the third model; the entity keywords are at least one keyword in the dialogue text, and at least one keyword is used together to characterize the summary of the dialogue text; the structured data, the first quality inspection result and the entity keywords are input into the large language model to obtain the second quality inspection result.
[0133] Optionally, the second model includes a text recognition model and an emotion recognition model; the dialogue quality inspection device 1200 also includes: The third input module is configured to input the dialogue speech into the text recognition model to obtain the dialogue text; the dialogue text includes the target dialogue text. The fourth input module is configured to input the dialogue speech into the emotion recognition model to obtain the target user emotion; the target user emotion includes the user emotion output by the interlocutor in the target dialogue text; The association module is configured to associate the target dialogue text with the target user's emotions to obtain the structured data.
[0134] Optionally, the third input module is further configured to input the dialogue speech into the text recognition model to obtain initial text; the initial text includes sub-words; candidate words matching the sub-words are filtered from a hot word database; the hot word database includes hot words related to the business scenario; and the dialogue text is obtained based on the dialogue speech, the initial text, and the candidate words.
[0135] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0137] This application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.
[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0139] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0140] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0142] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for dialogue quality inspection, characterized in that, include: Inputting the dialogue text into the model cluster yields various types of behavioral data in business scenarios. The model cluster includes multiple first models, and different types of behavioral data are output by different first models. The dialogue text is at least the text of a dialogue between two people. Obtain multiple quality inspection strategies under the business scenario; each quality inspection strategy is associated with at least one type of behavioral data under the business scenario. The quality inspection strategy under the business scenario is used to perform quality inspection on at least one type of related behavioral data to obtain a first quality inspection result; The first quality inspection result is used to characterize that the behavior of the target dialogue partner among the two dialogue partners conforms to the quality inspection strategy under the business scenario; Obtain the structured data output by the second model; The structured data includes the target dialogue text and the user sentiment corresponding to the target dialogue text; the target dialogue text includes a first dialogue text and a second dialogue text, wherein the first dialogue text is the text in the dialogue text that conforms to the quality inspection strategy, and the second dialogue text is the context of the first dialogue text. The structured data and the first quality inspection result are input into the large language model to obtain the second quality inspection result.
2. The method of claim 1, wherein, The model cluster includes multiple groups of first models, each group of first models includes multiple first models, and different groups of first models are associated with different business scenarios; the dialogue text is input into the model cluster to obtain various types of behavioral data under the business scenarios, including: The dialogue text is input into a set of first models associated with the business scenario to obtain various types of behavioral data under the business scenario.
3. The method as described in claim 1, characterized in that, The step of performing quality inspection on at least one type of associated behavioral data using the quality inspection strategy under the business scenario to obtain a first quality inspection result includes: The behavioral data of at least one type are merged according to different quality inspection strategies to obtain the behavioral data of at least one type associated with the different quality inspection strategies; The quality inspection strategy is used to perform quality inspection on at least one type of related behavioral data to obtain the first quality inspection result.
4. The method as described in claim 1, characterized in that, Multiple quality inspection strategies have priorities; the step of using the quality inspection strategies under the business scenario to perform quality inspection on at least one type of associated behavioral data to obtain a first quality inspection result includes: The quality inspection strategies are sequentially traversed in descending order of priority. For each quality inspection strategy traversed, the quality inspection strategy is used to perform quality inspection on at least one type of behavior label associated with the quality inspection strategy to obtain the quality inspection result. In response to the quality inspection result being non-compliant, the next quality inspection strategy is traversed until the quality inspection result of the traversed quality inspection strategy is compliant. The quality inspection result that meets the quality inspection criteria shall be taken as the first quality inspection result.
5. The method as described in claim 1, characterized in that, The step of performing quality inspection on at least one type of associated behavioral data using the quality inspection strategy under the business scenario to obtain a first quality inspection result includes: Multiple quality inspection strategies are employed in the business scenario to perform quality inspection on at least one type of behavioral data associated with each of the multiple quality inspection strategies, resulting in multiple quality inspection results; and at least one type of behavioral data associated with one quality inspection strategy. From the multiple quality inspection results, the quality inspection result that meets the quality inspection criteria is selected as the first quality inspection result.
6. The method as described in claim 1, characterized in that, The step of inputting the structured data and the first quality inspection result into a large language model to obtain a second quality inspection result includes: Obtain entity keywords output by the third model; the entity keywords are at least one keyword in the dialogue text, and at least one of the keywords are used together to represent a summary of the dialogue text; The structured data, the first quality inspection result, and the entity keywords are input into the large language model to obtain the second quality inspection result.
7. The method as described in claim 1, characterized in that, The second model includes a text recognition model and a sentiment recognition model; before obtaining the structured data output by the second model, the method further includes: The dialogue voice is input into the text recognition model to obtain the dialogue text; the dialogue text includes the target dialogue text. The dialogue voice is input into the emotion recognition model to obtain the target user's emotion; the target user's emotion includes the user's emotion under the target dialogue text output by the speaker. The structured data is obtained by associating the target dialogue text with the target user's emotions.
8. The method as described in claim 7, characterized in that, The step of inputting the dialogue speech into the text recognition model to obtain the dialogue text includes: The dialogue speech is input into the text recognition model to obtain initial text; the initial text includes subwords; Candidate words matching the sub-words are selected from a hot word database; the hot word database includes hot words related to the business scenario. The dialogue text is obtained based on the spoken dialogue, the initial text, and the candidate words.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.