Outbound service quality inspection method, device, equipment, medium and product

Abstract

The invention discloses an outbound service quality inspection method and device, equipment, a medium and a product, and relates to the technical field of intelligent quality inspection, and the method comprises the steps: collecting an outbound call audio of a robot, and carrying out the character transfer of the outbound call audio, and obtaining a character transfer result; performing vector fusion on the outbound call audio and the character transcription result through a multi-modal fusion model to obtain a multi-modal emotion feature vector, and performing customer emotion analysis based on the multi-modal emotion feature vector to obtain a customer emotion analysis result; and performing outbound result quality inspection on the robot according to the customer emotion analysis result to obtain a structured report. Therefore, the problem that multi-dimensional comprehensive analysis cannot be carried out on the call quality of the outbound robot to cause dissatisfaction of customers is solved, and the outbound service quality inspection efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent quality inspection technology, and in particular to a method, apparatus, equipment, medium and product for outbound call service quality inspection. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence and big data technologies, robot outbound calling services have been widely used in customer service, marketing and other fields. They achieve customer reach and information transmission through automated voice interaction, which significantly improves business processing efficiency. However, the quality inspection and labeling of robot outbound calling results still mainly rely on two methods: manual quality inspection and simple automated processing. These two methods together constitute the core solution for quality control of robot outbound calling results in the current technology.

[0003] Existing technical solutions have significant shortcomings and are difficult to meet the needs of large-scale and complex outbound calling scenarios. First, the speed of manual monitoring and evaluation cannot keep up with the daily outbound call volume of tens of thousands of calls, and subjective judgment is prone to inconsistent standards. In addition, automated processing systems rely on manually predefined rules or isolated keyword matching, which cannot adapt to the implicit outbound call service quality inspection in multi-turn dialogues. Their ability to analyze unstructured voice data is limited, making it difficult to capture the contextual logic and emotional fluctuations in the call. Finally, existing systems lack a dynamic strategy adjustment mechanism. When business scenarios (such as updates to industry compliance requirements) or customer needs change, the rule set needs to be manually redefined, making it impossible to achieve multi-dimensional comprehensive scoring of call quality, resulting in insufficient accuracy and adaptability of quality inspection.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method, device, equipment, medium and product for outbound call service quality inspection, which aims to solve the technical problem of being unable to conduct multi-dimensional comprehensive analysis of the call quality of outbound call robots, leading to customer dissatisfaction.

[0006] To achieve the above objectives, this application proposes an outbound call service quality inspection method, which includes: Collect the outbound call audio of the robot, and transcribe the outbound call audio into text to obtain the transcription result; The outbound call audio and text transcription results are fused using a multimodal fusion model to obtain a multimodal emotion feature vector. Customer emotion analysis is then performed based on the multimodal emotion feature vector to obtain the customer emotion analysis results. The robot's outbound call results are inspected based on the customer sentiment analysis results, and a structured report is obtained.

[0007] In one embodiment, the step of performing vector fusion on the outbound call audio and text transcription results using a multimodal fusion model to obtain a multimodal emotion feature vector, and then performing customer emotion analysis based on the multimodal emotion feature vector to obtain the customer emotion analysis result includes: Extract the customer's call audio from the outbound call audio, and extract the customer's text transcription result from the text transcription result; Calculate the time-domain and frequency-domain features of each frame of speech in the customer's call audio; The time-domain features and frequency-domain features are encoded to obtain a speech emotion vector; The text transcription result from the customer is semantically encoded to obtain a text sentiment vector. Based on the spoken emotion vector and the text emotion vector, vector fusion is performed through the attention mechanism of the multimodal fusion model to obtain a multimodal emotion feature vector; Customer sentiment analysis is performed based on the multimodal sentiment feature vectors to obtain customer sentiment analysis results.

[0008] In one embodiment, the step of performing quality inspection and tagging on the outbound call audio and text transcription results using a multimodal fusion model to obtain a structured report includes: Based on the text transcription results, the robot's business completion results and compliance information are determined through an intent classification model. Based on the text transcription results and outbound call audio, the robot's behavior is analyzed using a robot behavior analysis model to obtain the robot behavior analysis results; Based on the behavioral analysis results, business completion results, compliance information, and customer sentiment analysis results, the robot is subjected to outbound call result quality inspection and labeling through a pre-acquired dynamic rule set to obtain a comprehensive score and problem details; A structured report is generated based on the comprehensive score and issue details.

[0009] In one embodiment, before the step of performing outbound call result quality inspection and tagging on the robot using a pre-acquired dynamic rule set to obtain a comprehensive score, problem details, and improvement suggestions, the method further includes: Based on the robot's historical outbound call records, rules are initialized to obtain an initial quality inspection rule set; Based on historical quality inspection results and user feedback, the initial quality inspection rule set is learned and optimized through a knowledge graph rule engine to obtain an optimized quality inspection rule set. The optimized quality inspection rule set and the initial quality inspection rule set are integrated to obtain the final quality inspection rule set; Based on the behavioral analysis results, business completion results, compliance information, and customer sentiment analysis results, the final quality inspection rule set is weighted and prioritized to obtain a dynamic rule set.

[0010] In one embodiment, after the step of performing outbound call result quality inspection on the robot based on the customer sentiment analysis results to obtain a structured report, the method further includes: Based on the comprehensive score and problem details in the structured report, improvement suggestions are derived. The robot's call service is optimized using the aforementioned improvement suggestions to obtain an optimized robot; The improved models, intention classification models, and robot behavior analysis models are optimized based on the proposed improvements to obtain optimized parameters. Based on the optimization parameters, the multimodal fusion model, intent classification model, and robot behavior analysis model are fine-tuned to obtain the adjustment results; The dynamic rule set is adjusted based on the adjustment results to obtain the adjusted dynamic rule set.

[0011] In one embodiment, after the step of optimizing the robot's call service using the improvement suggestions to obtain an optimized robot, the method further includes: Abnormal call records are determined based on the robot's historical outbound call records; Based on the abnormal call records, a call simulation is performed using the optimized robot to obtain the call simulation results; The simulated call results are sent to a human client for review, and a review result is obtained from the human client. Receive the audit results and determine the keywords of the abnormal calls based on the audit results; Establish external anchor points based on the abnormal call keywords; When the optimized robot is detected to have triggered abnormal call keywords, an external call is initiated through the external anchor point.

[0012] Furthermore, to achieve the above objectives, this application also proposes an outbound call service quality inspection device, which includes: The acquisition module is used to acquire the robot's outbound call audio and transcribe the outbound call audio into text to obtain the transcription result. The analysis module is used to perform vector fusion on the outbound call audio and text transcription results through a multimodal fusion model to obtain a multimodal emotion feature vector, and to perform customer emotion analysis based on the multimodal emotion feature vector to obtain the customer emotion analysis result. The quality inspection module is used to perform outbound call quality inspection on the robot based on the customer sentiment analysis results and obtain a structured report.

[0013] In addition, to achieve the above objectives, this application also proposes an outbound call service quality inspection device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the outbound call service quality inspection method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the outbound call service quality inspection method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the outbound call service quality inspection method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes an outbound call service quality inspection method, apparatus, equipment, medium, and product. It collects outbound call audio from a robot and transcribes the audio into text, obtaining a transcription result. A multimodal fusion model is used to fuse the audio and the transcribed result into vectors, resulting in a multimodal emotion feature vector. Customer emotion analysis is then performed based on this vector, yielding a customer emotion analysis result. Finally, the robot's outbound call results are inspected based on the customer emotion analysis result, resulting in a structured report. Therefore, by fusing feature vectors from the collected outbound call audio and transcribed result using a multimodal fusion model, the problem of the inability to perform multi-dimensional analysis of outbound call robot call quality is solved. Subsequently, customer emotion analysis is performed based on the multimodal emotion feature vector, and finally, the robot's outbound call quality is inspected based on the customer emotion analysis result, resulting in a structured report of the service. This achieves quality inspection of outbound call services, solves the problem of the inability to perform multi-dimensional comprehensive analysis of outbound call robot call quality, which leads to customer dissatisfaction, and improves the efficiency of outbound call service quality inspection. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the outbound call service quality inspection method of this application. Figure 2 This is a schematic diagram illustrating customer sentiment analysis involved in the outbound call service quality inspection method of this application; Figure 3 This is a flowchart illustrating Embodiment 2 of the outbound call service quality inspection method of this application. Figure 4 This is a diagram illustrating the dynamic adjustment of rules related to the outbound call service quality inspection method in this application; Figure 5 This is a schematic diagram illustrating the model optimization involved in the outbound call service quality inspection method of this application; Figure 6 A simplified flowchart illustrating the outbound call service quality inspection method provided in Embodiment 2 of this application; Figure 7 This is a schematic diagram of the module structure of the outbound call service quality inspection device according to an embodiment of this application; Figure 8 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the outbound call service quality inspection method in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is as follows: extract the customer call audio from the outbound call audio and extract the customer text transcription result from the text transcription result; calculate the temporal and frequency domain features of each frame of speech in the customer call audio; encode the temporal and frequency domain features to obtain a speech emotion vector; perform text semantic encoding on the customer text transcription result to obtain a text emotion vector; perform vector fusion through the attention mechanism of the multimodal fusion model based on the speech emotion vector and the text emotion vector to obtain a multimodal emotion feature vector; perform customer emotion analysis based on the multimodal emotion feature vector to obtain a customer emotion analysis result. Based on the text transcription result, determine the robot's business completion result and compliance information through an intent classification model; perform robot behavior analysis through a robot behavior analysis model based on the text transcription result and the outbound call audio to obtain a robot behavior analysis result; perform outbound call result quality inspection and tagging on the robot through a pre-acquired dynamic rule set based on the behavior analysis result, business completion result, compliance information, and customer emotion analysis result to obtain a comprehensive score and problem details; generate a structured report based on the comprehensive score and problem details. Based on the robot's historical outbound call records, rules are initialized to obtain an initial quality inspection rule set. According to historical quality inspection results and user feedback, the initial quality inspection rule set is learned and optimized using a knowledge graph rule engine to obtain an optimized quality inspection rule set. The optimized and initial quality inspection rule sets are integrated to obtain a final quality inspection rule set. Based on the behavior analysis results, business completion results, compliance information, and customer sentiment analysis results, the final quality inspection rule set is weighted and prioritized to obtain a dynamic rule set. Improvement analysis is performed based on the comprehensive score and problem details in the structured report to obtain improvement suggestions. The robot's call service is optimized using these suggestions to obtain an optimized robot. The multimodal fusion model, intent classification model, and robot behavior analysis model are optimized using these suggestions to obtain optimization parameters. Based on these optimization parameters, the multimodal fusion model, intent classification model, and robot behavior analysis model are fine-tuned to obtain adjustment results. The dynamic rule set is then adjusted using these results to obtain an adjusted dynamic rule set. Abnormal call records are identified based on the robot's historical outbound call records; based on the abnormal call records, a call simulation is performed using the optimized robot to obtain a call simulation result; the call simulation result is sent to a human client for review, and a review result is obtained; the review result is received, and abnormal call keywords are identified based on the review result; an external anchor point is established based on the abnormal call keywords; when the optimized robot triggers an abnormal call keyword, an external call is initiated through the external anchor point.This solves the problem of customer dissatisfaction caused by the inability to conduct multi-dimensional comprehensive analysis of call quality from outbound call robots, enabling quality inspection of outbound call services and improving the efficiency of outbound call service quality inspection. Based on the solution of this invention, addressing the problems of low efficiency and high cost of traditional manual quality inspection methods, which are difficult to meet the needs of large-scale outbound call scenarios, this invention designs an outbound call service quality inspection method. The effectiveness of the outbound call service quality inspection method of this invention is verified during the quality inspection of outbound call services. Finally, the efficiency of outbound call service quality inspection using the method of this invention is significantly improved.

[0024] In this embodiment, for ease of description, the outbound call service quality inspection device will be used as the execution subject in the following description.

[0025] Due to the limitations of traditional manual quality inspection and existing automated systems, the quality inspection and tagging effects of robot outbound calls need further improvement. One issue is the reliance on manually defined rules. Traditional quality inspection relies on manually preset rules or simple keyword matching, lacking the ability to adapt to complex dialogue scenarios (such as changes in intent and contextual relationships in multi-turn dialogues). This leads to a decrease in the accuracy of quality inspection in complex scenarios. Another issue is the insufficient ability to analyze unstructured data. Existing automated systems have limited ability to analyze unstructured information such as voice emotion and tone, and cannot accurately identify customer emotional fluctuations or hidden violations. If only structured text analysis is relied upon, the quality inspection dimension is singular, and the missed detection rate will increase significantly. Furthermore, there is the lack of dynamic strategy adjustment and multi-dimensional scoring. Updating the quality inspection rule base requires manual intervention, and the scoring system often focuses on a single indicator, making it difficult to adapt to the dynamic changes in business scenarios. It also cannot conduct multi-dimensional evaluations from the perspectives of compliance, user experience, and business objectives, which also affects the efficiency and comprehensiveness of quality inspection. Therefore, in the current large-scale robot outbound call scenario, the quality inspection system still has significant shortcomings in terms of intelligence, dynamism, and comprehensive evaluation capabilities.

[0026] This application provides a solution that, based on the collected outbound call audio and text transcription results, fuses feature vectors through a multimodal fusion model to address the problem of the inability to perform multi-dimensional analysis of the call quality of outbound call robots. Subsequently, customer emotion analysis results are obtained based on multimodal emotion feature vector analysis. Finally, outbound call quality inspection of the robot is performed based on the customer emotion analysis results to obtain a structured service report. This realizes the quality inspection of outbound call services, solves the problem of the inability to perform multi-dimensional comprehensive analysis of the call quality of outbound call robots, which leads to customer dissatisfaction, and improves the efficiency of outbound call service quality inspection.

[0027] Based on this, the embodiments of this application provide a method for quality inspection of outbound call services, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the outbound call service quality inspection method of this application.

[0028] In this embodiment, the outbound call service quality inspection method includes steps S01 to S03: Step S01: Collect the robot's outbound call audio and transcribe the outbound call audio into text to obtain the transcription result; Before the implementation of this embodiment, it should be clear that in recent years, with the rapid development of artificial intelligence and big data technologies, robot outbound calling services have been widely used in customer service, marketing and other fields. They achieve customer reach and information transmission through automated voice interaction, which significantly improves business processing efficiency. However, the quality inspection and labeling of robot outbound calling results still mainly rely on two methods: manual quality inspection and simple automated processing. The two together constitute the core solution for quality control of robot outbound calling results in the prior art.

[0029] Existing technical solutions have significant shortcomings and are difficult to meet the needs of large-scale and complex outbound calling scenarios. First, the speed of manual monitoring and evaluation cannot keep up with the daily outbound call volume of tens of thousands of calls, and subjective judgment is prone to inconsistent standards. In addition, automated processing systems rely on manually predefined rules or isolated keyword matching, which cannot adapt to the implicit outbound call service quality inspection in multi-turn dialogues. Their ability to analyze unstructured voice data is limited, making it difficult to capture the contextual logic and emotional fluctuations in the call. Finally, existing systems lack a dynamic strategy adjustment mechanism. When business scenarios (such as updates to industry compliance requirements) or customer needs change, the rule set needs to be manually redefined, making it impossible to achieve multi-dimensional comprehensive scoring of call quality, resulting in insufficient accuracy and adaptability of quality inspection.

[0030] Therefore, in this embodiment, the robot's outbound call audio is first collected. In this embodiment, the outbound call audio is the audio of the robot making calls when performing sales tasks, follow-up tasks, or response tasks. Then, speech recognition is performed on the received audio to obtain the corresponding text transcription results.

[0031] Step S02: The outbound call audio and text transcription results are fused using a multimodal fusion model to obtain a multimodal emotion feature vector. Customer emotion analysis is then performed based on the multimodal emotion feature vector to obtain the customer emotion analysis results. After completing the acquisition of outbound call audio and the acquisition of text transcription results, a multimodal fusion model can be used to perform vector fusion on the outbound call audio and text transcription results to obtain multimodal emotion feature vectors. In this embodiment, the multimodal emotion feature vectors are fused using the attention mechanism in the multimodal fusion model to obtain more refined multimodal emotion features. Subsequently, customer emotion analysis is performed using the modal emotion feature vectors to obtain customer emotion analysis results.

[0032] Step S03: Based on the customer sentiment analysis results, perform quality inspection on the outbound call results of the robot to obtain a structured report.

[0033] After obtaining the customer sentiment analysis results, we can know whether the customer is satisfied with the service. In addition, we also need to analyze the professionalism of the robot. Therefore, in this embodiment, we perform outbound call quality inspection on the robot based on the customer sentiment analysis results to obtain a structured report of the call.

[0034] This enables quality inspection of outbound calling services, solves the problem of customer dissatisfaction caused by the inability to conduct multi-dimensional comprehensive analysis of the call quality of outbound calling robots, and improves the efficiency of outbound calling service quality inspection.

[0035] Specifically, step S02 above, which involves performing vector fusion on the outbound call audio and text transcription results using a multimodal fusion model to obtain a multimodal emotion feature vector, and then performing customer emotion analysis based on the multimodal emotion feature vector to obtain the customer emotion analysis results, includes the following steps: Step S021: Extract the customer call audio from the outbound call audio and extract the customer text transcription result from the text transcription result; Step S022: Calculate the time-domain and frequency-domain features of each frame of speech in the customer's call audio. Step S023: Encode the time-domain features and frequency-domain features to obtain a speech emotion vector; Step S024: Perform text semantic encoding on the customer's text transcription result to obtain a text sentiment vector; Step S025: Based on the speech emotion vector and the text emotion vector, perform vector fusion through the attention mechanism of the multimodal fusion model to obtain a multimodal emotion feature vector; Step S026: Perform customer sentiment analysis based on the multimodal sentiment feature vector to obtain customer sentiment analysis results.

[0036] like Figure 2 As shown, the solution in this embodiment extracts customer call audio. From the complete outbound call recording, audio separation technology (such as sound source localization, speaker separation model VAD (Voice Activity Detection)) is used to extract the customer's voice segment separately, excluding the agent's voice part. Then, automatic speech recognition (ASR) is performed on the entire outbound call audio to convert it into a complete text transcription result. Then, based on the timestamp or speaker tag of the customer call audio extracted in the previous step, the corresponding text transcription result of the customer's words is extracted from the complete text transcription result.

[0037] Subsequently, a detailed signal analysis was performed on the extracted customer call audio to extract acoustic features that reflect emotional changes. This mainly involved segmenting the continuous customer voice signal into short frames (typically 20-30 milliseconds per frame, with about 50% overlap), and applying a window function (such as a Hamming window) to each frame to reduce spectral leakage. This allowed for the corresponding feature extraction. Temporal feature extraction includes: Energy: Reflects the strength of the vocal signal and is related to the degree of emotional excitement (e.g., higher energy when angry, lower energy when depressed).

[0038] Zero Crossing Rate (ZCR): refers to the number of times a speech signal waveform crosses the zero level per unit of time. It can be used to distinguish between unvoiced and voiced sounds, and is also related to the level of emotional tension.

[0039] Short-term average amplitude difference, etc.

[0040] Frequency domain feature extraction includes: Fundamental Frequency (F0): refers to the basic frequency of vocal cord vibration, that is, the pitch of speech. The mean, range and rate of change of fundamental frequency vary significantly under different emotions (e.g., the fundamental frequency is usually higher and fluctuates more when angry).

[0041] Spectral features: The time-domain signal is converted to the frequency domain by Fourier transform (FFT) to extract spectral energy distribution, spectral centroid, spectral bandwidth, Mel-frequency cepstral coefficients (MFCC), etc. Among them, MFCC is one of the most widely used features in speech recognition and emotion recognition, as it simulates the auditory characteristics of the human ear.

[0042] Formants: The resonant frequencies of the vocal tract, whose frequency and bandwidth also carry emotional information.

[0043] Next, the high-dimensional, potentially redundant acoustic features extracted in the previous step are mapped to low-dimensional, emotion-discriminating vector representations through model learning. This requires preprocessing the extracted time-domain and frequency-domain features, such as normalization and standardization, to ensure feature stability and comparability. Subsequently, large-model-based speech sentiment analysis techniques (such as...) are used... Figure 2As shown, the customer tone and emotion recognition module calls a large model, inputting the preprocessed speech feature sequence into the large model (which can be a pre-trained model specifically designed for speech emotion, or a model finely tuned based on a general speech model). The large model uses its deep neural network structure (such as CNN, RNN, Transformer, etc.) to perform deep encoding and nonlinear transformation on these features, and finally generates a speech emotion vector that can represent the customer's speech emotional state. This vector contains the emotional information extracted from the speech.

[0044] Beyond feature extraction from call audio, it is also necessary to extract semantic sentiment information from the customer's text content. This includes cleaning the customer's text transcription results (such as removing noise and correcting recognition errors), word segmentation, stop word removal, and part-of-speech tagging. The preprocessed text sequence is then input into a pre-trained text semantic model (such as BERT, RoBERTa, XLNet, etc., which can also be regarded as a "large model" for processing text). These models can capture the contextual semantic information and sentiment in the text and obtain the text sentiment vector through their output layer (such as the hidden state of the [CLS] token or sentence-level pooling output). This vector contains the sentiment information extracted from the text semantics.

[0045] Since speech and text are two important modalities for expressing emotions, each with its own emphasis, the multimodal fusion in this embodiment aims to effectively combine the emotional information of these two modalities to obtain a more comprehensive emotional representation.

[0046] In this embodiment, the multimodal fusion model uses an attention mechanism to fuse speech emotion vector sequences and text emotion vector sequences as input. Then, through a self-attention mechanism, the model can learn the importance of features at different moments within the speech and the importance of different word / segment features within the text. In addition, through a cross-attention mechanism, the model can learn the correlation and mutual influence between speech features and text features. For example, a negative word in the text may enhance the emotion weight of the corresponding segment in the speech, and vice versa.

[0047] Based on the learned attention weights, the speech emotion vector and the text emotion vector are weighted and summed or combined in a more complex nonlinear way to obtain a multimodal emotion feature vector that integrates speech and text emotion information. This vector is considered to reflect the customer's true emotional state more accurately than a single-modal vector.

[0048] Finally, the fused multimodal emotion feature vectors are used to determine the final emotion state. The multimodal emotion feature vectors are input into a pre-trained emotion classification model. This classification model can be a neural network trained on a large amount of labeled emotion data (such as fully connected layers and classifier heads), or it can be a classification layer contained in the large model itself.

[0049] The sentiment classification model processes the input multimodal sentiment feature vectors, outputs the probability distribution of various sentiment categories through activation functions such as softmax, and finally determines and outputs the customer's sentiment label (such as satisfaction, dissatisfaction, anger, neutrality, joy, sadness, etc.). The obtained sentiment label is used as the customer sentiment analysis result, which can be further used for subsequent business analysis, customer service optimization, etc.

[0050] This completes the acquisition of customer sentiment analysis results, providing a more accurate data foundation for subsequent generation of structured reports.

[0051] More specifically, step S03 above, which involves performing quality inspection and tagging on the outbound call audio and text transcription results using a multimodal fusion model to obtain a structured report, includes: Step S031: Based on the text transcription result, the robot's business completion result and compliance information are determined by the intent classification model; Step S032: Based on the text transcription results and outbound call audio, the robot's behavior is analyzed using a robot behavior analysis model to obtain the robot behavior analysis results; Step S033: Based on the behavior analysis results, business completion results, compliance information, and customer sentiment analysis results, the robot is subjected to outbound call result quality inspection and labeling through a pre-acquired dynamic rule set to obtain a comprehensive score and problem details; Step S034: Generate a structured report based on the comprehensive score and problem details.

[0052] First, the ASR output text is denoised (e.g., correcting recognition errors and removing duplicate segments), labeled with roles (distinguishing between "robot" and "customer" dialogue content), and timestamp aligned (matching text with audio segments). Then, robot voice segments are extracted from the complete call audio (excluding customer voice) for subsequent behavior analysis (e.g., speech rate and pause detection).

[0053] After completing the corresponding preprocessing, the customer sentiment analysis results (such as sentiment tags, sentiment intensity, and sentiment change timestamps) are associated with the dialogue text and audio segments to form a multimodal analysis dataset.

[0054] Then, the text transcription results are processed through an intent classification model to output business completion results and compliance information. The core is to determine whether the robot has achieved its business objectives and whether it conforms to the standard script. For example, if the customer clearly states "I want to buy this product" in the dialogue text and the robot completes the order confirmation process, the business completion result is marked as "success". If the customer refuses multiple times and the robot does not provide further guidance, it is marked as "failure".

[0055] Based on a pre-set library of compliant dialogue scripts (such as "customer service terms must be informed" and "promises of uncertain benefits are prohibited"), keyword matching (such as regular expressions) is used to detect whether the robot uses the prescribed dialogue scripts and whether it omits necessary statements (such as "this call may be recorded"). For ambiguous scenarios (such as dialogue script paraphrase variations), a text similarity model (such as Siamese BERT) is used to compare the similarity between the robot's actual dialogue scripts and the standard dialogue scripts. If the similarity is lower than the threshold, it is marked as "suspected non-compliant".

[0056] Subsequently, using a robot behavior analysis model, the robot's interaction norms, process integrity, and voice quality are evaluated from both text and audio dimensions, outputting behavior analysis results. This behavior analysis includes both text-based and audio-based behavior analysis. 1. First, text-based behavioral analysis: compare the actual dialogue flow with the standard business flow (such as "greeting → introducing the product → answering questions → facilitating conversion"), and use sequence labeling models (such as BiLSTM-CRF) to identify whether process nodes are missing or in the wrong order (such as directly selling without greeting). In addition, it also includes interaction quality assessment: detect whether the robot interrupts the customer (based on the dialogue round timestamp), whether it responds to the customer's questions correctly (through intent matching, such as whether the robot answers the price when the customer asks "price"), and whether there are invalid repetitive phrases.

[0057] 2. Based on audio-based behavior analysis, extract speech features from robot audio clips, including speech rate (syllables / second), pause duration (pauses between sentences >2 seconds are marked as abnormal), volume stability (whether the variance exceeds the threshold), and tone (identifying whether it is "mechanical" or "impatient" through a speech emotion model). Also detect whether a prescribed speech template (such as the tone of a standard greeting) is used. Compare the differences between the actual speech and the standard speech using a speech similarity model (such as MFCC features + cosine similarity).

[0058] By combining behavioral analysis results, business completion results, compliance information, and customer sentiment analysis results, a comprehensive scoring and problem labeling are performed through a dynamic rule set. The core is to map objective data into quality inspection indicators.

[0059] For example, this embodiment uses rules 1, 2, and 3 as examples: Rule 1: If the business completion result = "failure" AND the customer emotion label = "anger" → deduct 20 points and mark it as "serious business failure + customer dissatisfaction"; Rule 2: If compliance information includes "omission of script" AND behavioral analysis = "missing process" → deduct 15 points and mark as "double violation of process compliance"; Rule 3: If the customer's emotional intensity is ≥0.8 (extremely negative) → trigger an emergency warning and mark it as "requires manual review".

[0060] In addition, the dynamic rule set in this embodiment can also set weights (such as business completion 40%, compliance 25%, behavior 20%, customer sentiment 15%), deduct / add points according to the rule matching results, with a total score of 100 points, and divide into levels (such as ≥90 points "excellent", 60-89 points "qualified", <60 points "unqualified").

[0061] Then, the rules engine iterates through all rules, accumulates scores based on matching results, generates a comprehensive score, and summarizes all violation tags (such as "omission of dialogue", "interrupting customers", "business failure"), associates them with specific evidence (text fragments, audio timestamps, emotional fluctuation points), and sorts them by severity (such as "serious", "moderate", "minor").

[0062] Finally, based on the comprehensive score and problem details, a standardized quality inspection report is automatically generated, which includes the following modules:

[0063] In this embodiment, a template engine (such as Jinja2) is used to populate the report template with structured data, which supports exporting to PDF / HTML format or connecting to a CRM system for real-time display.

[0064] Therefore, through multimodal data fusion and model-rule collaboration, a comprehensive, interpretable, and traceable quality inspection and evaluation of robot outbound call quality can be achieved.

[0065] This embodiment, through the above-described scheme, specifically collects the outbound call audio of the robot and performs text transcription on the audio to obtain the transcription result. A multimodal fusion model is then used to perform vector fusion on the outbound call audio and the transcription result to obtain a multimodal emotion feature vector. Customer emotion analysis is then performed based on this multimodal emotion feature vector to obtain the customer emotion analysis result. Finally, the robot's outbound call results are quality inspected based on the customer emotion analysis result, resulting in a structured report. Thus, by using a multimodal fusion model to fuse feature vectors based on the collected outbound call audio and transcription result, the problem of the inability to perform multi-dimensional analysis of the call quality of outbound call robots is solved. Subsequently, customer emotion analysis results are obtained based on the multimodal emotion feature vector, and finally, the robot's outbound call quality is inspected based on the customer emotion analysis result, resulting in a structured service report. This achieves quality inspection of outbound call services, solves the problem of the inability to perform multi-dimensional comprehensive analysis of the call quality of outbound call robots, which leads to customer dissatisfaction, and improves the efficiency of outbound call service quality inspection.

[0066] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Before step S033, which involves performing outbound call result quality inspection and tagging on the robot using a pre-acquired dynamic rule set to obtain a comprehensive score, problem details, and improvement suggestions, the outbound call service quality inspection method further includes steps S0331 to S0334: Step S0331: Initialize the rules based on the robot's historical outbound call records to obtain an initial quality inspection rule set; Step S0332: Based on historical quality inspection results and user feedback, the initial quality inspection rule set is learned and optimized through a knowledge graph rule engine to obtain an optimized quality inspection rule set. Step S0333: Integrate the optimized quality inspection rule set and the initial quality inspection rule set to obtain the final quality inspection rule set; Step S0334: Based on the behavior analysis results, business completion results, compliance information, and customer sentiment analysis results, adjust the weights and priorities of the final quality inspection rule set to obtain a dynamic rule set.

[0067] First, such as Figure 4As shown, all data from the robot's past outbound calls are collected, including call audio, text transcription results, manual quality inspection labels (such as "compliant," "violation," and "business success"), customer feedback records (such as complaints and satisfaction ratings), and business goal achievement data (such as conversion rate and problem resolution rate). Then, through expert experience analysis (such as business specifications and compliance requirements) and data-driven analysis (such as association rule mining and cluster analysis), initial rules are extracted from the historical data. Example rules in this embodiment include: Business rule: If a customer explicitly refuses 3 times, the robot must terminate the sales process; Compliance rule: Customers must be informed within 10 seconds of the start of the call that the call may be recorded; Emotion association rule: If a customer's emotion label is "angry" and the duration is >30 seconds, it is marked as a "high-risk call".

[0068] The extracted rules are converted into a structured format and stored in the rule base to form an initial quality inspection rule set, which serves as the basis for subsequent optimization.

[0069] Subsequently, we collected the execution results of the initial rule set in actual quality inspection (such as the number of rule hits, false positive rate, etc.), manual review and correction records (such as the case of "rule R002 missing 'not informed of service terms'", etc.), and rule adjustment suggestions from users (such as quality inspectors, business parties, etc.).

[0070] Subsequently, entities (such as "customer emotions", "business nodes", and "compliance terms") and relationships (such as "emotions affecting business results" and "compliance terms related to sales scripts") in the outbound call scenario are constructed into a knowledge graph as the semantic basis for rule reasoning. Contradictory rules in the initial rule set are identified (e.g., "Rule A requires 'continue sales after customer rejection', Rule B requires 'termination of process'"). These are then merged into unified rules through relational reasoning from the knowledge graph (e.g., "customer rejection → high emotional risk → process termination required"). For frequently misjudged rules, conditions are refined through historical data feature analysis (e.g., the original rule "customer emotions" does not...). The "satisfied → risk marker" is further refined to "customer emotion 'satisfied' and speech rate > 180 words / minute → high risk". In addition to some covered scenarios, it is also necessary to consider the uncovered scenarios. In this embodiment, for new uncovered scenarios (such as "customer expressing refusal in dialect"), the scope of application of the rule is generalized through entity association of knowledge graph ("dialect words → semantically equivalent 'refusal'"). The priority of the rule is adjusted according to the accuracy rate of the rule in historical quality inspection (such as "rule R003 accuracy rate 90% → weight increased to 8") and business importance (such as compliance rule weight being higher than business rule weight by default).

[0071] Subsequently, after learning by the knowledge graph rule engine, an updated and optimized quality inspection rule set is obtained, which includes newly added rules, corrected rules, and rules with adjusted weights.

[0072] After obtaining the initial rule set and the optimized rule set, compare the initial rule set and the optimized rule set, delete duplicate rules, retain the optimized new version, and divide the rules into levels according to rule type, such as core rules (such as compliance red lines, which cannot be adjusted), business rules (such as process specifications, which can be dynamically optimized), and auxiliary rules (such as customer emotion association, which have lower weight). The integrated rules are uniformly stored in a dynamic rule engine (such as Drools or Aviator), which supports adding, deleting, modifying, querying and calling rules in real time, forming the final quality inspection rule set.

[0073] Finally, by combining real-time robot interaction data (behavioral analysis, business results, compliance information, customer sentiment), the weight and priority of the rules are dynamically adjusted to ensure that the rules are suitable for the current scenario. Specifically, this includes: Input real-time analysis results, including behavioral analysis results (such as "the robot interrupted the customer twice" and "speaking too fast"), business completion results (such as "business success / failure" and "conversion rate of 15%), compliance information (such as "compliance / violation tags" and "number of violations"), and customer sentiment analysis results (such as "anger", "satisfaction", and "emotional fluctuation timestamps").

[0074] Subsequently, based on the weight and priority adjustment of real-time data, a rule weight calculation formula is set, and real-time adjustments are made in conjunction with real-time data characteristics. After real-time adjustments, a dynamic rule set for the current call quality inspection is obtained, ensuring that the rules can be flexibly adapted to the real-time status of customer interactions.

[0075] Through the above process, dynamic rule sets can achieve full lifecycle management from "learning from historical data → adapting to real-time scenarios → continuous feedback and optimization", ultimately providing accurate, flexible and evolvable rule support for robot outbound call quality inspection, improving quality inspection efficiency and business goal achievement rate.

[0076] Specifically, in the above embodiments, after step S03, which involves performing outbound call result quality inspection on the robot based on the customer sentiment analysis results to obtain a structured report, the method further includes: Step S04: Based on the comprehensive score and problem details in the structured report, conduct an improvement analysis to obtain improvement suggestions; Step S05: Optimize the robot's call service using the improvement suggestions to obtain an optimized robot; Step S06: Optimize the multimodal fusion model, intent classification model, and robot behavior analysis model using the improvement suggestions to obtain optimized parameters; Step S07: Based on the optimization parameters, fine-tune the multimodal fusion model, the intent classification model, and the robot behavior analysis model to obtain the adjustment results; Step S08: Adjust the rules of the dynamic rule set based on the adjustment results to obtain the adjusted dynamic rule set.

[0077] like Figure 5 As shown, key information is extracted from the report in the quality inspection module, including comprehensive scoring (e.g., 65 points "unqualified"), problem details (e.g., "compliance script omitted 3 times", "customer anger not identified", "business process interruption"), problem severity ranking (e.g., "serious: complaint risk", "moderate: speaking too fast"), and customer emotional fluctuation points (e.g., "sudden change in emotion when mentioning fees"). Then, the problem details are divided by module (e.g., "robot script problem", "model recognition problem", "rule adaptation problem") to obtain corresponding improvement suggestions.

[0078] Subsequently, based on the improvement suggestions, the robot script library was updated, business process nodes were optimized, and robot voice synthesis parameters were adjusted. After completion, the optimized robot version can be output for subsequent outbound call services.

[0079] In addition to adjusting the robot, model optimization is also needed. For example, the multimodal fusion model has the problem of "incorrect emotion recognition when the customer's voice is angry but the text is neutral," which is caused by "insufficient weights in the voice emotion vector." The intent classification model has the problem of "the customer's 'no longer needed' being identified as 'hesitant,'" which is caused by "insufficient capture of negative word semantic features." The robot behavior analysis model has the problem of "failure to detect the robot interrupting the customer," which is caused by "the voice activity detection (VAD) threshold being set too loosely." Based on the above problem identification, it was determined that for the multimodal fusion model, the weight coefficient of the voice vector in the attention mechanism should be adjusted (increased from 0.3 to 0.5), and the training weight of "sarcasm" samples should be increased. For the intent classification model, the proportion of "negative intent" samples in the data should be fine-tuned (increased from 10% to 20%), and the learning rate of the BERT model should be adjusted (decreased from 2e-5 to 1e-5). For the behavior analysis model, the VAD detection threshold should be tightened (the "voice activity judgment interval" should be shortened from 300ms to 200ms).

[0080] Finally, based on the model optimization results, the rule weights and triggering conditions are updated to make the interaction between the model and the rules more interactive.

[0081] Through the above process, the system achieves closed-loop management from "problem discovery → root cause analysis → solution optimization → effect verification", ensuring that the robot's service capabilities and model analysis accuracy continue to evolve with business scenarios.

[0082] More specifically, after step S05 above, where the robot's call service is optimized using the improvement suggestions to obtain the optimized robot, the method further includes: Step S0501: Determine abnormal call records based on the robot's historical outbound call records; Step S0502: Based on the abnormal call records, perform a call simulation using the optimized robot to obtain the call simulation results; Step S0503: The call simulation result is sent to the human client for review, and the review result is obtained. Step S0504: Receive the audit result and determine the abnormal call keywords based on the audit result; Step S0505: Establish an external anchor point based on the abnormal call keywords; Step S0506: When the optimized robot is detected to have triggered abnormal call keywords, an external call is initiated through the external anchor point.

[0083] First, based on the robot's historical outbound call records, abnormal call records (such as "customer hangs up midway" or "intense emotional conversation") are automatically filtered out. Then, the optimized robot is used to simulate and reproduce the customer's voice / text input in the abnormal call records (such as inputting customer questions and emotional voice fragments from historical calls into the robot) to generate simulated call results (including robot response content, emotion recognition results, and business process execution status).

[0084] Then, the simulated call results (text / audio) are sent to the human client for review by quality inspectors: to determine whether the robot correctly handled abnormal scenarios (such as whether the soothing dialogue was triggered when the customer was angry), to mark unresolved issues (such as whether the robot's response deviated from the dialogue), and to output the review results (including pass / fail and issue markings).

[0085] Next, based on the results of manual review, the core keywords / phrases that caused the abnormality are extracted from the abnormal call records (such as customer complaints of "high cost" or "poor service", or robot responses of "unclear" or "cannot be resolved"). The abnormal call keywords are then bound to external triggering rules to establish "external anchor points" (such as "when a customer says 'high cost' → trigger the manual transfer process"). The anchor point includes a list of keywords, triggering conditions (such as the number of times the keyword appears ≥ 1) and external actions (such as transferring to a human agent or activating the advanced script library).

[0086] Finally, in actual outbound calls, the optimized robot uses voice recognition to detect customer conversations in real time. If it hits abnormal keywords in the external anchor point (such as the customer mentioning "high cost"), it will automatically trigger external actions (such as pausing the robot's response and transferring the call to a human agent).

[0087] This embodiment, through the above-described scheme, specifically initializes rules based on the robot's historical outbound call records to obtain an initial quality inspection rule set; based on historical quality inspection results and user feedback, a knowledge graph rule engine is used to learn and optimize the initial quality inspection rule set to obtain an optimized quality inspection rule set; the optimized quality inspection rule set and the initial quality inspection rule set are integrated to obtain a final quality inspection rule set; and the final quality inspection rule set is adjusted in weight and priority based on the behavior analysis results, business completion results, compliance information, and customer sentiment analysis results to obtain a dynamic rule set. Thus, based on the collected outbound call audio and text transcription results, a multimodal fusion model is used to fuse feature vectors, solving the problem of the inability to perform multi-dimensional analysis of the call quality of outbound call robots. Subsequently, customer sentiment analysis results are obtained based on multimodal sentiment feature vector analysis, and finally, outbound call quality inspection of the robot is performed based on the customer sentiment analysis results, resulting in a structured service report. This achieves quality inspection of outbound call services, solves the problem of the inability to perform multi-dimensional comprehensive analysis of the call quality of outbound call robots, leading to customer dissatisfaction, and improves the efficiency of outbound call service quality inspection.

[0088] For example, to help understand the implementation process of the outbound call service quality inspection method obtained in this embodiment combined with the above embodiment one, please refer to... Figure 6 , Figure 6 A simplified flowchart of an outbound call service quality inspection method is provided, specifically: This embodiment includes (1) a speech recognition and transcription module, which is mainly used to perform speech recognition on robot outbound calls and generate text transcription results; (2) a customer tone and emotion recognition module: based on big model technology, it analyzes the tone and emotion of customers and identifies the emotional state of customers (such as satisfaction, dissatisfaction, anger, etc.); (3) an intent recognition and compliance analysis module: through the intent classification model, it judges whether the robot has fully covered the preset business objectives (such as user information collection, product recommendation) and detects whether it violates industry norms (such as advertising law, privacy protection clauses); (4) a robot behavior analysis module: analyzes the robot's behavior in the call, including semantics, tone, answer accuracy, etc.; (5) a dynamic rule set management module: based on historical quality inspection results and real-time analysis data, it dynamically adjusts the quality inspection rules to form a dynamic rule set; (6) an intelligent quality inspection and tagging module: combining customer tone and emotion, robot behavior and dynamic rule set, it performs intelligent quality inspection and tagging on the robot outbound call results. Output a structured report, including a comprehensive score, problem details and improvement suggestions, and (7) self-improvement and optimization module: by continuously learning the quality inspection results and user feedback, the manually corrected quality inspection results are used as new training data to optimize the quality inspection capabilities of the large model and improve the accuracy and adaptability of the system.

[0089] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the outbound call service quality inspection method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0090] This application also provides an outbound call service quality inspection device, please refer to... Figure 7 The outbound call service quality inspection device includes: The acquisition module 10 is used to acquire the outbound call audio of the robot and transcribe the outbound call audio into text to obtain the transcription result. The analysis module 20 is used to perform vector fusion on the outbound call audio and text transcription results through a multimodal fusion model to obtain a multimodal emotion feature vector, and to perform customer emotion analysis based on the multimodal emotion feature vector to obtain the customer emotion analysis result. The quality inspection module 30 is used to perform outbound call result quality inspection on the robot based on the customer sentiment analysis results and obtain a structured report.

[0091] The outbound call service quality inspection device provided in this application, employing the outbound call service quality inspection method described in the above embodiments, can solve the technical problem of customer dissatisfaction caused by the inability to perform multi-dimensional comprehensive analysis of the call quality of outbound call robots. Compared with the prior art, the beneficial effects of the outbound call service quality inspection device provided in this application are the same as those of the outbound call service quality inspection method described in the above embodiments, and other technical features in the outbound call service quality inspection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0092] This application provides an outbound call service quality inspection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the outbound call service quality inspection method in the above embodiment 1.

[0093] The following is for reference. Figure 8 The diagram illustrates a structural schematic suitable for implementing outbound call service quality inspection equipment in the embodiments of this application. The outbound call service quality inspection equipment in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8The outbound call service quality inspection equipment shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0094] like Figure 8 As shown, the outbound call service quality inspection equipment may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the outbound call service quality inspection equipment. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the outbound call service quality inspection equipment to communicate wirelessly or wiredly with other equipment to exchange data. Although the figure shows outbound call service quality inspection equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0095] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0096] The outbound call service quality inspection equipment provided in this application, employing the outbound call service quality inspection method described in the above embodiments, can solve the technical problem of customer dissatisfaction caused by the inability to perform multi-dimensional comprehensive analysis of the call quality of outbound call robots. Compared with the prior art, the beneficial effects of the outbound call service quality inspection equipment provided in this application are the same as those of the outbound call service quality inspection method described in the above embodiments, and other technical features of this outbound call service quality inspection equipment are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0097] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0098] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0099] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the outbound call service quality inspection method described in the above embodiments.

[0100] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0101] The aforementioned computer-readable storage medium may be included in the outbound call service quality inspection equipment; or it may exist independently and not be assembled into the outbound call service quality inspection equipment.

[0102] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the outbound call service quality inspection device, the outbound call service quality inspection device performs the following actions: collects the robot's outbound call audio, performs text transcription on the outbound call audio, and obtains the text transcription result; performs vector fusion on the outbound call audio and the text transcription result through a multimodal fusion model to obtain a multimodal emotion feature vector, performs customer emotion analysis based on the multimodal emotion feature vector, and obtains a customer emotion analysis result; and performs outbound call result quality inspection on the robot based on the customer emotion analysis result, and obtains a structured report.

[0103] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0105] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0106] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the aforementioned outbound call service quality inspection method. This solves the technical problem of the inability to perform multi-dimensional comprehensive analysis of the call quality of outbound call robots, leading to customer dissatisfaction. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the outbound call service quality inspection method provided in the above embodiments, and will not be elaborated upon here.

[0107] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the outbound call service quality inspection method described above.

[0108] The computer program product provided in this application can solve the technical problem of customer dissatisfaction caused by the inability to perform multi-dimensional comprehensive analysis of the call quality of outbound call robots. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the outbound call service quality inspection method provided in the above embodiments, and will not be repeated here.

[0109] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for quality inspection of outbound call services, characterized in that, The outbound call service quality inspection method includes: Collect the outbound call audio of the robot, and transcribe the outbound call audio into text to obtain the transcription result; The outbound call audio and text transcription results are fused using a multimodal fusion model to obtain a multimodal emotion feature vector. Customer emotion analysis is then performed based on the multimodal emotion feature vector to obtain the customer emotion analysis results. The robot's outbound call results are inspected based on the customer sentiment analysis results, and a structured report is obtained.

2. The outbound call service quality inspection method as described in claim 1, characterized in that, The steps of performing vector fusion on the outbound call audio and text transcription results using a multimodal fusion model to obtain a multimodal emotion feature vector, and then performing customer emotion analysis based on the multimodal emotion feature vector to obtain the customer emotion analysis result include: Extract the customer's call audio from the outbound call audio, and extract the customer's text transcription result from the text transcription result; Calculate the time-domain and frequency-domain features of each frame of speech in the customer's call audio; The time-domain features and frequency-domain features are encoded to obtain a speech emotion vector; The text transcription result from the customer is semantically encoded to obtain a text sentiment vector. Based on the spoken emotion vector and the text emotion vector, vector fusion is performed through the attention mechanism of the multimodal fusion model to obtain a multimodal emotion feature vector; Customer sentiment analysis is performed based on the multimodal sentiment feature vectors to obtain customer sentiment analysis results.

3. The outbound call service quality inspection method as described in claim 1, characterized in that, The step of performing quality inspection and tagging on the outbound call audio and text transcription results using a multimodal fusion model to obtain a structured report includes: Based on the text transcription results, the robot's business completion results and compliance information are determined through an intent classification model. Based on the text transcription results and outbound call audio, the robot's behavior is analyzed using a robot behavior analysis model to obtain the robot behavior analysis results; Based on the behavioral analysis results, business completion results, compliance information, and customer sentiment analysis results, the robot is subjected to outbound call result quality inspection and labeling through a pre-acquired dynamic rule set to obtain a comprehensive score and problem details; A structured report is generated based on the comprehensive score and issue details.

4. The outbound call service quality inspection method as described in claim 3, characterized in that, Before the step of performing quality inspection and tagging of the robot's outbound call results using a pre-acquired dynamic rule set to obtain a comprehensive score, problem details, and improvement suggestions, the method further includes: Based on the robot's historical outbound call records, rules are initialized to obtain an initial quality inspection rule set; Based on historical quality inspection results and user feedback, the initial quality inspection rule set is learned and optimized through a knowledge graph rule engine to obtain an optimized quality inspection rule set. The optimized quality inspection rule set and the initial quality inspection rule set are integrated to obtain the final quality inspection rule set; Based on the behavioral analysis results, business completion results, compliance information, and customer sentiment analysis results, the final quality inspection rule set is weighted and prioritized to obtain a dynamic rule set.

5. The outbound call service quality inspection method as described in claim 3, characterized in that, After the step of performing outbound call result quality inspection on the robot based on the customer sentiment analysis results and obtaining a structured report, the method further includes: Based on the comprehensive score and problem details in the structured report, improvement suggestions are derived. The robot's call service is optimized using the aforementioned improvement suggestions to obtain an optimized robot; The improved models, intention classification models, and robot behavior analysis models are optimized based on the proposed improvements to obtain optimized parameters. Based on the optimization parameters, the multimodal fusion model, intent classification model, and robot behavior analysis model are fine-tuned to obtain the adjustment results; The dynamic rule set is adjusted based on the adjustment results to obtain the adjusted dynamic rule set.

6. The outbound call service quality inspection method as described in claim 5, characterized in that, After the step of optimizing the robot's call service using the improvement suggestions to obtain the optimized robot, the method further includes: Abnormal call records are determined based on the robot's historical outbound call records; Based on the abnormal call records, a call simulation is performed using the optimized robot to obtain the call simulation results; The simulated call results are sent to a human client for review, and a review result is obtained from the human client. Receive the audit results and determine the keywords of the abnormal calls based on the audit results; Establish external anchor points based on the abnormal call keywords; When the optimized robot is detected to have triggered abnormal call keywords, an external call is initiated through the external anchor point.

7. An outbound call service quality inspection device, characterized in that, The outbound call service quality inspection device includes: The acquisition module is used to acquire the robot's outbound call audio and transcribe the outbound call audio into text to obtain the transcription result. The analysis module is used to perform vector fusion on the outbound call audio and text transcription results through a multimodal fusion model to obtain a multimodal emotion feature vector, and to perform customer emotion analysis based on the multimodal emotion feature vector to obtain the customer emotion analysis result. The quality inspection module is used to perform outbound call quality inspection on the robot based on the customer sentiment analysis results and obtain a structured report.

8. An outbound call service quality inspection device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the outbound call service quality inspection method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the outbound call service quality inspection method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the outbound call service quality inspection method as described in any one of claims 1 to 6.

Images