Customer service quality acquisition method and device

Abstract

The present disclosure relates to the technical field of computer, and particularly relates to a customer service quality acquisition method and device. The method comprises the following steps: preprocessing customer service audio data, obtaining an audio segment set, and performing speech transcription processing on each audio segment in the audio segment set through an automatic speech recognition engine and a Transformer model to obtain a text segment set; obtaining a text emotion analysis result according to the text segment set, and obtaining a speech emotion analysis result according to the audio segment set; performing fusion processing on the text emotion analysis result and the speech emotion analysis result to obtain a fused emotion analysis result; and analyzing the fused emotion analysis result by using a multi-dimensional customer service quality scoring model to obtain a customer service quality evaluation result. The present disclosure can improve the accuracy of service quality determination and improve the accuracy of customer service quality inspection.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for obtaining customer service quality. Background Technology

[0002] With the development of science and technology, customer service quality inspection can effectively and quickly detect whether users express dissatisfaction, satisfaction, or a lack of emotion in their messages. It can also detect whether customer service representatives' attitudes and wording are standardized. For example, pre-set scoring templates can be used to evaluate customer service quality, and the scoring items in the templates can be used to score the content to be inspected in the customer service quality inspection task. However, different scenarios require different quality inspection templates, but these templates have limitations for customer service quality inspection, resulting in relatively poor accuracy. Summary of the Invention

[0003] This disclosure provides a method and apparatus for obtaining customer service quality, which can improve the accuracy of service quality determination and the accuracy of customer service quality inspection. The technical solution of this disclosure is as follows:

[0004] According to a first aspect of the present disclosure, a method for obtaining customer service quality is provided, comprising: The customer service audio data is preprocessed to obtain a set of audio segments. Then, the audio segments in the set of audio segments are transcribed using an Automatic Speech Recognition Engine (ASR) and a Transformer model to obtain a set of text segments. Based on the set of text segments, obtain text sentiment analysis results, and based on the set of audio segments, obtain speech sentiment analysis results; The text sentiment analysis results and the speech sentiment analysis results are fused to obtain the fused sentiment analysis results. A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain customer service quality evaluation results.

[0005] According to some embodiments, the method further includes at least one of the following: Regular expressions are used to process each text segment in the text segment set to obtain a processed text segment set, wherein the processing includes at least one of punctuation restoration processing and noun correction processing; A text processing model is used to process each text segment in the text segment set to obtain a processed text segment set. The processing includes at least one of punctuation restoration processing and noun correction processing.

[0006] According to some embodiments, obtaining text sentiment analysis results based on the set of text segments and obtaining speech sentiment analysis results based on the set of audio segments includes: A Natural Language Processing (NLP) model is used to concatenate the text fragments in the text fragment set to obtain sentiment type labels; A rule dictionary is used to identify each text fragment to obtain a set of emotion words; Based on the context of each text fragment, obtain information on the trend of emotional change; Based on a preset number of consecutive text segments in the text segment set, obtain an emotion trend sequence, and obtain an emotion turning point based on the emotion trend sequence; Add the emotion type label, the emotion word set, the emotion change trend information, and the emotion turning point to the text emotion analysis result; An acoustic model is used to perform voiceprint feature analysis on the audio segment set to obtain voiceprint features; The voiceprint features are associated with the emotion dimension to obtain the voice emotion analysis results.

[0007] According to some embodiments, the method further includes: If the text sentiment analysis results and the voice sentiment analysis results do not meet the requirements for emotional state, it is determined that there is an emotional conflict between the text sentiment analysis results and the voice sentiment analysis results. Based on the emotional conflict information between the text sentiment analysis results and the voice sentiment analysis results, conflict processing is performed on the text segment set and the audio segment set, and the weights of the scene information corresponding to the audio data are adjusted.

[0008] According to some embodiments, the step of using a multi-dimensional customer service quality scoring model to analyze the fused sentiment analysis results and obtain customer service quality evaluation results includes: Based on the keywords and keyword lexicon corresponding to each text segment in the text segment set, the scene information corresponding to the audio data is obtained; Based on the scenario information, obtain the weights corresponding to each customer service quality dimension in the customer service quality dimension set; A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain multi-dimensional customer service quality information; Based on the multi-dimensional customer service quality information and the weights corresponding to each customer service quality dimension, the customer service quality evaluation results are obtained.

[0009] According to some embodiments, the multi-dimensional customer service quality information includes at least one of service efficiency dimension information, professional competence dimension information, and emotional experience dimension information. The step of using a multi-dimensional customer service quality scoring model to analyze the fused emotional analysis results to obtain multi-dimensional customer service quality information includes: A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain response time and dialogue effectiveness, and based on the response time and dialogue effectiveness, the service efficiency dimension information is obtained. The multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain process integrity information and knowledge accuracy information, and based on the process integrity information and the knowledge accuracy information, the professional competence dimension information is obtained. The multi-dimensional customer service quality scoring model is used to analyze the fused emotion analysis results to obtain customer emotion change information and customer service affinity information, and based on the customer emotion change information and the customer service affinity information, the emotion experience dimension information is obtained.

[0010] According to some embodiments, the method further includes at least one of the following: If the fused sentiment analysis result does not meet the sentiment requirements, a solution strategy corresponding to the audio data is obtained from the knowledge base, and the solution strategy is sent to the customer service terminal. The sentiment requirements include at least one of speech rate requirements, volume requirements, and pause frequency requirements. Based on the process of obtaining the customer service quality evaluation results, a service generation report corresponding to the audio data is generated.

[0011] According to a second aspect of the present disclosure, a customer service quality acquisition device is provided, comprising: The set acquisition unit is used to preprocess customer service audio data, acquire a set of audio segments, and perform speech-to-text processing on each audio segment in the set of audio segments using an automatic speech recognition engine and a Transformer model to acquire a set of text segments. The result acquisition unit is used to acquire text sentiment analysis results based on the set of text segments and to acquire speech sentiment analysis results based on the set of audio segments. The result acquisition unit is also used to perform fusion processing on the text sentiment analysis result and the voice sentiment analysis result to obtain the fused sentiment analysis result; The quality acquisition unit is used to analyze the fused sentiment analysis results using a multi-dimensional customer service quality scoring model to obtain customer service quality evaluation results.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the customer service quality acquisition method described in any one of the preceding aspects.

[0013] According to a fourth aspect of the present disclosure, a storage medium is provided that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the customer service quality acquisition method described in any of the preceding aspects.

[0014] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in any one of the preceding aspects.

[0015] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects: In some or related embodiments, customer service audio data is preprocessed to obtain an audio segment set. Then, an automatic speech recognition engine and a Transformer model are used to transcribe each audio segment in the audio segment set to obtain a text segment set. Based on the text segment set, text sentiment analysis results are obtained, and based on the audio segment set, voice sentiment analysis results are obtained. The text sentiment analysis results and the voice sentiment analysis results are fused to obtain a fused sentiment analysis result. A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis result to obtain a customer service quality evaluation result. Therefore, by using an automatic speech recognition engine and multi-modal sentiment analysis, customer service quality can be evaluated from both customer service audio data and sentiment analysis. This reduces the time required to obtain customer emotion change information, allows for dual verification of text content and voice features in customer service quality evaluation, and enhances the objectivity and operability of the scoring through multi-dimensional customer service quality scoring models. It also improves the accuracy of obtaining customer service quality evaluation results, the accuracy of service quality determination, and the accuracy of customer service quality inspection, ultimately improving overall service quality.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0018] Figure 1 This is a flowchart of the first customer service quality acquisition method provided in this disclosure embodiment; Figure 2 This is a flowchart of the second customer service quality acquisition method provided in this disclosure embodiment; Figure 3 This is an example schematic diagram of a customer service quality acquisition method provided in an embodiment of this disclosure; Figure 4 This is an example schematic diagram illustrating a customer service audio data processing method provided in an embodiment of this disclosure; Figure 5 This is an example diagram illustrating an audio data to text conversion method provided in this embodiment of the disclosure; Figure 6 This is an example schematic diagram of a multimodal fusion analysis provided in an embodiment of this disclosure; Figure 7 This is an example diagram illustrating a customer service quality evaluation method provided in this embodiment of the disclosure; Figure 8 This is an example schematic diagram of an emotion detection method provided in an embodiment of this disclosure; Figure 9 This is an example schematic diagram of the display interface of a customer service terminal provided in an embodiment of this disclosure; Figure 10 This is an example schematic diagram illustrating report generation provided in an embodiment of this disclosure; Figure 11 This is a block diagram illustrating a customer service quality acquisition device according to an exemplary embodiment; Figure 12 This is an example schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0020] This disclosure provides a customer service quality acquisition method, apparatus, electronic device, and storage medium. In some embodiments, the terms "customer service quality acquisition method" and "information processing method," "communication method," etc., can be used interchangeably; the terms "customer service quality acquisition apparatus" and "information processing apparatus," "communication apparatus," etc., can be used interchangeably; and the terms "information processing system," "communication system," etc., can be used interchangeably.

[0021] This disclosure is not exhaustive, but merely illustrative of some embodiments, and is not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0022] In each of the disclosed embodiments, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0023] The terminology used in the embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure.

[0024] In this disclosure, unless otherwise stated, elements expressed in the singular form, such as "a," "an," "the," "the," "the," "the," "the," "the," "this," etc., can mean "one and only one," or "one or more," "at least one," etc. For example, when using articles such as "a," "an," "the," etc. in translation, the noun following the article can be understood as either a singular or a plural expression.

[0025] In the embodiments disclosed herein, "multiple" refers to two or more.

[0026] In some embodiments, the terms “at least one of,” “one or more,” “a plurality of,” and “multiple” may be used interchangeably.

[0027] The prefixes "first," "second," etc., used in the embodiments of this disclosure are merely for distinguishing different descriptive objects and do not impose restrictions on the position, order, priority, quantity, or content of the descriptive objects. The description of the descriptive objects is found in the claims or the context of the embodiments, and the use of prefixes should not constitute unnecessary restrictions. For example, if the descriptive object is a "field," the ordinal numbers preceding "field" in "first field" and "second field" do not restrict the position or order of the "fields." "First" and "second" do not restrict whether the "fields" they modify are in the same message, nor do they restrict the order of "first field" and "second field." Similarly, if the descriptive object is a "level," the ordinal numbers preceding "level" in "first level" and "second level" do not restrict the priority between "levels." Furthermore, the number of descriptive objects is not limited by ordinal numbers and can be one or more. For example, in "first device," the number of "devices" can be one or more. Furthermore, the objects modified by different prefixes can be the same or different. For example, if the object being described is "device", then "first device" and "second device" can be the same device or different devices, and their types can be the same or different. Similarly, if the object being described is "information", then "first information" and "second information" can be the same information or different information, and their content can be the same or different.

[0028] In some embodiments, "electronic device" or "terminal device" may be referred to as "user equipment (UE)," "user electronic device," "mobile station (MS)," "mobile electronic device (MT)," "subscriber station," "mobile unit," "subscriber unit," "wireless unit," "remote unit," "mobile device," "wireless device," "wireless communication device," "remote device," "mobile subscriber station," "access terminal," "mobile electronic device," "wireless terminal," "remote terminal," "handset," "user agent," "mobile client," "client," etc.

[0029] In some embodiments, data, information, etc., may be obtained with the user's consent.

[0030] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] Figure 1 This is a flowchart of the first customer service quality acquisition method provided in this disclosure embodiment, such as... Figure 1 As shown, this customer service quality acquisition method can be used in scenarios where captured images are rendered in virtual reality technology devices, and includes the following steps: In step S11, the customer service audio data is preprocessed to obtain a set of audio segments, and the audio segments in the set of audio segments are transcribed into text using an automatic speech recognition engine and a Transformer model to obtain a set of text segments. In some embodiments, the implementing entity of this disclosure may be, for example, an electronic device. This electronic device does not specifically refer to a particular fixed electronic device. For example, when the identifier of the electronic device changes, the electronic device itself may also change accordingly. For example, when the structure of the electronic device changes, the electronic device may also change accordingly.

[0032] According to some embodiments, customer service audio data can be, for example, audio data generated by customer service representatives during the service process, and this customer service audio data does not specifically refer to any fixed data. For example, the customer service audio data can vary depending on the customer service representative's identifier or the customer. For example, the customer service audio data can also vary depending on the time point at which it is acquired. For example, the customer service audio data can also vary depending on the specific data. This customer service audio data can be data collected in real time or data acquired offline. The customer service audio data can also be audio data, voice data, customer service voice data, etc.

[0033] According to some embodiments, preprocessing does not specifically refer to a particular fixed processing method. The preprocessing could be, for example, an audio segment used for subsequent processing. This preprocessing includes, but is not limited to, noise reduction and enhancement processing, segmentation processing, etc.

[0034] In some embodiments, audio segments can be obtained by segmenting customer service audio data. These audio segments may include different rounds of dialogue. The audio segment does not specifically refer to a fixed segment. Different audio segment segmentation methods may correspond to different audio segments. The audio segment set may be a collection of at least one audio segment. This audio segment set does not specifically refer to a fixed set. For example, when the segmentation method of the audio segments changes, the audio segment set may also change accordingly. For example, when the number of audio segments in the audio segment set changes, the audio segment set may also change accordingly. For example, when the customer service audio data changes, the audio segment set may also change accordingly.

[0035] According to some embodiments, an automatic speech recognition engine can be, for example, a device that converts speech signals into text. This ASR can be, for example, a sequence-to-sequence probability mapping: inputting an audio sequence and outputting the most probable text sequence. The automatic speech recognition engine is not specifically defined by a single engine. For example, when the engine identifier of the automatic speech recognition engine changes, the automatic speech recognition engine can also change accordingly. For example, when different scenarios correspond to different automatic speech recognition engines, the automatic speech recognition engine can also change accordingly when the scenario of the customer service voice data changes.

[0036] In some embodiments, the Transformer model may be, for example, a deep learning model based on a self-attention mechanism. The Transformer model in this disclosure is not specifically defined as a fixed model. For example, the Transformer model may change accordingly when its parameters change.

[0037] According to some embodiments, speech-to-text processing can be, for example, the process of converting speech data into text. Speech-to-text processing can also be referred to as speech conversion processing, transcription processing, etc., and this disclosure does not limit it to these terms.

[0038] In some embodiments, the text fragment set may be, for example, a collection of at least one text fragment. This text fragment set does not specifically refer to a fixed set. For example, the text fragment set may change when the number of text fragments it includes changes. Similarly, the text fragment set may change when a particular text fragment changes. The number of text fragments in the text fragment set may be the same as the number of audio fragments in the audio fragment set.

[0039] In some embodiments, customer service audio data can be preprocessed to obtain a set of audio segments, and each audio segment in the set of audio segments can be transcribed into a text segment set using an automatic speech recognition engine and a Transformer model.

[0040] In step S12, text sentiment analysis results are obtained based on the set of text segments, and speech sentiment analysis results are obtained based on the set of audio segments; In some embodiments, the text sentiment analysis result may be, for example, a sentiment analysis result obtained from a set of text segments. This text sentiment analysis result is not specifically defined by a single fixed result. For example, the text sentiment analysis result may change accordingly when the set of text segments changes. Similarly, the text sentiment analysis result may change accordingly when the method of obtaining the text sentiment analysis result changes.

[0041] According to some embodiments, the voice emotion analysis result may be, for example, an emotion analysis result obtained from a set of audio segments. This voice emotion analysis result is not specifically defined by a single fixed result. For example, the voice emotion analysis result may change accordingly when the set of audio segments changes. Similarly, the voice emotion analysis result may change accordingly when the method of obtaining the voice emotion analysis result changes.

[0042] In some embodiments, text sentiment analysis results can be obtained based on a set of text segments, and speech sentiment analysis results can be obtained based on a set of audio segments.

[0043] In step S13, the text sentiment analysis results and the voice sentiment analysis results are fused to obtain the fused sentiment analysis results; According to some embodiments, the fusion process may be, for example, a multimodal fusion process, or a multimodal fusion analysis, and this disclosure does not limit it in this way. The fusion process may, for example, be a process of combining information from different sensing channels.

[0044] In some embodiments, the text sentiment analysis results and the voice sentiment analysis results can be fused to obtain the fused sentiment analysis results.

[0045] In step S14, a multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain customer service quality evaluation results.

[0046] According to some embodiments, the multi-dimensional customer service quality scoring model can be, for example, a trained model that can be used for customer service quality evaluation. This multi-dimensional customer service quality scoring model does not specifically refer to a fixed model. For example, when the model type of the multi-dimensional customer service quality scoring model changes, the multi-dimensional customer service quality scoring model can also change accordingly. For example, when the model parameters of the multi-dimensional customer service quality scoring model change, the multi-dimensional customer service quality scoring model can also change accordingly.

[0047] According to some embodiments, the customer service quality evaluation result may be a rating result obtained based on current customer service audio data. This customer service quality evaluation result does not refer to a specific, fixed result. For example, when the customer service audio data changes, the customer service quality evaluation result may also change accordingly. Similarly, when the algorithm or model used in obtaining the customer service quality evaluation result changes, the customer service quality evaluation result may also change accordingly.

[0048] In some embodiments, a multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain customer service quality evaluation results.

[0049] In some or related embodiments, customer service audio data is preprocessed to obtain an audio segment set. Then, an automatic speech recognition engine and a Transformer model are used to transcribe each audio segment in the audio segment set to obtain a text segment set. Based on the text segment set, text sentiment analysis results are obtained, and based on the audio segment set, voice sentiment analysis results are obtained. The text sentiment analysis results and voice sentiment analysis results are fused to obtain a fused sentiment analysis result. A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis result to obtain a customer service quality evaluation result. Therefore, customer service quality can be evaluated from both customer service audio data and sentiment analysis using an automatic speech recognition engine and multi-modal sentiment analysis. This reduces the time required to obtain customer emotion change information, allows for dual verification of text content and voice features in customer service quality evaluation, and improves the objectivity and operability of the scoring through multi-dimensional customer service quality scoring models. It also improves the accuracy of obtaining customer service quality evaluation results, the accuracy of service quality determination, and ultimately, service quality.

[0050] Figure 2 This is a flowchart of the second customer service quality acquisition method provided in this disclosure embodiment, such as... Figure 2 As shown, it includes the following steps: In step S21, the customer service audio data is preprocessed to obtain a set of audio segments, and the audio segments in the set of audio segments are transcribed into text using an automatic speech recognition engine and a Transformer model to obtain a set of text segments. In some embodiments, the implementing entity of this disclosure may be, for example, an electronic device. This electronic device does not specifically refer to a particular fixed electronic device. For example, when the identifier of the electronic device changes, the electronic device itself may also change accordingly. For example, when the structure of the electronic device changes, the electronic device may also change accordingly.

[0051] According to some embodiments, the technical solutions of this disclosure can be applied to business consulting in the field of operators, and also have the ability to extend to multiple fields, such as: bank and financial customer service, e-commerce and logistics customer service, and are also applicable to different technical forms, such as: telephone customer service, online voice customer service and intelligent voice robot.

[0052] The relevant processes can be as described above, and will not be repeated here.

[0053] According to some embodiments, Figure 3 This is an example schematic diagram of a customer service quality acquisition method provided in this disclosure embodiment, such as... Figure 3 As shown, it may include: The first step is to collect audio data in real time and preprocess the audio data. The second step involves processing the audio clips into text using the ASR engine, and then restoring punctuation and correcting noun errors in the transcribed text. The third step is to use the transcribed text to perform text sentiment analysis on customers, use audio data to perform voice sentiment analysis on customers, and perform multimodal fusion analysis. Step 4: Rate the quality of customer service through multiple preset evaluation dimensions; Step 5: Real-time detection of negative emotions; when strong negative emotions are detected, an alarm is automatically triggered. Step 6: Generate a report for this service based on the content of the call. The specific descriptions of each step are as follows.

[0054] In some embodiments, Figure 4 This is an example diagram illustrating a customer service audio data processing method provided in an embodiment of this disclosure, such as... Figure 4 As shown, the system can retrieve real-time customer service audio data during a call or access recorded audio files from historical calls. Intelligent processing technology reduces and enhances ambient noise to ensure audio clarity meets standards. It can distinguish between customer and customer service voice sources in the audio, generating independent audio tracks for collection and management. Based on dialogue semantic nodes, it intelligently segments the recorded material into batches of segments organized according to natural dialogue turns. The overall preprocessing stage outputs standardized audio track data and segmented structures, providing a standardized input source for subsequent analysis modules.

[0055] According to some embodiments, the method further includes at least one of the following: Regular expressions are used to process each text fragment in the text fragment set to obtain the processed text fragment set. The processing includes at least one of punctuation restoration processing and noun error correction processing. A text processing model is used to process each text segment in the text segment set to obtain a processed text segment set. The processing includes at least one of punctuation restoration processing and noun correction processing.

[0056] In some implementations, regular expressions are used to quickly find, match, replace, and extract content from strings that conforms to rules.

[0057] According to some embodiments, the text processing model may be, for example, a trained model capable of processing text segments. This text processing model is not specifically defined by a single, fixed model. For example, the text processing model may change when its type changes. Similarly, the text processing model may change when its parameters change.

[0058] In some embodiments, the punctuation restoration processing may, for example, be inserting corresponding punctuation marks into unpunctuated text. The noun error correction processing may also be referred to as proper noun error correction processing, for example. The noun error correction processing may, for example, be correction processing performed when a noun does not meet noun requirements or does not conform to text expression manners.

[0059] According to some embodiments, Figure 5 is an exemplary schematic diagram of converting audio data to text provided by an embodiment of the present disclosure, as shown in Figure 5 , segmented audio, that is, a set of audio segments, may be acquired, which may be input to an end-to-end ASR engine in combination with a Transformer model to perform basic text composition and perform dynamic enhancement, and autonomous punctuation restoration, proper noun error correction, and scenario weight adjustment may be performed.

[0060] According to some embodiments, the ASR engine and the Transformer model may be used to perform speech transcription processing on the segmented audio, and speech quality is enhanced in the ASR engine, including: noise reduction, echo cancellation, voice activity detection, and speech enhancement. Feature extraction may be performed on the enhanced speech, and before speech recognition, the ASR engine may extract speech features, such as Mel-Frequency Cepstral Coefficients (MFCC), Mel FilterBank (FBANK), Mel spectrum, and the like. High-quality features depend on clear input speech, and the enhanced speech will improve the stability of feature extraction. The speech recognition audio in the ASR engine is subjected to text recognition, the input is the enhanced speech signal, and the output is the recognized text.

[0061] According to some embodiments, when performing punctuation restoration and proper noun error correction on the transcribed text, the goal of punctuation restoration may, for example, be inserting appropriate punctuation marks (such as periods, commas, question marks, etc.) into the unpunctuated text to make the text structure clear and the semantics unambiguous. For the rule-based and dictionary-based method, regular expressions are used to match common sentence structures, a pause dictionary (such as "ma", "ne", "ba") is used to predict question marks, and sentence-ending words (such as "le", "a") are used to assist in determining periods. Models such as Transformer, Bidirectional Encoder Representations from Transformers (BERT) based on a Transformer encoder, Bidirectional Long Short-Term Memory network (Bidirectional LSTM, BiLSTM) + Conditional Random Field (CRF) and the like may also be used for end-to-end punctuation restoration.

[0062] In step S22, text sentiment analysis results are obtained based on the set of text segments, and speech sentiment analysis results are obtained based on the set of audio segments; The relevant processes can be as described above, and will not be repeated here.

[0063] In some embodiments, for example, the emotional states of both parties in a call can be comprehensively analyzed, and the objectivity and comprehensiveness of emotion determination can be achieved through dual verification of text content and voice features.

[0064] According to some embodiments, text sentiment analysis results are obtained based on a set of text segments, and speech sentiment analysis results are obtained based on a set of audio segments, including: An NLP model is used to concatenate the text fragments in the text fragment set to obtain sentiment type labels; A rule-based dictionary is used to identify each text segment and obtain a set of emotion words; Based on the context of each text fragment, obtain information on the trend of emotional changes; Based on a preset number of consecutive text fragments in the text fragment set, obtain the emotion trend sequence, and then obtain the emotion turning point based on the emotion trend sequence; Add emotion type labels, emotion word sets, emotion change trend information, and emotion turning points to the text sentiment analysis results; An acoustic model is used to perform voiceprint feature analysis on a set of audio segments to obtain voiceprint features; By associating voiceprint features with the emotion dimension, speech emotion analysis results are obtained. Therefore, text-based sentiment analysis can identify key expressions from the customer's speech based on the transcribed text, locate explicit emotional words, and determine the text-based sentiment analysis results. Speech emotion analysis, based on segmented customer audio tracks, analyzes changes in speech rate, volume, fluctuations, and pause frequencies, converting acoustic features into emotional states to determine the speech-based sentiment analysis results. By fusing these two types of sentiment analysis results, the user's emotional turning points can be identified, accurately capturing changes in customer emotions.

[0065] In some embodiments, the Natural Language Processing (NLP) model may be, for example, a model for processing text, and the NLP model is not specifically defined as a fixed model. When the model parameters of the NLP model change, the NLP model may also change accordingly.

[0066] According to some embodiments, emotion type labels can be, for example, label information obtained through text. Emotion type labels can also be called emotion type label information, emotion category, etc. The NLP model can identify each piece of dialogue text or the concatenated text of dialogue turns in the input and output emotion category labels. (Emotion category labels can include, for example, anger, dissatisfaction, satisfaction, neutrality, frustration, etc.).

[0067] In some embodiments, Figure 6 This is an example schematic diagram of a multimodal fusion analysis provided in an embodiment of this disclosure, such as... Figure 6 As shown, it can perform text sentiment analysis, voice sentiment analysis, and multimodal fusion analysis.

[0068] In some implementations, text sentiment analysis, based on the text transcribed from the dialogue, identifies key expressions made by customers and customer service representatives, and locates potentially explicit emotional words (such as "dissatisfaction," "thank you," "urgent," etc.) as well as implicit semantic tendencies (e.g., "I suggest you be on time next time" implies dissatisfaction with this lateness). Each dialogue segment is then annotated with sentiment mechanisms using an NLP model, rule base, and contextual modeling to identify specific emotional scenarios.

[0069] According to some embodiments, a rule dictionary can be constructed to identify each text fragment and obtain a set of emotion words. This set of emotion words may include, for example, explicit emotion words, implicit emotion words, negative words, emotion-enhancing words, and emotion-transition words. Explicit emotion words may include, for example, dissatisfaction, poor, anger, satisfaction, gratitude, and fury; implicit emotion words may include, for example, hoping next time will be on time, not very satisfied, still not resolved, still the same old thing; negative words and emotion-enhancing words may include, for example, very, especially, simply, fundamentally, thoroughly, and completely; and emotion-transition words may include, for example, but, however, actually, however, originally thought.

[0070] In some embodiments, context modeling can be used to identify trends in emotional changes. By identifying text from consecutive dialogue rounds (such as the last 5 rounds of dialogue), a sequence of emotional trends (such as the emotional score for each round) can be obtained; if the change in emotional score exceeds a set threshold (such as 0.3) for three consecutive rounds, it is determined to be an emotional turning point.

[0071] According to some embodiments, speech sentiment analysis inputs speech segments into an acoustic model for voiceprint feature analysis. The voiceprint features are analyzed for speech rate, pitch, and pauses. The number of syllables in each segment of the acoustic model is used to analyze changes in speech rate (4-5 characters per second is considered normal for Chinese), volume fluctuations, and pause frequency (detecting blank areas in the audio track segment). These acoustic features are then correlated with the emotion dimension and converted into emotional states (e.g., if a customer's speech rate increases, volume rises, and pauses decrease, it can be labeled as "anger"; if the speech rate slows down, volume decreases, and there are long pauses, it can be labeled as "frustration," etc.).

[0072] In step S23, the text sentiment analysis results and the voice sentiment analysis results are fused to obtain the fused sentiment analysis results; The relevant processes can be as described above, and will not be repeated here.

[0073] In some embodiments, such as Figure 6 As shown, multimodal fusion analysis combines information from different perceptual channels (speech and text), primarily aiming to eliminate misjudgments in single-modal analysis. Emotional labels are acquired separately for text and speech. When conflicts arise between the detected emotional states in text and speech, the conflict is resolved by considering the context. Not every dialogue requires correction; fusion analysis is only performed on segments where the difference between text and speech judgments exceeds a threshold (e.g., confidence difference > 30%). Consistent single-modal results are considered for direct adoption (e.g., three consecutive audio segments detecting "anger").

[0074] In some embodiments, when handling conflicts in multimodal fusion analysis, the first step is to acquire the unimodal emotion labels and confidence levels, i.e., the emotion category and confidence level of the text emotion output, and the emotion category and confidence level of the voice emotion output. Next, it is determined whether a conflict has occurred. The conflict determination criteria are: inconsistent emotion categories (e.g., text is "satisfied," voice is "angry"); and the confidence level difference exceeds a preset threshold (e.g., set to 0.3). Then, fusion analysis is triggered, combining contextual semantics: checking for emotional shifts or negative expressions in the dialogue context; determining whether the current emotion is reasonable; examining the emotional change trends in the previous rounds of the dialogue for any sudden emotional changes or repeated fluctuations; and verifying whether the voice tone is consistent with the text semantics.

[0075] In some embodiments, dynamic weight allocation is performed based on the scenario when a conflict occurs: if the text emotion confidence is high, and the text contains words with obvious emotional content (such as "too difficult to use," "too bad," etc., in which case the text emotion is more reliable than the voice emotion), the text weight is set to 70%; if the voice emotion confidence is high, and the voice contains irony or euphemism, the voice weight is set to 70%, in which case the tone, speed, etc. in the audio track better reflect the user's emotions. When technical terms are used, the voice weight is set to 80%, in which case the text may contain mostly neutral vocabulary, and more consideration is given to the emotional expression in the voice.

[0076] In step S24, a multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain customer service quality evaluation results. The relevant processes can be as described above, and will not be repeated here.

[0077] According to some embodiments, a multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain customer service quality evaluation results, including: Based on the keywords and keyword lexicon corresponding to each text fragment in the text fragment set, obtain the scene information corresponding to the audio data; Based on the scenario information, obtain the weights corresponding to each customer service quality dimension in the customer service quality dimension set; A multi-dimensional customer service quality scoring model was used to analyze the fused sentiment analysis results to obtain multi-dimensional customer service quality information. The customer service quality evaluation results are obtained by using multi-dimensional customer service quality information and the corresponding weights for each dimension. Therefore, the accuracy of weight acquisition and the accuracy of the obtained customer service quality evaluation results can be improved.

[0078] According to some implementations, customer service quality evaluation results can include three aspects: service efficiency, professional competence, and emotional experience. During the audio data transcription process, text features are analyzed in real time to identify business scenario types. Specifically, scenario identification can be based on rules and keywords: establishing a keyword dictionary (e.g., "complaint," "poor," "refund," "poor service"); matching words appearing in the text with keywords in the keyword dictionary; and determining the scenario category based on the frequency and position of keywords. Alternatively, scenario identification can be performed using machine learning models, by inputting the text into a trained machine learning model for scenario recognition.

[0079] In some implementations, the weight of each evaluation dimension can be adjusted according to the type of business scenario. For example, when a complaint scenario is identified, the customer service quality score parameters are automatically configured as follows: service efficiency accounts for 0.4, professional ability accounts for 0.3, and emotional experience accounts for 0.3.

[0080] According to some embodiments, the multi-dimensional customer service quality information includes at least one of service efficiency dimension information, professional competence dimension information, and emotional experience dimension information. A multi-dimensional customer service quality scoring model is used to analyze the fused emotional analysis results to obtain multi-dimensional customer service quality information, including: A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain response time and dialogue effectiveness, and service efficiency dimension information is obtained based on response time and dialogue effectiveness. A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain process integrity information and knowledge accuracy information, and based on the process integrity information and knowledge accuracy information, professional competence dimension information is obtained. A multi-dimensional customer service quality scoring model is employed to analyze the fused sentiment analysis results, obtaining information on customer sentiment shifts and customer service rapport. Based on this information, emotional experience dimension information is derived. Therefore, by constructing a multi-dimensional service quality evaluation system encompassing service efficiency, professional competence, and emotional experience, and designing specific scoring formulas for each dimension—meaning different dimensions can be acquired using different methods, and the weight of each evaluation dimension is adjusted according to the call scenario—the final customer service score can be determined. This approach improves the accuracy of dimensional information acquisition, enhances the accuracy of customer service quality evaluation results, and improves the objectivity and operability of the scoring.

[0081] In some embodiments, Figure 7 This is an example diagram illustrating a customer service quality evaluation method provided in this embodiment of the disclosure, such as... Figure 7 As shown, a service quality evaluation system can be constructed based on multi-dimensional data analysis to objectively reflect the overall performance of customer service work. The evaluation dimensions include three aspects: service efficiency, professional competence, and emotional experience.

[0082] According to some implementations, service efficiency dimension information includes two indicators: response time (response duration) and dialogue efficiency. The sum of the scores for these two indicators constitutes the service efficiency score. Response time is the time interval between a customer's inquiry and the customer service representative's response, while dialogue efficiency calculates the effective information exchange density of a single dialogue.

[0083] The effective information exchange density is calculated using formula (1): Effective information exchange density = (Number of key actions completed × Weight + Number of effective information units) ÷ Total number of dialogue rounds × Complexity coefficient. (1) The key action completion count refers to the core behaviors that drive problem-solving in each round of dialogue, including: request confirmation (e.g., "Are you here to check your bill?"), key information provision (e.g., "Your bill for this month is XX yuan"), solution achievement (e.g., "I have processed your XX service"), and invalid responses (e.g., "Please repeat that."). Different actions can be assigned different weights; for example, request confirmation can be assigned 0.3, key information provision 0.5, solution achievement 1, and invalid responses -0.2 to distinguish them from normal actions. A valid information unit refers to an independent information block in the dialogue that contains entities, intents, and key parameters. For example, "The 38 yuan package I activated yesterday has not taken effect," the resolvable entities are: yesterday, package, 38 yuan, and the valid unit count is 3. The total number of dialogue rounds counts all interaction rounds from the customer service greeting to the service end. The complexity coefficient is set by the business department according to the specific situation. For example, a simple query (balance, fee query, etc.) is set to 0.2; a business processing (changing the package, etc.) is set to 0.8; and a cross-departmental cooperation is required, which is set to 1.5, etc.

[0084] In some implementations, the professional competence dimension includes two indicators: knowledge accuracy and process completeness. The sum of the scores for these two indicators constitutes the professional competence score. The knowledge accuracy dimension verifies the degree to which the solution conforms to business specifications, while process completeness checks the execution of key service steps. Process completeness is used to assess whether key service steps in customer service conversations are executed according to standards, ensuring service compliance and completeness. The necessary implementation steps for the service process can be set according to business circumstances. For example, in a customer consultation scenario, typical steps include: identity verification, problem confirmation, solution provision, and service closure confirmation. Based on the collected conversation text, it is determined whether customer service personnel have completely executed each step included in the service.

[0085] The process integrity score is calculated using formula (2): Process integrity score = Σ(weight of completed steps) × sequential compliance coefficient ÷ total step weight. (2) The weighting rules for service steps can be as follows: missing key steps, 30%; incorrect sequence, 15%; no response after timeout, 10%.

[0086] For example, consider the following dialogue: "Customer Service: What is your order number? Customer Service: 2025XXXX. Customer Service: The order has been found, and a refund will be issued. Customer: Okay." This dialogue lacks a crucial step: identity verification and the refund timeframe are not disclosed, resulting in a 30% missing step. The reason for the refund is not confirmed, representing an incorrect process sequence, accounting for 15%. The final score is: (70% problem-solving steps × 0.85 sequence coefficient) / 100% = 59.5% According to some implementation examples, the emotional experience dimension includes two indicators: the effectiveness of customer emotional reassurance and the friendliness of customer service. The sum of the scores of the two indicators is the emotional experience score.

[0087] In some implementations, the customer emotional reassurance effect is recorded by identifying key points in the transition from negative to positive emotions, determined through multimodal fusion analysis. For example, if a point in the call shows a transition from negative to positive emotions, the customer emotional reassurance score increases by 1 point.

[0088] Customer service friendliness assessment evaluates customer service tone control and proactive care performance. Tone control is assessed through voice feature extraction, detecting intonation stability and speech rate adaptability, and identifying prohibited words such as negative expressions like "I don't know" or "there's no way."

[0089] The tone control score can be calculated using formula (3): Tone control score = 0.5 × intonation stability + 0.5 × speech rate matching - number of banned words × 0.1. (3) Among them, proactive care performance is tested to see if customer service representatives show emotional empathy (such as "I understand your feelings" and lower their speaking speed and tone), progress assurance (such as "I will serve you throughout the process"), or additional assistance (such as "I will look up alternative solutions for you" or "I will contact other departments for you").

[0090] According to some embodiments, the final score can be calculated after dynamically adjusting the weights of service efficiency, professional competence, and emotional experience dimensions. For example, in a complaint scenario, the final customer service score formula is formula (4): Customer service rating = 0.4 Service efficiency score +0.3 Professional competence score +0.3 Emotional experience score. (4) In the consultation scenario, the final customer service rating formula is formula (5): Customer service rating = 0.3 - Service efficiency score + 0.5 Professionalism score +0.2 Emotional experience score. (5) In step S25, if the text sentiment analysis results and the voice sentiment analysis results do not meet the requirements of the emotional state, it is determined that there is an emotional conflict between the text sentiment analysis results and the voice sentiment analysis results. The relevant processes can be as described above, and will not be repeated here.

[0091] According to some embodiments, an unsatisfactory emotional state can be, for example, a failure to meet the emotional requirements corresponding to the current scenario. This failure to meet the emotional requirements can be, for example, a strong negative emotion.

[0092] In step S26, based on the emotional conflict information between the text sentiment analysis results and the voice sentiment analysis results, conflict processing is performed on the text segment set and the audio segment set, and the weights of the scene information corresponding to the audio data are adjusted.

[0093] The relevant processes can be as described above, and will not be repeated here.

[0094] According to some embodiments, the method further includes at least one of the following: If the fused sentiment analysis results do not meet the sentiment requirements, a solution strategy corresponding to the audio data is obtained from the knowledge base and sent to the customer service terminal. The sentiment requirements include at least one of the following: speech rate requirements, volume requirements, and pause frequency requirements. Based on the process of obtaining customer service quality evaluation results, a service generation report corresponding to the audio data is generated.

[0095] In some embodiments, Figure 8 This is an example schematic diagram of an emotion detection method provided in this disclosure embodiment, such as... Figure 8 As shown, the system can detect negative emotions in customers in real time, and automatically trigger an alarm when the detected emotion value exceeds a threshold, indicating that the customer is experiencing strong negative emotions. For example, if a customer's emotion changes from neutral to angry, and the emotion value changes from 0.5 to 0.85, an alarm will be triggered if the emotion continues to deteriorate or suddenly intensifies.

[0096] In some embodiments, when detecting customer emotion values, for example, speech rate / pitch / pause analysis can be performed on voiceprint features. The number of syllables in each segment of the acoustic model is used to analyze changes in speech rate (normal for Chinese is generally 4-5 characters per second), volume fluctuations, and pause frequency (detecting blank areas in the audio track segment). The extracted features need to be input into a machine learning model for classification. The machine learning model will output a probability distribution, for example: anger: 0.75, sadness: 0.15, happiness: 0.08, neutral: 0.02. The "emotion value" can then be defined as: directly taking the maximum probability: 0.75, and corresponding to the label "anger". Alternatively, a comprehensive score can be constructed: for example, defining "anger emotion value" = (anger probability + 0.5) The formula is: (Neutral probability) - (Sadness probability + Happiness probability + Fear probability), which will give you a continuous positive and negative score.

[0097] In some implementations, recommended solutions can be linked to a knowledge base, and key clauses of business specifications can be highlighted in real time. When a customer's mood deteriorates, standard response templates or problem solutions from the knowledge base can be automatically retrieved based on the current conversation content (such as customer requests and problem types) and pushed to the customer service terminal for customer service personnel to refer to.

[0098] When pushing standard response templates or problem solutions, Natural Language Understanding (NLU) can be used to identify intent and extract entities from the current dialogue content, aiming to accurately understand the user's intent. Based on the output of the NLU module, a knowledge base can be invoked to retrieve solutions. This can include exact matching and semantic retrieval. Under exact matching, when NLU identifies the intent `query_balance` (a command to query account balance), it can directly match a predefined action (such as calling a database API to query the balance), and then fill the query result (e.g., 50 yuan) into the corresponding response template to generate the final reply: "Hello, your current phone balance is 50 yuan," and push it to customer service.

[0099] In semantic retrieval, when user questions are very open-ended and their intent cannot be precisely categorized (e.g., "Why does my 5G phone sometimes drop to 4G?"), it's necessary to find the most relevant fragments from a massive amount of documents. Here, Vector Similarity Search is used, employing deep learning models (such as Sentence-BERT) to transform all document fragments in the knowledge base and the user's question into high-dimensional vectors (Embeddings). These vectors contain the semantic information of the text. The cosine similarity between the vector of the user's question and the vectors of all document fragments in the knowledge base is calculated, and the document fragments with the highest similarity are returned as a reference for the answer. The customer service terminal's display interface could be, for example, as shown below. Figure 9 As shown.

[0100] When customers are emotionally agitated, customer service staff should be reminded to follow service guidelines, such as not using prohibited language, remaining calm, and proactively apologizing. Real-time prompts should be used to prevent a decline in service quality due to emotional outbursts.

[0101] In some embodiments, Figure 10 This is an example schematic diagram illustrating a report generation method provided in this disclosure embodiment, such as... Figure 10As shown, a report can be generated based on the content of the call. Based on the report information, a corresponding SMS message or video ringback tone containing a summary of key questions and answers can be generated and sent to the terminal. This terminal could be, for example, the customer service terminal. After the service is completed, the solution, combined with the video ringback tone or SMS message, is sent to the user through the service result feedback module to improve the user experience.

[0102] 1. Key information extraction and summary generation: Using the prompt word "Prompt," input the complete dialogue history between customer service and the user, along with the original text of the final adopted solution. By setting the prompt word "Prompt," the Large Language Model (LLM) is guided to complete the task. The final output is structured data.

[0103] Example prompts: "You are a professional customer service summary robot. Please extract information from the following dialogue. Please output strictly according to the JSON format, including the following fields: customer_problem: Summarize the user's core problem in one sentence. root_cause: Analyze the most likely cause of the problem in one sentence. solution_summary: Summarize the provided solution in one sentence. key_quote: Generate an attractive slogan of up to 30 characters to promote the solution, starting with "【】". tags: Generate 3-5 keyword tags. The dialogue is as follows: <Insert dialogue text here>".

[0104] 2. Generate SMS: Based on the structured data output from the above steps, a simple template can be filled in to generate an SMS containing key question and answer summaries.

[0105] 3. Generate Video Ringtones / Voice: This function supports text-to-speech (TTS). Input: A summary of the solution generated in step 1. The speech synthesis model is invoked to generate the speech. Output: An audio file in .mp3 or .wav format.

[0106] Video Template Matching and Synthesis: A series of dynamic video templates with different styles and themes (such as "network optimization," "package upgrade," and "troubleshooting") are pre-stored. Templates can have transparent backgrounds or reserved text space. Example: A short video template with an animated operator logo and a flowing 5G symbol background. Based on the output question tags, the most matching video template is automatically selected. For example, if the tags contain ["5G," "network switching"], then the template in the "5G network" category will be matched. Key summary text can be rendered as dynamic subtitles into the reserved area of ​​the video template. The TTS-generated audio and the rendered subtitles are synthesized and aligned to generate the final video file. Specifically, the final video file can be uploaded to a video ringback tone platform, and the user number will be automatically subscribed to and set with this video ringback tone.

[0107] In one or related embodiments, if the text sentiment analysis results and the voice sentiment analysis results do not meet the emotional state requirements, it is determined that there is an emotional conflict between the text sentiment analysis results and the voice sentiment analysis results. Based on the emotional conflict information between the text sentiment analysis results and the voice sentiment analysis results, conflict processing is performed on the text segment set and the audio segment set, and the weights of the scene information corresponding to the audio data are adjusted. Therefore, accurately determining the customer's emotional change information through dual verification of text content and voice features can improve the accuracy of obtaining emotional change information and the accuracy of obtaining customer service quality evaluation results.

[0108] A block diagram illustrating a customer service quality acquisition device according to an exemplary embodiment. (Refer to...) Figure 11 The device 1100 includes: The set acquisition unit 1101 is used to preprocess customer service audio data, acquire a set of audio segments, and perform speech-to-text processing on each audio segment in the set of audio segments through an automatic speech recognition engine and a Transformer model to acquire a set of text segments. The result acquisition unit 1102 is used to acquire text sentiment analysis results based on the set of text segments and to acquire speech sentiment analysis results based on the set of audio segments. The result acquisition unit 1102 is also used to fuse the text sentiment analysis results and the speech sentiment analysis results to obtain the fused sentiment analysis results. The quality acquisition unit 1103 is used to analyze the fused sentiment analysis results using a multi-dimensional customer service quality scoring model to obtain customer service quality evaluation results.

[0109] According to some embodiments, the collection acquisition unit 1101 is also specifically used for at least one of the following: Regular expressions are used to process each text fragment in the text fragment set to obtain the processed text fragment set. The processing includes at least one of punctuation restoration processing and noun error correction processing. A text processing model is used to process each text segment in the text segment set to obtain a processed text segment set. The processing includes at least one of punctuation restoration processing and noun correction processing.

[0110] According to some embodiments, when the result acquisition unit 1102 is used to acquire text sentiment analysis results based on a set of text segments and to acquire speech sentiment analysis results based on a set of audio segments, it is specifically used for: An NLP model is used to concatenate the text fragments in the text fragment set to obtain sentiment type labels; A rule-based dictionary is used to identify each text segment and obtain a set of emotion words; Based on the context of each text fragment, obtain information on the trend of emotional changes; Based on a preset number of consecutive text fragments in the text fragment set, obtain the emotion trend sequence, and then obtain the emotion turning point based on the emotion trend sequence; Add emotion type labels, emotion word sets, emotion change trend information, and emotion turning points to the text sentiment analysis results; An acoustic model is used to perform voiceprint feature analysis on a set of audio segments to obtain voiceprint features; By associating voiceprint features with the emotion dimension, the results of voice emotion analysis can be obtained.

[0111] According to some embodiments, the result acquisition unit 1102 is further specifically used for: If the text sentiment analysis results and the voice sentiment analysis results do not meet the requirements for emotional state, it is determined that there is an emotional conflict between the text sentiment analysis results and the voice sentiment analysis results. Based on the emotional conflict information between the text sentiment analysis results and the voice sentiment analysis results, conflict resolution is performed on the text segment set and the audio segment set, and the weights of the scene information corresponding to the audio data are adjusted.

[0112] According to some embodiments, the quality acquisition unit 1103 is used to analyze the fused sentiment analysis results using a multi-dimensional customer service quality scoring model to obtain customer service quality evaluation results, specifically for: Based on the keywords and keyword lexicon corresponding to each text fragment in the text fragment set, obtain the scene information corresponding to the audio data; Based on the scenario information, obtain the weights corresponding to each customer service quality dimension in the customer service quality dimension set; A multi-dimensional customer service quality scoring model was used to analyze the fused sentiment analysis results to obtain multi-dimensional customer service quality information. The customer service quality evaluation results are obtained based on multi-dimensional customer service quality information and the corresponding weights of each customer service quality dimension.

[0113] According to some embodiments, the multi-dimensional customer service quality information includes at least one of service efficiency dimension information, professional ability dimension information, and emotional experience dimension information. The quality acquisition unit 1103 is used to analyze the fused emotional analysis results using a multi-dimensional customer service quality scoring model. Specifically, when acquiring the multi-dimensional customer service quality information, it is used for: A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain response time and dialogue effectiveness, and service efficiency dimension information is obtained based on response time and dialogue effectiveness. A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain process integrity information and knowledge accuracy information, and based on the process integrity information and knowledge accuracy information, professional competence dimension information is obtained. A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results, obtain customer sentiment change information and customer service affinity information, and obtain emotional experience dimension information based on customer sentiment change information and customer service affinity information.

[0114] According to some embodiments, the quality acquisition unit 1103 is also specifically used for at least one of the following: If the fused sentiment analysis results do not meet the sentiment requirements, a solution strategy corresponding to the audio data is obtained from the knowledge base and sent to the customer service terminal. The sentiment requirements include at least one of the following: speech rate requirements, volume requirements, and pause frequency requirements. Based on the process of obtaining customer service quality evaluation results, a service generation report corresponding to the audio data is generated.

[0115] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0116] Figure 12A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device 1200 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0117] like Figure 12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the electronic device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0118] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of displays, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0119] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above. For example, in some embodiments, the above methods can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 1201 can be configured to perform the above methods by any other suitable means (e.g., by means of firmware).

[0120] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0122] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0124] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0125] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0126] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0127] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for obtaining customer service quality, characterized in that, include: Preprocess the customer service audio data to obtain a set of audio segments, and use an automatic speech recognition engine and Transformer model to perform speech-to-text processing on each audio segment in the set of audio segments to obtain a set of text segments. Based on the set of text segments, obtain text sentiment analysis results, and based on the set of audio segments, obtain speech sentiment analysis results; The text sentiment analysis results and the speech sentiment analysis results are fused to obtain the fused sentiment analysis results. A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain customer service quality evaluation results.

2. The method according to claim 1, characterized in that, The method further includes at least one of the following: Regular expressions are used to process each text segment in the text segment set to obtain a processed text segment set, wherein the processing includes at least one of punctuation restoration processing and noun correction processing; A text processing model is used to process each text segment in the text segment set to obtain a processed text segment set. The processing includes at least one of punctuation restoration processing and noun correction processing.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining text sentiment analysis results based on the set of text segments and obtaining speech sentiment analysis results based on the set of audio segments includes: An NLP model is used to concatenate the text fragments in the text fragment set to obtain sentiment type labels; A rule dictionary is used to identify each text fragment to obtain a set of emotion words; Based on the context of each text fragment, obtain information on the trend of emotional change; Based on a preset number of consecutive text segments in the text segment set, obtain an emotion trend sequence, and obtain an emotion turning point based on the emotion trend sequence; Add the emotion type label, the emotion word set, the emotion change trend information, and the emotion turning point to the text emotion analysis result; An acoustic model is used to perform voiceprint feature analysis on the audio segment set to obtain voiceprint features; The voiceprint features are associated with the emotion dimension to obtain the voice emotion analysis results.

4. The method according to claim 1, characterized in that, The method further includes: If the text sentiment analysis results and the voice sentiment analysis results do not meet the requirements for emotional state, it is determined that there is an emotional conflict between the text sentiment analysis results and the voice sentiment analysis results. Based on the emotional conflict information between the text sentiment analysis results and the voice sentiment analysis results, conflict processing is performed on the text segment set and the audio segment set, and the weights of the scene information corresponding to the audio data are adjusted.

5. The method according to claim 1, characterized in that, The process involves using a multi-dimensional customer service quality scoring model to analyze the fused sentiment analysis results and obtain customer service quality evaluation results, including: Based on the keywords and keyword lexicon corresponding to each text segment in the text segment set, the scene information corresponding to the audio data is obtained; Based on the scenario information, obtain the weights corresponding to each customer service quality dimension in the customer service quality dimension set; A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain multi-dimensional customer service quality information; Based on the multi-dimensional customer service quality information and the weights corresponding to each customer service quality dimension, the customer service quality evaluation results are obtained.

6. The method according to claim 5, characterized in that, in, The multi-dimensional customer service quality information includes at least one of service efficiency dimension information, professional competence dimension information, and emotional experience dimension information. The multi-dimensional customer service quality scoring model is used to analyze the fused emotional analysis results to obtain multi-dimensional customer service quality information, including: A multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain response time and dialogue effectiveness, and based on the response time and dialogue effectiveness, the service efficiency dimension information is obtained. The multi-dimensional customer service quality scoring model is used to analyze the fused sentiment analysis results to obtain process integrity information and knowledge accuracy information, and based on the process integrity information and the knowledge accuracy information, the professional competence dimension information is obtained. The multi-dimensional customer service quality scoring model is used to analyze the fused emotion analysis results to obtain customer emotion change information and customer service affinity information, and based on the customer emotion change information and the customer service affinity information, the emotion experience dimension information is obtained.

7. The method according to claim 1, characterized in that, The method further includes at least one of the following: If the fused sentiment analysis result does not meet the sentiment requirements, a solution strategy corresponding to the audio data is obtained from the knowledge base, and the solution strategy is sent to the customer service terminal. The sentiment requirements include at least one of speech rate requirements, volume requirements, and pause frequency requirements. Based on the process of obtaining the customer service quality evaluation results, a service generation report corresponding to the audio data is generated.

8. A customer service quality acquisition device, characterized in that, include: The set acquisition unit is used to preprocess customer service audio data, acquire a set of audio segments, and perform speech-to-text processing on each audio segment in the set of audio segments using an automatic speech recognition engine and a Transformer model to acquire a set of text segments. The result acquisition unit is used to acquire text sentiment analysis results based on the set of text segments and to acquire speech sentiment analysis results based on the set of audio segments. The result acquisition unit is also used to perform fusion processing on the text sentiment analysis result and the voice sentiment analysis result to obtain the fused sentiment analysis result; The quality acquisition unit is used to analyze the fused sentiment analysis results using a multi-dimensional customer service quality scoring model to obtain customer service quality evaluation results.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the customer service quality acquisition method as described in any one of claims 1 to 7.

10. A storage medium storing instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the customer service quality acquisition method as described in any one of claims 1 to 7.

Images