Intelligent evaluation method, system and medium for call voice

Abstract

The present invention discloses an intelligent evaluation method, system and medium for call voice, which belongs to the field of voice processing technology, including: converting question-answer voice into standard voice based on voice quality, extracting question-answer text; the respondent outputs answer voice and corresponding reply text based on question-answer text, and stores call records to generate call paragraphs and call recordings; extracts emotional vocabulary in call paragraphs and answer time distribution in call recordings respectively, generates question-answer scores based on emotional vocabulary, and generates answer scores based on answer time distribution; retrieves reference texts corresponding to question understanding characters in an answer database, fuses all reference texts to generate summary texts, compares the overlap between reply text and summary text, and generates reply scores based on the overlap; summarizes question-answer scores, answer scores and reply scores, and generates call evaluation values ​​of call records. The present invention can improve the accuracy and comprehensiveness of intelligent evaluation of call voice.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech processing, and in particular relates to an intelligent evaluation method, system and medium for call speech. Background Art

[0002] With the development of communication technology, the evaluation of call voice quality has become increasingly important. Traditional voice quality evaluation methods mainly rely on manual listening, which is not only inefficient but also highly subjective and difficult to meet the needs of large-scale and automated evaluation. Therefore, it is necessary to automatically, quickly and accurately evaluate the quality and effectiveness of call voice.

[0003] Similar prior art includes a Chinese patent application with publication number CN112735421A, which discloses a real-time quality inspection method and device for voice calls, and relates to the field of computer technology. The method includes: collecting the voice media stream generated in real time during the voice call, performing voice recognition on the voice media stream to obtain corresponding text data; performing semantic analysis on the text data, and matching the semantic analysis results with pre-set quality inspection rules to generate quality inspection results. The method can generate quality inspection results in real time by collecting real-time voice media streams, performing voice recognition and semantic analysis on the voice media streams, and matching rules, and thus is highly efficient. In addition, similar prior art also includes a Chinese patent application with publication number CN114220419A, which discloses a speech evaluation method, which includes: obtaining speech to be evaluated and reference text, extracting acoustic features from the speech, inputting the acoustic features into K sub-acoustic models of the acoustic model, and obtaining a comprehensive state posterior probability; aligning the phonemes of the speech with the reference text according to the comprehensive state posterior probability and the segmentation network constructed by the reference text, and obtaining a scoring feature based on the alignment result; inputting the scoring feature into the scoring model to regress the score of the speech. Since the K sub-acoustic models are trained by their own in-set data and out-of-set data, and the out-of-set data of each sub-acoustic model is different, the acoustic model has the ability to distinguish the out-of-set data, thereby improving the scoring accuracy of abnormal speech, solving the problem of unstable scoring of abnormal speech, and having high reliability and availability.

[0004] However, the above-mentioned prior art only performs voice quality inspection on the voice media stream and scores the stability of abnormal voice. In actual situations, the call voice not only needs to correct the voice quality, but also needs to evaluate the response satisfaction of the call voice. Summary of the invention

[0005] To solve the above problems, the present invention provides a method, system and medium for intelligent evaluation of call speech to solve the problems in the prior art.

[0006] In order to achieve the above-mentioned object of the invention, the present invention proposes an intelligent evaluation method for call speech, comprising:

[0007] S1: acquiring the question-and-answer voice of the questioner after the call starts, identifying the voice quality of the question-and-answer voice, converting the question-and-answer voice into a standard voice based on the voice quality, extracting text characters of the standard voice and setting them as the question-and-answer text;

[0008] S2: The respondent outputs a response voice based on the question and answer text, obtains a reply text corresponding to the response voice, and stores the call record between the questioner and the respondent to generate a call segment and a call recording;

[0009] S3: extracting emotional words in the call segment and answering time distribution in the call recording respectively, generating a question-answering score based on the emotional words, and generating an answering score based on the answering time distribution;

[0010] S4: Obtain the question understanding characters corresponding to the question and answer text, build an answer database, retrieve the reference texts corresponding to the question understanding characters in the answer database, merge all the reference texts to generate a summary text, compare the overlap between the answer text and the summary text, and generate a reply score based on the overlap;

[0011] S5: Summarize the question and answer score, the response score, and the reply score to generate a call evaluation value of the call record.

[0012] Further, converting the question-answering speech into a standard speech based on the speech quality comprises the following steps:

[0013] Collect a speech sample set, the speech sample set including a plurality of normal speech and ambiguous speech, obtain a speech waveform set corresponding to the speech sample set, extract speech features of all speech waveforms in the speech waveform set, compare differences between normal speech waveforms and ambiguous speech waveforms in the speech waveform set based on the speech features, generate a speech training model for identifying the ambiguous speech waveform based on the differences, input the speech waveform set into the speech training model, and pre-train the speech training model using a self-supervised learning method to generate a speech conversion model;

[0014] The question-and-answer speech is input into the speech conversion model, the speech conversion model predicts the pitch and intensity corresponding to the question-and-answer speech, and generates a spectrogram based on the pitch and the intensity, and reconstructs the spectrogram to generate a conversion waveform, and the conversion waveform generates a converted speech based on a frequency domain conversion method, and is set as the standard speech.

[0015] Furthermore, generating the call evaluation value of the call record includes the following steps:

[0016] Setting emotion tags, wherein the emotion tags include a joy tag, a calm tag, and a dull tag, establishing an emotion evaluation vocabulary based on the emotion tags, respectively obtaining the semantic similarity between the call segment and any phrase in the emotion evaluation vocabulary, extracting the emotion vocabulary in the call segment based on the semantic similarity, obtaining the emotion tag corresponding to the emotion vocabulary and setting it as the emotion type;

[0017] Constructing a coordinate system based on the emotion type, accumulating the number of the same emotion type and drawing a score histogram in the coordinate system, obtaining the cumulative value of any emotion label in the score histogram, calculating the ratio of the sum of all the cumulative values ​​to the cumulative value corresponding to the boring label, and setting it as the question and answer score;

[0018] Dividing the response time distribution into cumulative communication duration and cumulative silence duration based on the call recording waveform strength, calculating the ratio between the total duration of the response time distribution and the cumulative silence duration, and setting it as the response score;

[0019] The answer score F is calculated based on a first formula, which is: Where i is a positive integer from 1 to I, I is the total number of reply texts in the call record, and n i is the number of identical characters between the ith reply text and the corresponding ith summary text, N i is the total number of characters in the i-th summary text, and the question and answer score, the answer score and the reply score are accumulated to generate the call evaluation value.

[0020] Furthermore, the step of obtaining the question understanding characters corresponding to the question and answer text comprises the following steps:

[0021] Convert the question-and-answer text into a standard form text, construct a knowledge dictionary, compare the standard form text with the semantic text stored in the knowledge dictionary, identify the object identifier and the relationship identifier in the standard form text, set the relationship identifier as a root node, set the object identifier as a child node, connect the child node corresponding to the object identifier to the root node corresponding to the associated relationship identifier, generate a tree structure based on the root node and the child node, and repeat this step until all root nodes and all child nodes are included in the tree structure;

[0022] The question understanding character is generated by sequentially combining the identifiers corresponding to all nodes in the tree structure based on the arrangement order of the relationship identifiers in the standard form text, wherein the nodes include the root node and the child nodes, and the identifiers include the thing identifier and the relationship identifier.

[0023] Furthermore, converting the question and answer text into a standard form text comprises the following steps:

[0024] A standard word structure for a sentence is set, and the phrases and corresponding word types contained in the question and answer text are obtained. Based on the standard word structure for the sentence, the missing word type of the word type is obtained. Based on the knowledge dictionary and the semantic matching method, the characters corresponding to the missing word type are completed and defined as a supplementary phrase. Based on the standard word structure for the sentence, the phrases and the supplementary phrases are sorted and combined to generate a completed sentence, and the completed sentence is set as the standard form text.

[0025] Further, searching the answer database for a reference text corresponding to the question understanding character includes:

[0026] The answer database includes question and answer information and answer information, and the character similarity between any identifier in the question understanding character and the question and answer information is obtained respectively, and the question and answer information whose character similarity is greater than or equal to a first preset value is extracted and set as a key question phrase, and the answer information corresponding to the key question phrase in the answer database is obtained, and all the answer information is combined and set as the reference text.

[0027] Furthermore, the step of fusing all reference texts to generate summary text includes the following steps:

[0028] The question understanding character is set as a query text, all the reference texts are sequentially combined based on the arrangement order of any of the identifiers in the question understanding character, and are set as source texts, and the query text and the source text are respectively split into a plurality of word sequences;

[0029] A binary cross entropy function is set as an objective function and a summary generation model is constructed. The summary generation model estimates the importance of the word sequence of the source text based on the word sequence of the query text, and outputs a predicted value of the word sequence in the source text based on the importance, sets the word sequence whose predicted value is greater than or equal to a second preset value as a summary sequence, sets a summary character length, and the summary generation model combines all the summary sequences based on the summary character length to generate the summary text.

[0030] Furthermore, based on the time series, it is determined whether the question and answer text belongs to the first question and answer. If not, the semantic relevance between the question and answer text and its previous question and answer text is calculated. If the semantic relevance is greater than or equal to a third preset value, the question and answer text and the summary text corresponding to the previous question and answer text are fused to generate a new question and answer text.

[0031] The present invention also provides an intelligent evaluation system for call speech, which is used to implement the above-mentioned intelligent evaluation method for call speech, and the system mainly includes:

[0032] A voice acquisition module, which acquires the question-and-answer voice of the questioner after the call starts, identifies the voice quality of the question-and-answer voice, converts the question-and-answer voice into a standard voice based on the voice quality, extracts text characters of the standard voice and sets them as the question-and-answer text;

[0033] A statistics module, in which the respondent outputs a response voice based on the question and answer text, obtains a reply text corresponding to the response voice, and stores the call records between the questioner and the respondent to generate call segments and call recordings;

[0034] An analysis module, which extracts the emotional words in the call segment and the answering time distribution in the call recording, generates a question-answering score based on the emotional words, and generates an answering score based on the answering time distribution;

[0035] An evaluation output module obtains the question understanding characters corresponding to the question and answer text, constructs an answer database, retrieves the reference text corresponding to the question understanding characters in the answer database, fuses all reference texts to generate a summary text, compares the overlap between the reply text and the summary text, and generates a reply score based on the overlap; summarizes the question and answer score, the answer score and the reply score to generate a call evaluation value for the call record.

[0036] The present invention also provides a computer storage medium, which stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the above-mentioned intelligent evaluation method for call voice.

[0037] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0038] The present invention first recognizes and corrects the question and answer voice, which can accurately generate normal standard voice and improve the accuracy of call voice. Then, by constructing an answer database, it can realize intelligent generation of summary text corresponding to the question and answer text as a reference answer, and evaluate the validity and completeness of the reply text output by the respondent based on the summary text. Finally, the call evaluation value of the call record is calculated by statistically analyzing the question and answer score, the answer score and the reply score, which can improve the accuracy of the intelligent evaluation of call voice.

[0039] The present invention can also improve the rapidity of call voice by generating a voice conversion model to accurately identify the voice quality of question and answer voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1A flowchart of a method for intelligently evaluating call speech according to the present invention;

[0041] Figure 2 An example diagram of generating a tree structure in the present invention;

[0042] Figure 3 This is a structural diagram of an intelligent evaluation system for call speech according to the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0045] like Figure 1 As shown, an intelligent evaluation method for call voice includes:

[0046] S1: Acquire the question-and-answer voice of the questioner after the call starts, identify the voice quality of the question-and-answer voice, convert the question-and-answer voice into a standard voice based on the voice quality, extract text characters of the standard voice and set them as the question-and-answer text.

[0047] Specifically, in this embodiment, the call voice refers to the voice conversation content between the questioner and the respondent when used for consulting services. By intelligently evaluating the call voice, the quality of consulting services can be improved. After the call starts, the question-and-answer voice of the questioner is collected, where the questioner refers to the consultant and the question-and-answer voice refers to the voice transmitted by the questioner through mobile communication. Voice quality refers to the clarity of the question-and-answer voice. Under normal circumstances, there may be unclear pronunciation or unstable communication signals during voice calls, resulting in poor voice quality. Therefore, the question-and-answer voice with poor voice quality needs to be corrected and converted into standard voice, that is, semantically clear voice. Text characters refer to character information converted from voice to text, and question-and-answer text can be generated through a natural language processing model.

[0048] S2: The respondent outputs a response voice based on the question and answer text, obtains a reply text corresponding to the response voice, and stores the call record between the questioner and the respondent to generate a call segment and a call recording.

[0049] Specifically, in this embodiment, the respondent refers to the customer service staff, and all question and answer texts and corresponding reply texts are combined in sequence based on the time series to generate a call segment, and all question and answer voices and corresponding reply voices are combined in sequence to generate a call recording. Among them, the time series refers to the time feature information, the call segment refers to the characters composed of all question and answer texts and reply texts in the call voice in sequence, and the call recording refers to all call voices from the beginning to the end of the call, that is, all question and answer voices and corresponding reply voices. By summarizing the call segments and call recordings in the voice call, a basis can be provided for the subsequent evaluation of the call voice.

[0050] S3: Extract emotional words from the call segments and answer time distribution from the call recordings respectively, generate question-answer scores based on the emotional words, and generate answer scores based on the answer time distribution.

[0051] Specifically, in this embodiment, emotional vocabulary is a vocabulary used to express emotions, including but not limited to relatively negative violation vocabulary, relatively mild common descriptive vocabulary and relatively positive adjective vocabulary. The response time distribution refers to the pause time between the question and answer voice and the answer voice in the call recording, and the corresponding occupied time of the question and answer voice and the answer voice. The emotional vocabulary can be used to describe the emotions of the questioner to generate a question and answer score, and the response time distribution can be used to describe the timeliness of the respondent's response to generate a response score.

[0052] S4: Obtain the question understanding characters corresponding to the question and answer text, build an answer database, retrieve the reference texts corresponding to the question understanding characters in the answer database, fuse all reference texts to generate a summary text, compare the overlap between the answer text and the summary text, and generate a reply score based on the overlap.

[0053] Specifically, in this embodiment, the question understanding character refers to the character generated after semantic understanding and analysis of the question and answer text, the answer database refers to the database for reference answers, the reference text refers to the reference answer of the question and answer understanding character searched in the answer database, and the summary text refers to the summary character generated after merging all the reference texts corresponding to the question understanding character. Simply put, the summary text is a concise and complete reference answer character corresponding to the question and answer text. The overlap degree refers to the degree of character similarity between the answer text and the summary text. By counting all the answer texts in the call record, a reply score can be generated, wherein the reply score is used to evaluate the correctness of the respondent's answer voice.

[0054] S5: Summarize the question and answer scores, the answer scores, and the reply scores to generate a call evaluation value of the call record.

[0055] Specifically, by summarizing the question and answer scores, the response scores, and the reply scores, a call evaluation value can be generated, which can improve the completeness and accuracy of the intelligent evaluation of the call voice.

[0056] As a preferred technical solution of the present invention, converting the question-answering speech into the standard speech based on the speech quality comprises the following steps:

[0057] A speech sample set is collected, which includes a variety of normal speech and ambiguous speech. A speech waveform set corresponding to the speech sample set is obtained, and speech features of all speech waveforms in the speech waveform set are extracted. The differences between normal speech waveforms and ambiguous speech waveforms in the speech waveform set are compared based on the speech features. A speech training model for identifying ambiguous speech waveforms is generated based on the differences. The speech waveform set is input into the speech training model, and the speech training model is pre-trained using a self-supervised learning method to generate a speech conversion model.

[0058] The question-and-answer speech is input into the speech conversion model. The speech conversion model predicts the pitch and intensity corresponding to the question-and-answer speech, generates a spectrogram based on the pitch and intensity, reconstructs the spectrogram to generate a conversion waveform, and generates a converted speech based on the frequency domain conversion method based on the conversion waveform, which is set as the standard speech.

[0059] Specifically, in this embodiment, the speech sample set refers to a speech set of different human voices, normal speech refers to speech with standard and clear pronunciation, fuzzy speech refers to speech with relatively fuzzy pronunciation, and the speech waveform set refers to a set of speech waveforms, wherein the speech waveform refers to the frequency waveform corresponding to each speech in the speech sample set obtained by digital audio processing, and the speech feature refers to a series of parameters or attributes extracted from the speech waveform that can represent the characteristics of the speech, and the speech feature can be extracted by a convolutional neural network. The normal speech waveform refers to the speech waveform corresponding to the normal speech, and the fuzzy speech waveform refers to the speech waveform corresponding to the fuzzy speech. By comparing the speech features corresponding to the normal speech waveform and the fuzzy speech waveform, the difference between the speech features can be generated, and the types of differences include but are not limited to spectrum differences, noise differences, clarity differences, and rhythmic feature differences. The speech training model is a pre-learning model, and a masked language model can be pre-learned through all the differences, that is, the speech training model, and then all the normal speech and fuzzy speech in the speech waveform set are input into the speech training model for training and learning. The self-supervised learning method is a machine learning paradigm that allows the model to learn from unlabeled data, and uses the structure or attributes of the data itself to generate pseudo labels, and then trains the model to predict these pseudo labels. Using a self-supervised learning method, the speech training model can be self-trained by predicting the speech features of the masked part, and the pre-trained speech training model can be set as a speech conversion model.

[0060] The speech conversion model can identify and predict the speech features of ambiguous speech, and then convert the ambiguous speech into normal speech. When the question-answering speech is input into the speech conversion model, a spectrogram can be generated by predicting the pitch and intensity of the question-answering speech. Pitch is related to the frequency of the sound, and intensity refers to the loudness of the sound. Pitch and intensity are both speech features. The spectrogram is a spectrum characteristic diagram obtained by Fourier transforming the pitch and intensity in the sound signal and using a filter bank to extract speech features. It can be used to visualize and compare the spectral features of the question-answering speech. The conversion waveform is a sound waveform generated by the spectrogram through vocoding synthesis or spectrum regeneration. The frequency domain conversion method is a signal processing method that converts the time domain signal into the frequency domain signal to generate the corresponding sound, which is the converted speech. Setting the converted speech as the standard speech can obtain a question-answering speech with higher clarity. The speech conversion model generated by pre-training can effectively identify and predict the spectral features of the question-answering speech, and improve the accuracy and completeness of generating standard speech.

[0061] As a preferred technical solution of the present invention, generating a call evaluation value of a call record includes the following steps:

[0062] Set emotion tags, including joy tags, calm tags and dull tags. Establish an emotion evaluation vocabulary based on the emotion tags. Obtain the semantic similarity between the call paragraph and any phrase in the emotion evaluation vocabulary. Extract the emotion words in the call paragraph based on the semantic similarity. Obtain the emotion tags corresponding to the emotion words and set them as emotion types.

[0063] A coordinate system is constructed based on the emotion type, the number of the same emotion type is accumulated and a score histogram is drawn in the coordinate system, the accumulated value of any emotion label in the score histogram is obtained, the ratio of the sum of all accumulated values ​​to the accumulated value corresponding to the boring label is calculated, and set as the question and answer score.

[0064] Based on the call recording waveform intensity, the answer time distribution is divided into cumulative communication time and cumulative silence time. The ratio between the total answer time distribution and the cumulative silence time is calculated and set as the answer score.

[0065] The answer score F is calculated based on the first formula, which is: Where i is a positive integer from 1 to I, I is the total number of reply texts in the call log, and n i is the number of identical characters between the ith reply text and the corresponding ith summary text, N i is the total number of characters in the i-th summary text, and the question-answer score, answer score, and reply score are accumulated to generate a call evaluation value.

[0066] Specifically, in this embodiment, the emotion label refers to the label information corresponding to various emotion types, wherein the joy label corresponds to characters with more distinct emotions, such as "thank you", "very good" and "thank you", etc.; the calm label corresponds to characters with calm emotions, such as "please", "okay", "hello" and "goodbye", etc.; the dull label is used to represent characters with more depressed emotions, such as "hurry up", "hurry up" and "annoying" as well as banned words, etc. The emotion evaluation word library refers to a database composed of characters with the same or similar meanings corresponding to various emotion labels. The semantic similarity can be calculated by the cosine similarity between word vectors, wherein the word vector maps the phrase into a continuous vector space so that the semantically similar phrases are closer in this space. Therefore, the emotion words with higher semantic similarity in the call paragraph can be obtained by setting a threshold. By obtaining the emotion words and the phrases with higher semantic similarity in the emotion evaluation word library, the emotion label corresponding to the phrase can be set as the emotion type of the emotion word. For example, the semantic similarity between the emotion word "thank you very much" and the character "thank you" in the joy label is 0.9, and the emotion type of the emotion word is the joy label.

[0067] The coordinate system is a histogram coordinate system, where the horizontal axis is the emotion label and the vertical axis is the number of emotion words of the same emotion type. Therefore, a score histogram can be drawn, which can clearly describe the frequency distribution of all emotion words in the call segment. The cumulative value corresponding to the boring label refers to the number of emotion words in the call segment whose emotion type belongs to the boring label. For example, the total number of emotion words in the call segment is 10, of which the number of emotion words belonging to the boring label is 2, then the question and answer score is: 10 / 2=5.

[0068] The cumulative communication time refers to the length of time in the call recording when the amplitude of the waveform intensity changes greatly and voice appears. The cumulative silence time refers to the length of time in the call recording when the amplitude of the waveform intensity is low and the change is small. The response time distribution can be used to obtain the response reaction time of the respondent in the call recording, as well as the question and answer time of the questioner. For example, if the total duration of the response time distribution is 10 and the cumulative silence time is 1, the response score is: 10 / 1=10.

[0069] In the first formula, the overlap between each reply text and the reference text in the call record is summarized. For example, the call record contains 2 reply texts, where the overlap between reply text L1 and the corresponding summary text is (8 / 10) = 0.8, and the overlap between reply text L2 and the corresponding summary text is (2 / 5) = 0.4. The reply score is (0.8 + 0.4) = 1.2. Therefore, the call evaluation value is: 5 + 10 + 1.2 = 16.2. The higher the call evaluation value, the higher the effectiveness of the call voice.

[0070] As a preferred technical solution of the present invention, obtaining the question understanding characters corresponding to the question and answer text includes the following steps:

[0071] Convert the question and answer text into a standard form text, build a knowledge dictionary, compare the standard form text with the semantic text stored in the knowledge dictionary, identify the object identifier and relationship identifier in the standard form text, set the relationship identifier as the root node, set the object identifier as the child node, connect the child node corresponding to the object identifier to the root node corresponding to the associated relationship identifier, generate a tree structure based on the root node and the child node, and repeat this step until all root nodes and all child nodes are included in the tree structure.

[0072] Based on the arrangement order of the relationship identifiers in the standard form text, the identifiers corresponding to all nodes in the tree structure are combined in sequence to generate question understanding characters, wherein the nodes include root nodes and child nodes, and the identifiers include thing identifiers and relationship identifiers.

[0073] Specifically, in this embodiment, the standard form text refers to the characters of the preset form generated after adjusting or supplementing the word order of the question and answer text. For example, the preset form is "subject + object + predicate", then the question and answer text "How to use A application software" is converted into the standard form text "What is your method of using A application software", and the specific conversion method is described later. Therefore, the question and answer text can be made more complete through the standard form text. The knowledge dictionary refers to a dictionary set used to describe the part-of-speech features of various phrases, and is used to store the semantic texts of multiple phrases, wherein the semantic text refers to the meaning interpretation of the phrase, for example, the part-of-speech feature of "method" is "noun", and the corresponding semantic text refers to "the way or method to solve the problem". Both the thing identifier and the relationship identifier are elements used for semantic text. The thing identifier represents a specific concept or entity and is unique; the relationship identifier is used to describe the relationship or attribute between two thing identifiers. For example, in the standard form text "What is your method of using A application software", the thing identifiers are: "you", "A application software" and "method", and the relationship identifiers are: "use" and "is (what)". The tree structure is composed of a root node and each child node. The tree structure is constructed using object identifiers and relationship identifiers. Each object identifier is a node, and the relationship identifier describes and defines the connection between each node. Repeat this step to include all object identifiers and relationship identifiers in the standard form text into the tree structure by setting child nodes or root nodes.

[0074] like Figure 2As shown, the tree structure is composed of multiple root nodes and child nodes in sequence. The child nodes at the bottom of the tree structure are connected to the root node in sequence, and the child nodes and root nodes at other layers are combined in sequence to generate question understanding characters. For example, the arrangement order is combined in sequence: "you" → "use" → "A application software" → "is (what)" → "method", so the question understanding character corresponding to the question and answer text "How to use A application software" is "what is your method of using A application software".

[0075] As a preferred technical solution of the present invention, converting the question and answer text into a standard form text comprises the following steps:

[0076] Set the standard word structure of the sentence, obtain the phrases and corresponding word types contained in the question and answer text, obtain the missing word types of the word types based on the standard word structure of the sentence, complete the characters corresponding to the missing word types based on the knowledge dictionary and semantic matching method, and define them as supplementary phrases, sort and combine the phrases and supplementary phrases based on the standard word structure of the sentence to generate a completed sentence, and set the completed sentence as a standard form text.

[0077] Specifically, in this embodiment, the standard word structure of a sentence is a set of rules for constructing and organizing language elements (such as vocabulary, phrases, and sentences), which determines how to convert a question-and-answer text into a semantic text and how to express complex semantic relationships in a semantic text. For example, the syntactic structure in the standard word structure of a sentence can be set to "subject + object + predicate" corresponding to a preset form. The word type refers to the type of component of a phrase, including but not limited to subject, predicate, and object, etc. The phrase refers to multiple words split in the question-and-answer text. The missing word type refers to the missing word type when the word type is compared with the standard word structure of the sentence. The semantic matching method refers to matching the components in the question-and-answer text with the data in the knowledge dictionary to determine the missing word type, and generating a phrase in the knowledge dictionary to complete the question-and-answer text, that is, a supplementary phrase, wherein the semantic matching method is implemented by a machine learning algorithm, for example, using the BERT model (Bidirectional Encoder Representations from Transformers) to set the semantic matching method. The completed sentence refers to a text with the same word type as the standard word structure of the sentence, and the most semantically reasonable text with the question-and-answer text. Therefore, the completion statement is set to the standard form text.

[0078] As a preferred technical solution of the present invention, searching the reference text corresponding to the question understanding character in the answer database includes:

[0079] The answer database includes question and answer information and answer information. The character similarity between any identifier in the question understanding character and the question and answer information is obtained respectively, and the question and answer information with character similarity greater than or equal to a first preset value is extracted and set as the key question phrase, and the answer information corresponding to the key question phrase in the answer database is obtained. All the answer information is combined and set as the reference text.

[0080] Specifically, in this embodiment, the question and answer information refers to the label information of the question subject, and the answer information refers to the answer text corresponding to the question and answer information. The question and answer information and the answer information form an answer database. The character similarity is used to compare the similarity between two groups of characters. The character similarity P can be calculated by the second formula, which is: Wherein, m is the number of identical characters between any identifier and the question and answer information, and M is the total number of characters in any identifier. For example, if the identifier is "A application software" and the question and answer information is "application software", the character similarity is (4 / 5) = 0.8. The first preset value is a pre-set value. Question and answer information greater than or equal to the first preset value is set as a key question phrase, indicating that the question and answer information is an alternative reference question, and the corresponding answer information is an alternative reference answer. Combining all the answer information can generate a complete reference text. The reference text contains all possible answer information of the question and answer text. At the same time, there may be broader and less concise answer information. Therefore, the reference text needs to be fused and processed to generate a summary text.

[0081] As a preferred technical solution of the present invention, all reference texts are integrated to generate summary text:

[0082] The question understanding character is set as the query text, all reference texts are sequentially combined based on the arrangement order of any identifier in the question understanding character and set as the source text, and the query text and the source text are respectively split into multiple word sequences.

[0083] The binary cross entropy function is set as the objective function and a summary generation model is constructed. The summary generation model estimates the importance of the word sequence of the source text based on the word sequence of the query text, and outputs the predicted value of the word sequence in the source text based on the importance. The word sequence with a predicted value greater than or equal to a second preset value is set as the summary sequence, and the summary character length is set. The summary generation model combines all summary sequences based on the summary character length to generate a summary text.

[0084] Specifically, in this embodiment, first, the corresponding reference texts are combined in sequence according to the arrangement order of each identifier in the question understanding character to generate multiple source texts in the form of sentences. The word sequence represents a sequence of characters corresponding to a phrase or word and having position features.

[0085] The binary cross entropy function is a loss function commonly used in classification problems, especially in binary classification problems. When constructing a summary generation model, the binary cross entropy can be used as the objective function to train the model, which can measure the difference between the summary generated by the model and the real summary, and minimize the difference between the summary generated by the model and the real summary. The summary generation model is a neural network model constructed by the objective function, which outputs a one-dimensional word selection score for each word sequence in the source text, and sets the word selection score as the importance of the word sequence. It is also used to predict the probability of each word sequence in the source text being included in the summary, and sets the probability as the predicted value of the corresponding word sequence. Extracting a word sequence whose predicted value is greater than or equal to the second preset value can output the corresponding summary sequence, reducing unnecessary word sequences. The summary character length refers to the total number of characters in the preset summary text. The summary text can be simplified by setting the summary character length.

[0086] As a preferred technical solution of the present invention, it is determined whether the question and answer text belongs to the first question and answer based on the time series. If not, the semantic relevance between the question and answer text and its previous question and answer text is calculated. If the semantic relevance is greater than or equal to a third preset value, the question and answer text and the summary text corresponding to the previous question and answer text are fused to generate a new question and answer text.

[0087] Specifically, in this embodiment, after the call starts, the question-and-answer voice and the answer voice are recorded in sequence, and the first question-and-answer text of the questioner is set as the first question-and-answer. The semantic relevance between the current question-and-answer text and the previous question-and-answer text is determined, wherein the semantic relevance refers to the calculation of the character similarity. If the semantic relevance of two adjacent question-and-answer texts is greater than or equal to the third preset value, it means that the question-and-answer text has a high relevance to the previous question-and-answer text. Therefore, the summary text corresponding to the previous question-and-answer text can be merged with the question-and-answer text to generate a new question-and-answer text, which is conducive to completing the question-and-answer text. For example, the question-and-answer text is: "What about B application software?" The corresponding previous question-and-answer text is: "How to use A application software?" By merging the summary text corresponding to the previous question-and-answer text, the question-and-answer text "What about B application software" can be modified into a new question-and-answer text "How to use B application software". The relevance of the question-and-answer voice in the voice call can be achieved, and the call effectiveness can be improved.

[0088] like Figure 3 As shown, the present invention also provides an intelligent evaluation system for call voice, which is used to implement the above-mentioned intelligent evaluation method for call voice, and the system mainly includes:

[0089] The voice acquisition module acquires the question-and-answer voice of the questioner after the call starts, identifies the voice quality of the question-and-answer voice, converts the question-and-answer voice into a standard voice based on the voice quality, extracts the text characters of the standard voice and sets them as the question-and-answer text.

[0090] The statistical module is used for the respondent to output a response voice based on the question and answer text, obtain the reply text corresponding to the response voice, and store the call records between the questioner and the respondent to generate call segments and call recordings.

[0091] The analysis module extracts the emotional words in the call paragraphs and the answering time distribution in the call recordings, generates question-answering scores based on the emotional words, and generates answering scores based on the answering time distribution.

[0092] The evaluation output module obtains the question understanding characters corresponding to the question and answer text, builds an answer database, retrieves the reference text corresponding to the question understanding characters in the answer database, fuses all the reference texts to generate a summary text, compares the overlap between the reply text and the summary text, and generates a reply score based on the overlap; summarizes the question and answer scores, answer scores, and reply scores to generate a call evaluation value for the call record.

[0093] The present invention also provides a computer storage medium, which stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the above-mentioned intelligent evaluation method for call voice.

[0094] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0095] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0096] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0098] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent evaluation method for call speech, characterized in that: The method comprises the following steps: S1: acquiring the question-and-answer voice of the questioner after the call starts, identifying the voice quality of the question-and-answer voice, converting the question-and-answer voice into a standard voice based on the voice quality, extracting text characters of the standard voice and setting them as the question-and-answer text; Wherein, converting the question-answering speech into a standard speech based on the speech quality comprises the following steps: Collect a speech sample set, the speech sample set including a plurality of normal speech and ambiguous speech, obtain a speech waveform set corresponding to the speech sample set, extract speech features of all speech waveforms in the speech waveform set, compare differences between normal speech waveforms and ambiguous speech waveforms in the speech waveform set based on the speech features, generate a speech training model for identifying the ambiguous speech waveform based on the differences, input the speech waveform set into the speech training model, and pre-train the speech training model using a self-supervised learning method to generate a speech conversion model; The question-and-answer speech is input into the speech conversion model, the speech conversion model predicts the pitch and intensity corresponding to the question-and-answer speech, generates a spectrogram based on the pitch and the intensity, reconstructs the spectrogram to generate a conversion waveform, and generates a conversion speech based on a frequency domain conversion method using the conversion waveform, and sets the converted speech as the standard speech; S2: The respondent outputs a response voice based on the question and answer text, obtains a reply text corresponding to the response voice, and stores the call record between the questioner and the respondent to generate a call segment and a call recording; S3: extracting emotional words in the call segment and answering time distribution in the call recording respectively, generating a question-answering score based on the emotional words, and generating an answering score based on the answering time distribution; S4: Obtain the question understanding characters corresponding to the question and answer text, build an answer database, retrieve the reference texts corresponding to the question understanding characters in the answer database, merge all the reference texts to generate a summary text, compare the overlap between the answer text and the summary text, and generate a reply score based on the overlap; S5: Summarize the question and answer score, the response score, and the reply score to generate a call evaluation value of the call record.

2. The method according to claim 1, characterized in that Generating the call evaluation value of the call record comprises the following steps: Setting emotion tags, wherein the emotion tags include a joy tag, a calm tag, and a dull tag, establishing an emotion evaluation vocabulary based on the emotion tags, respectively obtaining the semantic similarity between the call segment and any phrase in the emotion evaluation vocabulary, extracting the emotion vocabulary in the call segment based on the semantic similarity, obtaining the emotion tag corresponding to the emotion vocabulary and setting it as the emotion type; Constructing a coordinate system based on the emotion type, accumulating the number of the same emotion type and drawing a score histogram in the coordinate system, obtaining the cumulative value of any emotion label in the score histogram, calculating the ratio of the sum of all the cumulative values ​​to the cumulative value corresponding to the boring label, and setting it as the question and answer score; Dividing the response time distribution into cumulative communication duration and cumulative silence duration based on the call recording waveform strength, calculating the ratio between the total duration of the response time distribution and the cumulative silence duration, and setting it as the response score; The answer score F is calculated based on a first formula, which is: Where i is a positive integer from 1 to I, I is the total number of reply texts in the call record, and n i is the number of identical characters between the ith reply text and the corresponding ith summary text, N i is the total number of characters in the i-th summary text, and the question and answer score, the answer score and the reply score are accumulated to generate the call evaluation value.

3. The method according to claim 1, characterized in that The step of obtaining the question understanding characters corresponding to the question and answer text comprises the following steps: Convert the question-and-answer text into a standard form text, construct a knowledge dictionary, compare the standard form text with the semantic text stored in the knowledge dictionary, identify the object identifier and the relationship identifier in the standard form text, set the relationship identifier as a root node, set the object identifier as a child node, connect the child node corresponding to the object identifier to the root node corresponding to the associated relationship identifier, generate a tree structure based on the root node and the child node, and repeat this step until all root nodes and all child nodes are included in the tree structure; The question understanding character is generated by sequentially combining the identifiers corresponding to all nodes in the tree structure based on the arrangement order of the relationship identifiers in the standard form text, wherein the nodes include the root node and the child nodes, and the identifiers include the thing identifier and the relationship identifier.

4. The method according to claim 3, characterized in that The step of converting the question and answer text into a standard form text comprises the following steps: A standard word structure for a sentence is set, and the phrases and corresponding word types contained in the question and answer text are obtained. Based on the standard word structure for the sentence, the missing word type of the word type is obtained. Based on the knowledge dictionary and the semantic matching method, the characters corresponding to the missing word type are completed and defined as a supplementary phrase. Based on the standard word structure for the sentence, the phrases and the supplementary phrases are sorted and combined to generate a completed sentence, and the completed sentence is set as the standard form text.

5. The method according to claim 3, characterized in that: Retrieving the reference text corresponding to the question understanding character in the answer database includes: The answer database includes question and answer information and answer information, and the character similarity between any identifier in the question understanding character and the question and answer information is obtained respectively, and the question and answer information whose character similarity is greater than or equal to a first preset value is extracted and set as a key question phrase, and the answer information corresponding to the key question phrase in the answer database is obtained, and all the answer information is combined and set as the reference text.

6. The method according to claim 5, characterized in that The method of fusing all reference texts to generate summary text comprises the following steps: The question understanding character is set as a query text, all the reference texts are sequentially combined based on the arrangement order of any of the identifiers in the question understanding character, and are set as source texts, and the query text and the source text are respectively split into a plurality of word sequences; A binary cross entropy function is set as an objective function and a summary generation model is constructed. The summary generation model estimates the importance of the word sequence of the source text based on the word sequence of the query text, and outputs a predicted value of the word sequence in the source text based on the importance, sets the word sequence whose predicted value is greater than or equal to a second preset value as a summary sequence, sets a summary character length, and the summary generation model combines all the summary sequences based on the summary character length to generate the summary text.

7. The method according to claim 1, characterized in that Based on the time series, determine whether the question and answer text belongs to the first question and answer. If not, calculate the semantic relevance between the question and answer text and its previous question and answer text. If the semantic relevance is greater than or equal to a third preset value, merge the question and answer text with the summary text corresponding to its previous question and answer text to generate a new question and answer text.

8. An intelligent evaluation system for call speech, used to implement the method according to any one of claims 1 to 7, characterized in that: The system includes the following modules: A speech acquisition module is provided for acquiring the question-and-answer speech of the questioner after the call starts, identifying the speech quality of the question-and-answer speech, converting the question-and-answer speech into a standard speech based on the speech quality, extracting text characters of the standard speech and setting them as the question-and-answer text; collecting a speech sample set, the speech sample set including a plurality of normal speech and fuzzy speech, acquiring a speech waveform set corresponding to the speech sample set, extracting speech features of all speech waveforms in the speech waveform set, comparing the difference between the normal speech waveform and the fuzzy speech waveform in the speech waveform set based on the speech features, generating a speech training model for identifying the fuzzy speech waveform based on the difference, inputting the speech waveform set into the speech training model, and pre-training the speech training model using a self-supervised learning method to generate a speech conversion model; inputting the question-and-answer speech into the speech conversion model, the speech conversion model predicting the pitch and intensity corresponding to the question-and-answer speech, generating a spectrogram based on the pitch and the intensity, reconstructing the spectrogram to generate a conversion waveform, generating a conversion speech based on a frequency domain conversion method, and setting the conversion waveform as the standard speech; A statistics module, in which the respondent outputs a response voice based on the question and answer text, obtains a reply text corresponding to the response voice, and stores the call records between the questioner and the respondent to generate call segments and call recordings; An analysis module, which extracts the emotional words in the call segment and the answering time distribution in the call recording, generates a question-answering score based on the emotional words, and generates an answering score based on the answering time distribution; An evaluation output module obtains the question understanding characters corresponding to the question and answer text, constructs an answer database, retrieves the reference text corresponding to the question understanding characters in the answer database, fuses all reference texts to generate a summary text, compares the overlap between the reply text and the summary text, and generates a reply score based on the overlap; summarizes the question and answer score, the answer score and the reply score to generate a call evaluation value for the call record.

9. A computer storage medium, characterized in that: The computer storage medium stores program instructions, wherein when the program instructions are executed, the device where the computer storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images