Method and device for evaluating sales dialogue response content and electronic equipment
By automating the analysis of sales dialogue audio data using large language models, identifying speakers and matching tag rules, the inefficiency of traditional evaluation methods is solved, achieving efficient and accurate sales dialogue evaluation, improving the quality of the sales process and reducing resource consumption.
Patent Information
- Application Number
- CN202410684262.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-30
- Publication Date
- 2025-12-30
AI Technical Summary
Traditional sales conversation evaluation methods rely on manual listening, which is inefficient and makes it difficult to guarantee the accuracy and consistency of the evaluation results, especially when the customer and salesperson speak at the same time, making it difficult to accurately identify the speaker's identity.
The system uses a large language model to automatically analyze sales conversation audio data, acquire audio data in real time, identify the speaker, convert it into text labeled with the speaker's identity, analyze customer focus characteristics, and evaluate it by matching tag rules.
It improves the efficiency and accuracy of assessments, reduces human intervention, enhances the quality of the sales process and user experience, reduces energy and resource consumption, and promotes environmental protection.
Smart Images
Figure CN121237117A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer technology, in particular to a method and device for evaluating response content of a sales dialogue and electronic equipment. BACKGROUND
[0002] In the field of modern automobile sales, especially in 4S stores, the importance of sales dialogue in business is increasingly prominent. The success or failure of sales dialogue is directly related to the performance of the enterprise and the satisfaction of customers. However, traditional sales dialogue evaluation methods often rely on manual monitoring and analysis, which not only is inefficient, but also cannot guarantee the accuracy and consistency of the evaluation results.
[0003] In addition, sales dialogue usually involves multiple rounds of interaction, with the speeches of customers and salespersons appearing alternately, which brings greater challenges to the evaluation of sales dialogue. Traditional evaluation methods often cannot accurately identify the speaker's identity, resulting in biased evaluation results.
[0004] Therefore, a method and device for evaluating response content of a sales dialogue and electronic equipment are proposed. SUMMARY
[0005] The present application provides a method and device for evaluating response content of a sales dialogue and electronic equipment, which greatly improves the evaluation efficiency through automatic analysis of sales dialogue by a large language model.
[0006] The present application provides a method for evaluating response content of a sales dialogue, comprising:
[0007] real-time acquisition of inquiry process audio data;
[0008] analysis of the inquiry process audio data by a large language model to obtain a sales dialogue response evaluation result, comprising:
[0009] identification of the inquiry process audio data to determine the speaker's identity; the speaker's identity includes salesperson and customer;
[0010] conversion of the inquiry process audio data into inquiry process text labeled with the speaker's identity in combination with the speaker's identity;
[0011] analysis of the inquiry process text to obtain customer attention features, and matching of corresponding label rules based on the customer attention features;
[0012] matching of the latest salesperson's response data with the label rules to obtain a sales dialogue response evaluation result.
[0013] Optionally, the identification of the inquiry process audio data to determine the speaker's identity comprises:
[0014] Voiceprint features are extracted from the audio data of the inquiry process, and the voiceprint features are matched with the voiceprint features in the employee database to determine the voiceprint features corresponding to the salesperson.
[0015] The remaining voiceprint features are categorized to obtain the voiceprint features corresponding to the customer.
[0016] Optionally, the step of identifying the speaker's identity by recognizing the audio data of the inquiry process includes:
[0017] Keyword extraction is performed on the audio data of the inquiry process, and the frequency of their occurrence is recorded;
[0018] The speaker's identity is determined based on the keywords and their frequency of occurrence.
[0019] Optionally, the step of converting the audio data of the inquiry process into text labeled with the speaker's identity, in conjunction with the speaker's identity, includes:
[0020] The audio data of the inquiry process is subjected to language recognition using Bayes' theorem to obtain the text of the inquiry process, including:
[0021] P(W|O)={\frac{P(O|W)P(W)}{P(O)}
[0022] Wherein, P(W|O) is the conditional probability of the occurrence of the known acoustic signal sequence (O) and word sequence (W) in the audio data of the inquiry process; P(O|W) is the conditional probability of the occurrence of the known word sequence (W) and acoustic signal sequence (O) in the audio data of the inquiry process; P(W) is the prior probability of the word sequence (W), that is, the probability of generating the word sequence (W) when no acoustic signal is observed; P(O) is the marginal probability of the observed acoustic signal sequence (O), also known as evidence; {\frac{P(O|W)P(W)} is the posterior probability of the word sequence (W) under the condition that the acoustic signal (O) is observed, obtained by Bayes' theorem.
[0023] By combining the speaker's identity with the query process text, a query process text labeled with the speaker's identity is obtained.
[0024] Optionally, the analysis of the inquiry process text to obtain customer attention characteristics includes:
[0025] The query process text is segmented and tagged with parts of speech.
[0026] Based on a pre-set keyword library and / or contextual semantic analysis, identify the keywords and / or key phrases in the customer's question;
[0027] Based on the weight and inverse document frequency of the keywords, and / or the weight and inverse document frequency of the key phrases, the attention ranking of the keywords and / or the key phrases is obtained, and the top N keywords and / or the key phrases are taken as customer attention features.
[0028] Optionally, the process of obtaining the weight of the keyword includes:
[0029] The keywords are input into the embedding layer to obtain the keyword vector;
[0030] The keyword vector is transformed into query vector, key vector, and value vector using different weight matrices.
[0031] An attention score is obtained based on the query vector and the key vector, and the attention score is converted into keyword weights using the softmax function.
[0032] Optionally, obtaining the inverse document frequency of the keyword includes:
[0033] IDF(w) = log(N / n(w))
[0034] Wherein, IDF(w) is the inverse document frequency of the keyword, w is the keyword, N is the total number of words in the query process text, and n(w) is the number of keywords.
[0035] This specification provides an apparatus for evaluating the content of responses in a sales conversation, comprising:
[0036] The acquisition module is used to acquire audio data of the inquiry process in real time;
[0037] The analysis module is used to analyze the audio data of the inquiry process using a large language model to obtain sales dialogue response evaluation results, including:
[0038] The identification unit is used to identify the audio data of the inquiry process and determine the speaker's identity; the speaker's identity includes salesperson and customer;
[0039] A conversion unit is used to convert the audio data of the inquiry process into text of the inquiry process labeled with the speaker's identity, based on the speaker's identity.
[0040] The analysis unit is used to analyze the query process text, obtain customer attention features, and match corresponding tag rules based on the customer attention features;
[0041] The evaluation unit is used to match the latest salesperson's response data with the labeling rules to obtain the sales dialogue response evaluation result.
[0042] Optionally, the identification unit includes:
[0043] Voiceprint features are extracted from the audio data of the inquiry process, and the voiceprint features are matched with the voiceprint features in the employee database to determine the voiceprint features corresponding to the salesperson.
[0044] The remaining voiceprint features are categorized to obtain the voiceprint features corresponding to the customer.
[0045] Optionally, the identification unit includes:
[0046] Keyword extraction is performed on the audio data of the inquiry process, and the frequency of their occurrence is recorded;
[0047] The speaker's identity is determined based on the keywords and their frequency of occurrence.
[0048] Optionally, the conversion unit includes:
[0049] The audio data of the inquiry process is subjected to language recognition using Bayes' theorem to obtain the text of the inquiry process, including:
[0050] P(W|O)={\frac{P(O|W)P(W)}{P(O)}
[0051] Wherein, P(W|O) is the conditional probability of the occurrence of the known acoustic signal sequence (O) and word sequence (W) in the audio data of the inquiry process; P(O|W) is the conditional probability of the occurrence of the known word sequence (W) and acoustic signal sequence (O) in the audio data of the inquiry process; P(W) is the prior probability of the word sequence (W), that is, the probability of generating the word sequence (W) when no acoustic signal is observed; P(O) is the marginal probability of the observed acoustic signal sequence (O), also known as evidence; {\frac{P(O|W)P(W)} is the posterior probability of the word sequence (W) under the condition that the acoustic signal (O) is observed, obtained by Bayes' theorem.
[0052] By combining the speaker's identity with the query process text, a query process text labeled with the speaker's identity is obtained.
[0053] Optionally, the analysis unit includes:
[0054] The query process text is segmented and tagged with parts of speech.
[0055] Based on a pre-set keyword library and / or contextual semantic analysis, identify the keywords and / or key phrases in the customer's question;
[0056] Based on the weight and inverse document frequency of the keywords, and / or the weight and inverse document frequency of the key phrases, the attention ranking of the keywords and / or the key phrases is obtained, and the top N keywords and / or the key phrases are taken as customer attention features.
[0057] Optionally, the process of obtaining the weight of the keyword includes:
[0058] The keywords are input into the embedding layer to obtain the keyword vector;
[0059] The keyword vector is transformed into query vector, key vector, and value vector using different weight matrices.
[0060] An attention score is obtained based on the query vector and the key vector, and the attention score is converted into keyword weights using the softmax function.
[0061] Optionally, obtaining the inverse document frequency of the keyword includes:
[0062] IDF(w) = log(N / n(w))
[0063] Wherein, IDF(w) is the inverse document frequency of the keyword, w is the keyword, N is the total number of words in the query process text, and n(w) is the number of keywords.
[0064] This specification also provides an electronic device, wherein the electronic device includes:
[0065] Processor; and,
[0066] A memory that stores computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.
[0067] This specification also provides a computer-readable storage medium that stores one or more programs that, when executed by a processor, implement any of the methods described above.
[0068] This invention significantly reduces the need for manual intervention by automating the generation and updating of labeling rules, greatly improving processing and analysis speed. It responds in real-time to dynamic changes in sales conversations, rapidly adjusting labeling rules and effectively improving work efficiency. An advanced large language model can deeply understand the complex semantics of sales conversations, generating more accurate labeling rules. This improves label accuracy, thereby enhancing the overall quality of sales process verification and reducing the cost of subsequent error handling and correction. The system operates automatically, requiring only minimal configuration from the user, greatly reducing operational complexity and difficulty. It reduces manual intervention, allowing users to quickly complete basic configurations through a simple interface, improving user experience and satisfaction. High processing efficiency and accuracy reduce system energy consumption and minimize resources wasted due to error handling. This indirectly promotes energy and raw material conservation, contributing positively to environmental protection. Digitalization and automation reduce paper usage and the need for physical equipment, lowering environmental resource consumption and potential pollution. As part of enterprise digital transformation, it helps reduce carbon footprint and contributes to environmental protection. Attached Figure Description
[0069] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0070] Figure 1 A schematic diagram illustrating the principle of a method for evaluating the content of a sales dialogue response, provided as an embodiment of this specification;
[0071] Figure 2 A schematic diagram illustrating the principle of step S120 provided in the embodiments of this specification;
[0072] Figure 3 A schematic diagram of the structure of an apparatus for evaluating the content of sales dialogue responses, provided as an embodiment of this specification;
[0073] Figure 4 A schematic diagram of the structure of the analysis module 20 provided in the embodiments of this specification;
[0074] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification;
[0075] Figure 6 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification. Detailed Implementation
[0076] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.
[0077] The following is in conjunction with the appendix Figures 1-6 Exemplary embodiments of the invention will be described more fully here. However, exemplary embodiments can be implemented in many forms and should not be construed as limiting the invention to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention more comprehensive and complete, and to facilitate a full communication of the inventive concept to those skilled in the art. The same reference numerals in the figures denote the same or similar elements, components, or parts, and therefore repeated descriptions of them are omitted.
[0078] Subject to the technical concept of this invention, the features, structures, characteristics or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.
[0079] In the description of specific embodiments, the features, structures, characteristics, or other details described in this invention are intended to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can practice the technical solutions of this invention without one or more of the specific features, structures, characteristics, or other details.
[0080] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0081] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0082] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.
[0083] Figure 1 This is a schematic diagram illustrating the principle of a method for evaluating the content of a sales dialogue response, provided in an embodiment of this specification. The method may include:
[0084] S110: Real-time acquisition of audio data during the inquiry process;
[0085] In the specific implementation described in this specification, in modern sales environments, to improve service quality and efficiency, many companies have begun equipping their sales personnel with advanced name tags that have built-in microphones. These name tags not only possess traditional identification functions but also integrate high-tech audio recording systems capable of capturing and recording conversations between sales personnel and customers in real time. This audio data is automatically transmitted to the company's server for storage and analysis.
[0086] The company places great importance on data security when collecting and processing audio data. All audio data is encrypted and stored on secure servers. Only authorized personnel can access this data, ensuring that customer privacy is not compromised.
[0087] Of course, authorization from the customer is required before starting to record employee ID cards.
[0088] S120: Analyze the audio data of the inquiry process using a large language model to obtain the sales dialogue response evaluation results.
[0089] In the specific implementation of this specification, the large language model includes the AutoTagger model.
[0090] Figure 2 This is a schematic diagram of the principle of the method of step S120 provided in the embodiments of this specification. The method may include:
[0091] S121: Identify the audio data of the inquiry process to determine the speaker's identity; the speaker's identity includes salesperson and customer;
[0092] Optionally, S121 includes:
[0093] Voiceprint features are extracted from the audio data of the inquiry process, and the voiceprint features are matched with the voiceprint features in the employee database to determine the voiceprint features corresponding to the salesperson.
[0094] The remaining voiceprint features are categorized to obtain the voiceprint features corresponding to the customer.
[0095] In the specific implementation of this specification, advanced voiceprint recognition technology is used to extract voiceprint features from the collected audio data. Voiceprint features typically include characteristics that represent the speaker's unique voice, such as the spectrum, fundamental frequency, and formants.
[0096] The extracted voiceprint features are matched against voiceprint features in the employee database. The employee database should pre-contain voiceprint feature data of sales personnel, which may be collected during onboarding or periodically.
[0097] If the conversation involves multiple customers, further analysis and classification may be needed to determine the voiceprint characteristics of each customer. This may require more complex algorithms and techniques, such as multi-speaker separation and recognition.
[0098] Optionally, S121 includes:
[0099] Keyword extraction is performed on the audio data of the inquiry process, and the frequency of their occurrence is recorded;
[0100] The speaker's identity is determined based on the keywords and their frequency of occurrence.
[0101] In the specific implementation of this specification, natural language processing (NLP) techniques, such as text mining or speech recognition (if the audio data has been converted to text), are used to extract keywords from the audio data of the customer inquiry process. These keywords may include product names, service types, prices, offers, questions, and other words directly related to the conversation between the customer and the salesperson.
[0102] Calculate the frequency of each keyword in the audio data. This can help understand which topics or issues are more prominent in the conversation.
[0103] Salespeople typically use specific vocabulary in conversations, such as product descriptions, price explanations, promotional information, and purchase suggestions; these terms may appear frequently. Customers, on the other hand, are more likely to ask questions about product details, prices, comparisons, services, and any doubts they may have, and these terms may also appear frequently in the conversation.
[0104] As mentioned earlier, voiceprint recognition is a more direct and accurate method for identifying speakers. It can be combined with keyword extraction and frequency analysis to provide even more accurate results. Of course, analyzing turn-taking in a conversation—that is, who is speaking and who is listening—can also help determine the speaker's identity.
[0105] S122: Combine the speaker's identity to convert the audio data of the inquiry process into text of the inquiry process labeled with the speaker's identity;
[0106] Optionally, S122 includes:
[0107] The audio data of the inquiry process is subjected to language recognition using Bayes' theorem to obtain the text of the inquiry process, including:
[0108] P(W|O)={\frac{P(O|W)P(W)}{P(O)}
[0109] Wherein, P(W|O) is the conditional probability of the occurrence of the known acoustic signal sequence (O) and word sequence (W) in the audio data of the inquiry process; P(O|W) is the conditional probability of the occurrence of the known word sequence (W) and acoustic signal sequence (O) in the audio data of the inquiry process; P(W) is the prior probability of the word sequence (W), that is, the probability of generating the word sequence (W) when no acoustic signal is observed; P(O) is the marginal probability of the observed acoustic signal sequence (O), also known as evidence; {\frac{P(O|W)P(W)} is the posterior probability of the word sequence (W) under the condition that the acoustic signal (O) is observed, obtained by Bayes' theorem.
[0110] By combining the speaker's identity with the query process text, a query process text labeled with the speaker's identity is obtained.
[0111] In the specific implementation of this specification, [P(W|O)] is the probability ultimately required for speech recognition, because the goal of speech recognition is to identify the correct text information (word sequence) given an acoustic signal. (P(O|W)) is a model that predicts the acoustic representation of a given word sequence, and is often used to evaluate the correctness of acoustic signal matching with text when training a speech recognition system.
[0112] Prior probabilities are typically estimated using language models and reflect the frequency of certain word sequences in natural language. For example, in everyday conversation, "I want to buy a car" is more likely to occur than "a car wants to buy me".
[0113] P(O) represents the overall probability of observing the acoustic signal (O) across all possible word sequences. In speech recognition, this value is used to normalize the final probability. Although (P(O)) itself is a constant, it ensures that the calculation of (P(W|O)) results in an efficient probability distribution.
[0114] Text annotation involves clearly identifying which speaker (salesperson or customer) said each part of the text. This can be achieved by adding specific tags or markers to the text, such as using different colors, fonts, or parentheses to distinguish between the salesperson's and customer's remarks.
[0115] Output the approved annotation text in the required format (such as TXT, CSV, JSON, etc.) and store it in an appropriate location for later use.
[0116] S123: Analyze the inquiry process text to obtain customer attention features, and match corresponding tag rules based on the customer attention features;
[0117] Optionally, S123 includes:
[0118] The query process text is segmented and tagged with parts of speech.
[0119] Based on a pre-set keyword library and / or contextual semantic analysis, identify the keywords and / or key phrases in the customer's question;
[0120] Based on the weight and inverse document frequency of the keywords, and / or the weight and inverse document frequency of the key phrases, the attention ranking of the keywords and / or the key phrases is obtained, and the top N keywords and / or the key phrases are taken as customer attention features.
[0121] In the specific implementation of this specification, a Natural Language Processing (NLP) tool, such as the jieba word segmentation tool, is used to segment the query process text. The segmentation results are separated by words and spaces to form the segmented text. Simultaneously with word segmentation, the part-of-speech tagging function of jieba or other NLP tools is used to tag each word with its part of speech.
[0122] Construct a pre-defined keyword library containing common words and phrases related to customer questions. Use string matching algorithms (such as regular expressions, fuzzy matching, etc.) to search for keywords in the keyword library within the segmented text. For words not found in the keyword library, contextual semantic analysis (such as deep learning-based text representation models) can be used to identify key phrases. This typically involves calculating the semantic similarity of words or phrases to identify question-related phrases. Assign weights to each identified keyword and key phrase. Weights can be determined based on factors such as the importance of the keyword in the library and its frequency of occurrence in historical data. Calculate the Inverse Document Frequency (IDF) for each keyword and key phrase. IDF measures the rarity of a word in a document set; generally, the higher the IDF value, the more important the word. Calculate the attention score for each keyword / key phrase based on its weight and IDF value. The score can be the product of the weight and IDF or other combinations thereof. Sort all keyword / key phrase attention scores to obtain an attention ranking. Based on the attention ranking, select the top N keywords and / or key phrases with the highest scores as customer attention features. The specific value of N can be set according to the actual application scenario and requirements.
[0123] Optionally, the process of obtaining the weight of the keyword includes:
[0124] The keywords are input into the embedding layer to obtain the keyword vector;
[0125] The keyword vector is transformed into query vector, key vector, and value vector using different weight matrices.
[0126] An attention score is obtained based on the query vector and the key vector, and the attention score is converted into keyword weights using the softmax function.
[0127] In the specific implementation of this specification, [\text{Attention}(Q, K, V)=\text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V]:
[0128] Where Q is the query matrix, K is the key matrix, V is the value matrix, and (d_k) is the dimension of the key vector. `Attention(Q, K, V)` is a function that encodes the correlations between different positions in the input sequence and outputs a new sequence in the form of a weighted sum. This sequence is typically used in subsequent processing layers or prediction tasks. In the context of self-attention, the output at each position depends not only on its own input features but also on the input features of the entire sequence.
[0129] (QK^T) multiplies the query matrix (Q) and the transpose (K^T) of the key matrix (K). The result is a tensor where each element (i, j) represents the dot product (inner product) of the (i)th query vector and the (j)th key vector. This dot product is used to measure the similarity or association between the corresponding query and the key.
[0130] The result of (QK^T) is passed through a softmax layer, which normalizes the dot product of each row into a probability distribution. Therefore, for each query, its association with all keys is transformed into a probability distribution indicating the importance of each position to the current query position.
[0131] Optionally, obtaining the inverse document frequency of the keyword includes:
[0132] IDF(w) = log(N / n(w))
[0133] Wherein, IDF(w) is the inverse document frequency of the keyword, w is the keyword, N is the total number of words in the query process text, and n(w) is the number of keywords.
[0134] S124: Match the latest salesperson's response data with the labeling rules to obtain the sales dialogue response evaluation results.
[0135] In the specific implementation of this specification, regular expressions are used to match the latest salesperson response data with labeling rules to obtain sales dialogue response evaluation results. These evaluation results are then fed back to the salespersons to help them understand their performance in sales dialogues, identify shortcomings, and receive improvement suggestions. Based on actual sales performance and customer feedback, the labeling rules are continuously optimized and updated to ensure the accuracy and effectiveness of the evaluation results. Machine learning technology can be used to automatically learn and optimize the labeling rules, improving the automation level of the evaluation.
[0136] It adopts modern front-end technology stacks, such as React and Vue.js, to provide users with a simple and easy-to-use interface for operation and viewing results.
[0137] This invention significantly reduces the need for manual intervention by automating the generation and updating of labeling rules, greatly improving processing and analysis speed. It responds in real-time to dynamic changes in sales conversations, rapidly adjusting labeling rules and effectively improving work efficiency. An advanced large language model can deeply understand the complex semantics of sales conversations, generating more accurate labeling rules. This improves label accuracy, thereby enhancing the overall quality of sales process verification and reducing the cost of subsequent error handling and correction. The system operates automatically, requiring only minimal configuration from the user, greatly reducing operational complexity and difficulty. It reduces manual intervention, allowing users to quickly complete basic configurations through a simple interface, improving user experience and satisfaction. High processing efficiency and accuracy reduce system energy consumption and minimize resources wasted due to error handling. This indirectly promotes energy and raw material conservation, contributing positively to environmental protection. Digitalization and automation reduce paper usage and the need for physical equipment, lowering environmental resource consumption and potential pollution. As part of enterprise digital transformation, it helps reduce carbon footprint and contributes to environmental protection.
[0138] Figure 3 This specification provides a schematic diagram of an apparatus for evaluating the content of sales dialogue responses, which may include:
[0139] Module 10 is used to acquire audio data of the inquiry process in real time;
[0140] Analysis module 20 is used to analyze the audio data of the inquiry process through a large language model to obtain the sales dialogue response evaluation results.
[0141] Figure 4 This is a schematic diagram of the structure of the analysis module 20 provided in the embodiments of this specification. The device may include:
[0142] The identification unit 21 is used to identify the audio data of the inquiry process and determine the speaker's identity; the speaker's identity includes salesperson and customer;
[0143] The conversion unit 22 is used to convert the audio data of the inquiry process into text of the inquiry process labeled with the speaker's identity, based on the speaker's identity.
[0144] Analysis unit 23 is used to analyze the inquiry process text, obtain customer attention features, and match corresponding tag rules based on the customer attention features;
[0145] Evaluation unit 24 is used to match the latest salesperson's response data with the labeling rules to obtain the sales dialogue response evaluation result.
[0146] Optionally, the identification unit 21 includes:
[0147] Voiceprint features are extracted from the audio data of the inquiry process, and the voiceprint features are matched with the voiceprint features in the employee database to determine the voiceprint features corresponding to the salesperson.
[0148] The remaining voiceprint features are categorized to obtain the voiceprint features corresponding to the customer.
[0149] Optionally, the identification unit 21 includes:
[0150] Keyword extraction is performed on the audio data of the inquiry process, and the frequency of their occurrence is recorded;
[0151] The speaker's identity is determined based on the keywords and their frequency of occurrence.
[0152] Optionally, the conversion unit 22 includes:
[0153] The audio data of the inquiry process is subjected to language recognition using Bayes' theorem to obtain the text of the inquiry process, including:
[0154] P(W|O)={\frac{P(O|W)P(W)}{P(O)}
[0155] Wherein, P(W|O) is the conditional probability of the occurrence of the known acoustic signal sequence (O) and word sequence (W) in the audio data of the inquiry process; P(O|W) is the conditional probability of the occurrence of the known word sequence (W) and acoustic signal sequence (O) in the audio data of the inquiry process; P(W) is the prior probability of the word sequence (W), that is, the probability of generating the word sequence (W) when no acoustic signal is observed; P(O) is the marginal probability of the observed acoustic signal sequence (O), also known as evidence; {\frac{P(O|W)P(W)} is the posterior probability of the word sequence (W) under the condition that the acoustic signal (O) is observed, obtained by Bayes' theorem.
[0156] By combining the speaker's identity with the query process text, a query process text labeled with the speaker's identity is obtained.
[0157] Optionally, the analysis unit 23 includes:
[0158] The query process text is segmented and tagged with parts of speech.
[0159] Based on a pre-set keyword library and / or contextual semantic analysis, identify the keywords and / or key phrases in the customer's question;
[0160] Based on the weight and inverse document frequency of the keywords, and / or the weight and inverse document frequency of the key phrases, the attention ranking of the keywords and / or the key phrases is obtained, and the top N keywords and / or the key phrases are taken as customer attention features.
[0161] Optionally, the process of obtaining the weight of the keyword includes:
[0162] The keywords are input into the embedding layer to obtain the keyword vector;
[0163] The keyword vector is transformed into query vector, key vector, and value vector using different weight matrices.
[0164] An attention score is obtained based on the query vector and the key vector, and the attention score is converted into keyword weights using the softmax function.
[0165] Optionally, obtaining the inverse document frequency of the keyword includes:
[0166] IDF(w) = log(N / n(w))
[0167] Wherein, IDF(w) is the inverse document frequency of the keyword, w is the keyword, N is the total number of words in the query process text, and n(w) is the number of keywords.
[0168] The functions of the apparatus in this embodiment have been described in the above method embodiments. Therefore, for any parts not detailed in this embodiment, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.
[0169] Based on the same inventive concept, embodiments of this specification also provide an electronic device.
[0170] The following describes embodiments of the electronic device of the present invention, which can be considered as specific implementations of the methods and apparatus embodiments of the present invention described above. Details described in the embodiments of the electronic device of the present invention should be considered as supplements to the methods or apparatus embodiments described above; details not disclosed in the embodiments of the electronic device of the present invention can be implemented with reference to the methods or apparatus embodiments described above.
[0171] Figure 5 This is a schematic diagram of an electronic device provided as an embodiment of this specification. Refer to the following... Figure 5 The electronic device 300 according to this embodiment of the present invention will be described. Figure 5 The electronic device 300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.
[0172] like Figure 5 As shown, the electronic device 300 is presented in the form of a general-purpose computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including storage unit 320 and processing unit 310), a display unit 340, etc.
[0173] The storage unit stores program code that can be executed by the processing unit 310, causing the processing unit 310 to perform the steps described in the processing method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 310 can perform, for example... Figure 1 The steps are shown.
[0174] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache storage unit 3202, and may further include a read-only memory unit (ROM) 3203.
[0175] The storage unit 320 may also include a program / utility 3204 having a set (at least one) program module 3205, such program module 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0176] Bus 330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0177] Electronic device 300 can also communicate with one or more external devices 400 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable viewers to interact with electronic device 300, and / or with any device that enables electronic device 300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 350. Furthermore, electronic device 300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 360. Network adapter 360 can communicate with other modules of electronic device 300 via bus 330. It should be understood that, although... Figure 5 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0178] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described in this invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to this invention. When the computer program is executed by a data processing device, it enables the computer-readable medium to implement the above-described method of this invention, i.e.: as... Figure 1 The method shown.
[0179] Figure 6 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification.
[0180] accomplish Figure 1The computer program of the method shown can be stored on one or more computer-readable media. A computer-readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.
[0181] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.
[0182] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the audience's computing device, partially on the audience's device, as a standalone software package, partially on the audience's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the audience's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0183] In summary, this invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the invention. The invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the invention can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0184] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
[0185] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0186] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method of evaluating a sales dialogue response content, characterized by, The method comprises the following steps: real-time acquisition of inquiry process audio data; analysis of the inquiry process audio data by a large language model to obtain a sales dialogue response evaluation result, comprising: identifying the inquiry process audio data to determine the identity of the speaker; the speaker identity includes a salesperson and a customer; converting the inquiry process audio data into inquiry process text annotated with the speaker identity in combination with the speaker identity; analyzing the inquiry process text to obtain customer attention features and matching corresponding label rules based on the customer attention features; matching the latest salesperson response data with the label rules to obtain a sales dialogue response evaluation result.
2. The method of claim 1, wherein, The method comprises the following steps: extracting voiceprint features from the inquiry process audio data and matching the voiceprint features with employee library voiceprint features to determine the voiceprint features corresponding to the salesperson; classifying the remaining voiceprint features to obtain the voiceprint features corresponding to the customer.
3. The method of Claim 1, wherein, The method comprises the following steps: extracting keywords from the inquiry process audio data and recording their occurrence frequencies; determining the speaker identity based on the keywords and their occurrence frequencies.
4. The method of Claim 1, wherein, The method comprises the following steps: performing language recognition on the inquiry process audio data by Bayes theorem to obtain inquiry process text, comprising: P(W|O)={\frac{P(O|W)P(W)}{P(O)} where P(W|O) is the conditional probability of the known acoustic signal sequence (O) and the word sequence (W) in the inquiry process audio data; (P(O|W) is the conditional probability of the known word sequence (W) and the acoustic signal sequence (O) in the inquiry process audio data; P(W) is the prior probability of the word sequence (W), i.e. the probability of generating the word sequence (W) without observing any acoustic signal; P(O) is the marginal probability of the observed acoustic signal sequence (O), also known as evidence; {\frac{P(O|W)P(W)} is the posterior probability of the word sequence (W) under the condition of observing the acoustic signal (O) obtained by Bayes theorem. Combining the speaker identity with the inquiry process text to obtain inquiry process text annotated with the speaker identity.
5. The method of evaluating sales conversation response content of claim 4, wherein, The method comprises the following steps: performing word segmentation and part-of-speech tagging on the inquiry process text; identifying the keywords and / or key phrases of customer questions based on a pre-set keyword library and / or context semantic analysis; obtaining the attention ranking of the keywords and / or key phrases according to the weight and inverse document frequency of the keywords, and / or the weight and inverse document frequency of the key phrases, and taking the top N keywords and / or key phrases as customer attention features.
6. The method of evaluating sales conversation response content of claim 5, wherein, The process of obtaining the weight of the keyword comprises the following steps: inputting the keyword into an embedding layer to obtain a keyword vector; The keyword vector is transformed into a query vector, a key vector, and a value vector through different weight matrices; An attention score is obtained based on the query vector and the key vector, and the attention score is converted into a keyword weight through a softmax function.
7. The method of evaluating sales conversation response content of claim 6, wherein, An inverse document frequency of the keyword is obtained, including: IDF(w) = log(N / n(w)) Wherein, IDF(w) is the inverse document frequency of the keyword, w is the keyword, N is the total number of segmented words of the inquiry process text, and n(w) is the number of the keyword.
8. An apparatus for evaluating a response content of a sales dialogue, characterized by Including: An acquisition module is configured to acquire inquiry process audio data in real time; An analysis module is configured to analyze the inquiry process audio data through a large language model to obtain a sales dialogue response evaluation result, including: An identification unit is configured to identify the inquiry process audio data to determine a speaker identity; the speaker identity includes a salesperson and a customer; A conversion unit is configured to convert the inquiry process audio data into inquiry process text labeled with the speaker identity; An analysis unit is configured to analyze the inquiry process text to obtain a customer attention feature, and match a corresponding label rule based on the customer attention feature; An evaluation unit is configured to match the latest salesperson response data with the label rule to obtain a sales dialogue response evaluation result.
9. An electronic device, comprising: The electronic device includes: a processor; and a memory storing computer-executable instructions that, when executed, cause the processor to perform the method of any of claims 1-7.
10. A computer readable storage medium, wherein, The computer-readable storage medium stores one or more programs that, when executed by a processor, implement the method of any of claims 1-7.