Multi-dimensional customer identification method and system based on online voice

Abstract

The invention discloses a multi-dimensional customer identification method and system based on online voice. The method comprises the following steps: pre-establishing a blacklist word library database; reading call content and collecting audio data during a call; directly converting the audio data into text data, and segmenting the text data into words and word units; converting the audio data into a character format based on the audio features and segmenting the audio data into words and word units; comparing the words with the word units based on a blacklist word library database, and obtaining blacklist incoming call numbers based on a comparison result; on-line voices of the identified non-blacklist incoming call number are screened, emotion analysis is carried out, and positive, negative and neutral emotions are identified; according to the method and the device, the number of the crank call and the number of the junk information are added into the blacklist, so that unnecessary interference and disturbance can be effectively reduced, and the working efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of multi-dimensional customer identification based on online voice, and in particular to a multi-dimensional customer identification method and system based on online voice. Background Art

[0002] After-sales customer service is an important bridge of communication between enterprises and consumers. It is mainly responsible for handling various problems and needs after the product or service is sold. High-quality after-sales customer service can not only solve customer problems, but also enhance customer satisfaction and loyalty, and have a positive impact on the brand image of the enterprise.

[0003] The main functions of after-sales customer service are: 1. Answer questions: Provide customers with guidance on product use, maintenance, etc., and answer any questions customers have about the product; 2. Handling complaints: accept and properly handle customer complaints, including product quality issues, logistics delays, etc., and try to provide satisfactory solutions; 3. Returns and exchanges: Handle return and exchange requests in accordance with company policies to ensure a transparent and efficient process; 4. Feedback collection: Collect customer opinions and suggestions to help improve products and services and enhance customer experience; 5. Repair support: For products that require repair, coordinate repair matters, track repair progress, and update customers on status in a timely manner; 6. Continuous follow-up: Continue to follow up after the problem is solved to confirm whether the customer is satisfied and ask if there is any other help needed.

[0004] After-sales customer service is not only a reflection of the quality of a company's services, but also the key to building long-term customer relationships. By continuously improving service levels and applying technology, we can effectively enhance customer trust and loyalty, thereby promoting sustained business growth.

[0005] After-sales customer service needs to answer a large number of after-sales calls every day. It is time-consuming and laborious to manually answer and handle them one by one. Therefore, how to automatically classify and identify after-sales calls to facilitate subsequent classification and processing by customer service plays a vital role in improving customer service work efficiency and customer satisfaction. Summary of the invention

[0006] In view of the problems in the related art, the present invention proposes a multi-dimensional customer identification method and system based on online voice to overcome the above-mentioned technical problems existing in the existing related art.

[0007] To this end, the specific technical solution adopted by the present invention is as follows: A multi-dimensional customer identification method based on online voice includes the following steps: S1. Pre-establish a blacklist word database, which includes promotional words, inappropriate remarks and illegal words; S2, reading the call content and collecting audio data during the call, preprocessing the audio data, and standardizing the audio data; S3, using automatic speech recognition technology to directly convert the audio data into text data, and segment the text data into words and word units, where the word units are segmented from words; S4, using a deep learning model to extract features from the audio data to obtain audio features, converting the audio data into text format based on the audio features and dividing the audio data into words and word units; S5, comparing the words and word units of the text data obtained in S3 and S4 with the words and word units obtained based on the audio features based on a blacklist word database, and obtaining a blacklist caller number based on the comparison result; S6, filtering the online voice of the non-blacklisted caller number identified in S5, using a real-time emotion recognition tool to perform emotion analysis on the audio data and text data respectively, and identifying positive, negative and neutral emotions; S7. Filter the online speech with negative and neutral emotions in S6, and use the speech intent recognition model to perform speech intent recognition on the audio data and text data respectively to determine the user's intention.

[0008] Furthermore, when converting audio data into text data, the audio data is uniformly converted using standard Mandarin pronunciation.

[0009] Furthermore, when extracting audio features, the dialect type to which the speech may belong is identified based on the dialect recognition model, and the audio data is converted into appropriate text according to the pronunciation and emotional habits of the dialect. Finally, the text is divided into words and character units.

[0010] Further, in step S5, when the words and word units of the text data obtained in S3 and S4 and the words and word units obtained based on the audio features are compared based on the blacklist word library database, S5.1. First, compare several groups of words and word units obtained based on audio features with the blacklist word database to obtain several groups of results, and extract the incoming call numbers identified as containing blacklist words; S5.2, directly converting the audio data into text data and identifying the incoming call number containing the blacklist vocabulary by comparing the segmented words and word units with the blacklist word database; S5.3. Compare the recognition results of steps S5.1 and S5.2, and output the incoming call numbers that are recognized as containing blacklist words in both steps.

[0011] Further, segmenting words and word units includes: A1. First, split the words word by word, filter out the words without emotional expression intention, and then output them; A2. Split the words again to determine whether the original words can be split into several words. If so, output the split words, and then filter out the words without emotional expression intent before outputting them.

[0012] Furthermore, during the sentiment analysis in step S6, the online speech is classified into two types, namely, Mandarin and dialect, and sentiment analysis is performed based on the language type.

[0013] Furthermore, the training process of the speech intent recognition model includes: S7.1. Collect conversation data, including customer questions and corresponding intent labels, and clean and preprocess the data. S7.2. Training speech intent recognition model based on deep learning algorithm; S7.3. Evaluate and optimize the speech intent recognition model through feedback, and regularly optimize and train the model to improve the accuracy of intent recognition.

[0014] According to another aspect of the present invention, there is also provided a multi-dimensional customer identification system based on online voice, which is used to implement any of the multi-dimensional customer identification methods based on online voice as described above, including: The word database module: used to pre-establish a blacklist word database, which includes promotional words, inappropriate remarks and illegal words; Call content reading module: used to read the call content and collect audio data during a call, pre-process the audio data, and standardize the audio data; Audio conversion module: used to directly convert audio data into text data using automatic speech recognition technology, and to segment the text data into words and word units, where word units are segmented from words; Audio to feature extraction module: used to extract features from audio data using a deep learning model to obtain audio features, convert the audio data into text format based on the audio features, and divide it into words and word units; Blacklist caller number screening module: used for comparing the words and word units of the text data obtained by the audio conversion module and the audio-to-feature extraction module with the words and word units obtained based on the audio features based on the blacklist word database, and obtaining the blacklist caller number based on the comparison result; Sentiment analysis module: used to filter the online voice of non-blacklisted caller numbers identified by the blacklisted caller number screening module, and use real-time emotion recognition tools to perform emotion analysis on audio data and text data respectively to identify positive, negative and neutral emotions; Speech intent recognition module: used to filter online speech with negative and neutral emotions in the sentiment analysis module, and use the speech intent recognition model to perform speech intent recognition on audio data and text data respectively to determine the user's intention.

[0015] The beneficial effects of the present invention are: 1. In actual use, the present invention can effectively reduce unnecessary interference and interruption, improve work efficiency, and devote more time and energy to serving and developing high-value customers by adding numbers with harassing calls and spam messages to the blacklist. Adding numbers involving fraudulent behavior or security risk markers to the blacklist can identify and prevent problems in advance and protect the interests of enterprises and customers. 2. In actual use, the present invention converts audio data into text data with standard Mandarin pronunciation, and converts audio data into appropriate text according to the pronunciation and emotional habits of the dialect to which the audio data belongs, and comprehensively judges whether there are words matching the blacklist word database in the voice content, so as to screen the blacklist caller numbers more accurately; 3. In actual use, the present invention can refine and screen the blacklisted caller numbers more accurately by dividing the voice into words and character units; 4. In actual use, the present invention can accurately identify customers with negative emotions and the intentions of the customers through layer-by-layer screening, which facilitates subsequent processing by customer service and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 The present invention is a flowchart of a multi-dimensional customer identification method based on online voice according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] According to an embodiment of the present invention, a multi-dimensional customer identification method based on online voice is provided, such as Figure 1As shown, the following steps are included: S1. Pre-establish a blacklist word database, which includes promotional words, inappropriate remarks and illegal words.

[0020] S2. Read the call content and collect audio data during the call, pre-process the audio data, standardize the audio data, use the noise reduction algorithm to remove background noise, standardize the audio signal, ensure the consistency of the audio data input into the model, and divide the long audio into short clips for easy processing and annotation.

[0021] S3. Use automatic speech recognition technology to directly convert audio data into text data, and divide the text data into words and word units. The word units are composed of words. This step directly converts the audio data into text using standard Mandarin pronunciation.

[0022] S4. Use a deep learning model to extract features from audio data to obtain audio features. When extracting audio features, identify the type of dialect that the speech may belong to based on the dialect recognition model. Convert the audio data into appropriate text according to the pronunciation and emotional habits of the dialect. Finally, divide the text into words and word units. Because different dialects have different pronunciations, it is inevitable to make mistakes when using the same logic for speech recognition. This includes the different intensity of emotional expression in different regions. Even if some special words are recognized in time, the meaning of the expression may be biased. For example, in Henan dialect: This table is really difficult to assemble. The customer wants to express that the product is difficult to assemble. However, if "real" is recognized as a swear word and the customer's number is recognized as a harassing call, there will be recognition errors. Therefore, it is necessary to convert the speech into text based on the habits of the dialect.

[0023] S5, comparing the words and word units of the text data obtained in S3 and S4 with the words and word units obtained based on the audio features based on the blacklist word database, specifically, including: S5.1. First, compare several groups of words and word units obtained based on audio features with the blacklist word database to obtain several groups of results, and extract the incoming call numbers identified as containing blacklist words; S5.2, directly converting the audio data into text data and identifying the incoming call number containing the blacklist vocabulary by comparing the segmented words and word units with the blacklist word database; S5.3. Compare the recognition results of steps S5.1 and S5.2, and output the incoming call numbers that are recognized as containing blacklist words in both steps.

[0024] This step can be combined with dialect translation to convert speech into text, which can improve the accuracy of speech recognition.

[0025] When segmenting words and character units, first split the words word by word, filter out the characters without emotional expression intention, and then output them; then split the words into words to determine whether the original words can be split into several words. If so, output the split words, and then filter out the words without emotional expression intention, and then output them. This can fine-tune the screening of words that affect emotional judgment, which is very helpful for improving recognition accuracy.

[0026] S6. Filter the online voices of non-blacklisted caller numbers identified in S5, use real-time emotion recognition tools to perform emotion analysis on audio data and text data respectively, identify positive, negative and neutral emotions, and perform language type identification and classification on the online voices during emotion analysis, including Mandarin and dialects. Emotion analysis based on language type is more accurate.

[0027] The specific sentiment analysis includes the following steps: S6.1. Recognize and classify the customer's voice into two categories: Mandarin and dialect, and determine the language of the customer's voice at the beginning of the call; S6.2. Use real-time emotion recognition tools for Mandarin to extract audio data and text data, and quickly and accurately extract key acoustic features from speech signals; S6.3. Use the rich training data provided by Emo-DB to train the emotion recognition model to ensure the accuracy and robustness of the model; S6.4. Integrate real-time emotion recognition tools into the customer service system to analyze the emotional state of customers during calls in real time and identify positive, negative and neutral emotions; S6.5. Based on the emotion recognition results, the system provides real-time feedback to customer service personnel.

[0028] S7. Filter the online voices with negative and neutral emotions in S6. This step can filter out customers who need key services. Use the voice intent recognition model to perform voice intent recognition on the audio data and text data respectively to determine the user's intention, which is convenient for customer service to handle the problem later.

[0029] The specific application scenarios of this application are as follows: The audio data is converted to text as follows: Customer: Hello, does your company have any intention of investing in financial management? By comparing the sales pitch words with those in the blacklist word database, the system determines that the customer is making a harassing call and directly hangs up the harassing call, which can improve the work efficiency of customer service.

[0030] During a voice call, every word the customer says is identified. For example, if the customer asks whether he needs to swipe an order, receive welfare benefits, invest in financial management, or take out an online loan, etc., the customer will be put on the blacklist and marked as such when there is fraud or harassment.

[0031] If the audio data is converted to text as follows: Customer: Why hasn’t the problem I reported yesterday been solved yet? The text is segmented into words and character units, and the key words "feedback", "problem" and "solution" are identified. It is determined that the customer's emotions are negative and that he wants to get the problem resolved as soon as possible. The system provides feedback to customer service personnel in real time, so that customer service can handle and resolve the problem in a timely manner and gradually reduce complaints.

[0032] During the voice call, after the customer's voice is classified and determined, each sentence of the dialect customer's voice is analyzed. For example, a customer from Henan said: "You are such a bull, you half-baked guy" (meaning "your behavior is really incomprehensible"), "I'm a good boy, you're a bear" (meaning "Oh my God, what are you doing"), etc. Similar dialect voices are subjected to sentiment analysis and feedback is given after the emotion analysis. While helping customer service understand the customer service dialect text, it can also help customer service perceive the positive and negative emotions of dialect customers, making it easier for customer service to respond based on the customer's positive and negative emotions.

[0033] In summary, with the help of the above-mentioned technical scheme of the present invention, in actual use, the present invention can effectively reduce unnecessary interference and interruption and improve work efficiency by adding numbers of harassing calls and spam messages to the blacklist. By adding number segments or regional number segments of certain customers to the blacklist, the enterprise can optimize the allocation of customer resources and devote more time and energy to serving and developing high-value customers, thereby improving sales results and customer satisfaction. By adding numbers involved in fraud or marked with security risks to the blacklist, problems can be identified and prevented in advance, thereby protecting the interests of the enterprise and customers.

[0034] In actual use, the present invention, through dialect speech sentiment analysis, is applicable to people all over the country, and can help companies more deeply understand the thoughts and feelings of different customers across the country, identify and solve specific problems that lead to customer dissatisfaction, replicate success, and develop new products and services that meet customer preferences and emotional needs. Real-time emotion recognition tools can collect a wide range of data sets from multiple sources, help to better manage customer service operations, determine the priority of inquiries and automatically track bad interactions, thereby improving customer service. In addition, emotion recognition can be performed for different regional dialects, making it easier for customer service to serve users in different regions.

[0035] In actual use, the present invention uses dialect emotion recognition technology to facilitate customer service to understand the implicit intentions of different users across the country. The system can help customer service to respond to user needs more accurately, reduce the occurrence of erroneous commands or irrelevant answers, and provide personalized services and suggestions based on the user's implicit intentions, thereby increasing user satisfaction.

[0036] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-dimensional customer identification method based on online voice, characterized in that: The following steps are involved: S1. Pre-establish a blacklist word database, which includes promotional words, inappropriate remarks and illegal words; S2, reading the call content and collecting audio data during the call, preprocessing the audio data, and standardizing the audio data; S3, using automatic speech recognition technology to directly convert the audio data into text data, and segment the text data into words and word units, where the word units are segmented from words; S4, using a deep learning model to extract features from the audio data to obtain audio features, converting the audio data into text format based on the audio features and dividing the audio data into words and word units; S5, comparing the words and word units of the text data obtained in S3 and S4 with the words and word units obtained based on the audio features based on a blacklist word database, and obtaining a blacklist caller number based on the comparison result; S6, filtering the online voice of the non-blacklisted caller number identified in S5, using a real-time emotion recognition tool to perform emotion analysis on the audio data and text data respectively, and identifying positive, negative and neutral emotions; S7. Filter the online speech with negative and neutral emotions in S6, and use the speech intent recognition model to perform speech intent recognition on the audio data and text data respectively to determine the user's intention.

2. The multi-dimensional customer identification method based on online voice according to claim 1 is characterized in that: When converting audio data into text data, the audio data is uniformly converted using standard Mandarin pronunciation.

3. A multi-dimensional customer identification method based on online voice according to claim 1 or 2, characterized in that: When extracting audio features, the dialect type that the speech may belong to is identified based on the dialect recognition model. According to the pronunciation and emotional habits of the dialect, the audio data is converted into appropriate text, and finally the text is divided into words and word units.

4. The multi-dimensional customer identification method based on online voice according to claim 3 is characterized in that: In step S5, when the words and word units of the text data obtained in S3 and S4 and the words and word units obtained based on the audio features are compared based on the blacklist word library database, S5.

1. First, compare several groups of words and word units obtained based on audio features with the blacklist word database to obtain several groups of results, and extract the incoming call numbers identified as containing blacklist words; S5.2, directly converting the audio data into text data and identifying the incoming call number containing the blacklist vocabulary by comparing the segmented words and word units with the blacklist word database; S5.

3. Compare the recognition results of steps S5.1 and S5.2, and output the incoming call numbers that are recognized as containing blacklist words in both steps.

5. The multi-dimensional customer identification method based on online voice according to claim 1 is characterized in that: Segmentation of words and word units includes: A1. First, split the words word by word, filter out the words without emotional expression intention, and then output them; A2. Split the words again to determine whether the original words can be split into several words. If so, output the split words, and then filter out the words without emotional expression intent before outputting them.

6. The multi-dimensional customer identification method based on online voice according to claim 1 is characterized in that: In step S6, during the sentiment analysis, the online speech is classified into two types: Mandarin and dialect, and sentiment analysis is performed based on the language type.

7. The multi-dimensional customer identification method based on online voice according to claim 1 is characterized in that: The training process of the speech intent recognition model includes: S7.

1. Collect conversation data, including customer questions and corresponding intent labels, and clean and preprocess the data. S7.

2. Training speech intent recognition model based on deep learning algorithm; S7.

3. Evaluate and optimize the speech intent recognition model through feedback, and regularly optimize and train the model to improve the accuracy of intent recognition.

8. A multi-dimensional customer identification system based on online voice, used to implement a multi-dimensional customer identification method based on online voice as described in any one of claims 1 to 7, characterized in that: include: The word database module: used to pre-establish a blacklist word database, which includes promotional words, inappropriate remarks and illegal words; Call content reading module: used to read the call content and collect audio data during a call, pre-process the audio data, and standardize the audio data; Audio conversion module: used to directly convert audio data into text data using automatic speech recognition technology, and to segment the text data into words and word units, where word units are segmented from words; Audio to feature extraction module: used to extract features from audio data using a deep learning model to obtain audio features, convert the audio data into text format based on the audio features, and divide it into words and word units; Blacklist caller number screening module: used for comparing the words and word units of the text data obtained by the audio conversion module and the audio-to-feature extraction module with the words and word units obtained based on the audio features based on the blacklist word database, and obtaining the blacklist caller number based on the comparison result; Sentiment analysis module: used to filter the online voice of non-blacklisted caller numbers identified by the blacklisted caller number screening module, and use real-time emotion recognition tools to perform emotion analysis on audio data and text data respectively to identify positive, negative and neutral emotions; Speech intent recognition module: used to filter online speech with negative and neutral emotions in the sentiment analysis module, and use the speech intent recognition model to perform speech intent recognition on audio data and text data respectively to determine the user's intention.

Images