Intelligent system for customer service quality inspection based on multi-source data

Abstract

The invention discloses an intelligent system for customer service quality inspection based on multi-source data, and particularly relates to the technical field of customer service quality inspection. The method comprises the following steps: acquiring session texts generated in a customer service process and associated product purchase and consumption data in a customer account in real time, and performing cleaning and standardization processing; a large-model automatic marking algorithm is constructed to carry out category marking on customer problems, and recognition of problem backgrounds and customer appeals is realized through context correlation analysis; performing double evaluation on customer attitude and customer service quality by using an emotion analysis model and a customer service verbal skill analysis model; and generating a comprehensive evaluation result of the overall quality of the customer service through fusion analysis, finally predicting a customer conversion rate and a contract extension rate based on the overall quality evaluation result, and generating a targeted customer service optimization strategy, thereby realizing accurate evaluation of the customer service quality and intelligent management of the customer relationship.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of customer service quality inspection, and more particularly to an intelligent system for customer service quality inspection based on multi-source data. BACKGROUND

[0002] With the expansion of customer service scale and the diversification of customer demand, traditional customer service quality evaluation methods are increasingly difficult to comprehensively and real-time analyze multi-dimensional information in the customer service process, especially difficult to comprehensively analyze the internal correlation between customer product purchase, use experience, emotional state and service performance of customer service personnel, so that enterprises cannot effectively find hidden customer churn risk in the service process, and also cannot improve customer satisfaction and customer conversion effect, resulting in low customer retention rate and business renewal rate. SUMMARY

[0003] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide an intelligent method and system for customer service quality inspection based on multi-source data to solve the problems raised in the background art.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0005] An intelligent system for customer service quality inspection based on multi-source data, comprising:

[0006] A data acquisition unit: real-time acquisition of customer service session text, product purchase and consumption data in customer account, and data cleaning, output of standardized multi-source customer service data;

[0007] A problem tagging unit: based on the standardized multi-source customer service data, a large model automatic tagging algorithm is constructed, problem category discrimination is performed, and the category label of customer problem is output;

[0008] A complaint recognition unit: based on the category label of customer problem, problem background and customer complaint recognition are performed through context association analysis, and problem background and customer complaint data are output;

[0009] A dialogue evaluation unit: based on the category label of customer problem, customer attitude and customer dialogue quality are evaluated through an emotional analysis model and a customer dialogue analysis model, and customer emotional tendency data and customer dialogue quality data are output;

[0010] A quality analysis unit: problem background and customer complaint data, customer emotional tendency data and customer dialogue quality data are fused and analyzed, and the overall quality evaluation result of customer service is generated;

[0011] A strategy optimization unit: based on the overall quality evaluation result of customer service, customer conversion rate and renewal rate are predicted, and customer service optimization strategy is generated.

[0012] In one preferred embodiment, the data collection unit, in particular:

[0013] obtain the text of the customer service session based on the transmission control protocol in real time through the session access interface;

[0014] obtain the product purchase and consumption data in the customer account in real time through the billing data interface;

[0015] time align and clean the data of the customer service session text and the product purchase and consumption data in the customer account;

[0016] convert the customer service session text and the product purchase and consumption data in the customer account after time alignment and data cleaning into a unified field naming and data type format using a field standard mapping table, and generate standardized multi-source customer service data.

[0017] In one preferred embodiment, the question tagging unit, in particular:

[0018] construct a large model automatic tagging algorithm based on the standardized multi-source customer service data, the large model automatic tagging algorithm including a text feature vectorization unit and a question category prediction unit;

[0019] The text feature vectorization unit uses text word embedding technology to convert the customer service session text in the standardized multi-source customer service data into text feature vector data;

[0020] The question category prediction unit uses a fully connected neural network structure to execute a question category prediction task using the text feature vector data and the product purchase and consumption data in the customer account as network input;

[0021] The question category prediction task outputs the category label of the customer question according to a pre-set multi-task classification label system.

[0022] In one preferred embodiment, the claim recognition unit, in particular:

[0023] extract the context window text corresponding to the category label of the customer question in the customer service session text;

[0024] perform entity recognition on the context window text to extract product names, function names, error code information, and time sequence markers;

[0025] analyze the association relationship between the named entities to generate an entity dependency relationship graph;

[0026] extract a question background field set in combination with the entity dependency relationship graph and the category label of the customer question;

[0027] According to the problem background field set and the word slot mapping table, the customer demand field is labeled;

[0028] The problem background field set and the customer demand field are combined to form problem background and customer demand data.

[0029] In a preferred embodiment, the script evaluation unit, specifically:

[0030] According to the category label of the customer problem, a sentiment analysis model and a customer service script analysis model are constructed;

[0031] The sentiment analysis model performs a semantic tendency analysis task on the customer expression statements in the customer service conversation text, classifies the customer expression statements into positive, neutral or negative sentiment tendency types, and outputs customer sentiment tendency data;

[0032] The customer service script analysis model performs a semantic content quality analysis task based on the customer service personnel reply statements in the customer service conversation text, analyzes the degree of accuracy of professional words, the standardization of expression methods, the completeness of introduction content, and the appropriateness of service attitude of the customer service personnel reply statements, and outputs customer service script quality evaluation data.

[0033] In a preferred embodiment, the quality analysis unit, specifically:

[0034] The problem background and customer demand data, customer sentiment tendency data, and customer service script quality evaluation data are subjected to numerical normalization, category field one-hot encoding, and missing field zero padding to obtain a unified format fusion input matrix;

[0035] A weight vector corresponding to the fields of the unified format fusion input matrix is generated according to a preset weight distribution rule;

[0036] The unified format fusion input matrix and the weight vector are subjected to matrix point multiplication calculation to output a field weighted score vector;

[0037] The field weighted score vector is summed and normalized to generate a customer service overall quality evaluation result.

[0038] In a preferred embodiment, the strategy optimization unit, specifically:

[0039] A prediction model of customer conversion rate and customer renewal rate is constructed based on the customer service overall quality evaluation result; the prediction model of customer conversion rate and customer renewal rate is a logistic regression model;

[0040] The input of the logistic regression model is the field weighted score vector in the customer service overall quality evaluation result;

[0041] The output of the logistic regression model is the probability of predicting the customer conversion rate and the customer renewal rate;

[0042] set a customer conversion and renewal probability threshold according to the predicted customer conversion rate and customer renewal rate probability;

[0043] When the predicted customer conversion rate and customer renewal rate probability is lower than the customer conversion and renewal probability threshold, a corresponding customer service optimization strategy is generated according to a preset customer service quality improvement mapping table.

[0044] The technical effects and advantages of the intelligent system for customer service quality inspection based on multi-source data are as follows:

[0045] Through real-time collection and standardized processing of multi-source customer service data, high-precision data fusion and information integration of the whole customer service process are realized, and the comprehensiveness and timeliness of customer service quality inspection are improved. The introduction of large model automatic tagging algorithm and context intelligent analysis technology makes the customer problem type discrimination and appeal recognition more accurate, and can automatically aggregate and interpret the conversation context, customer appeal and problem background. Combined with sentiment analysis and speech quality evaluation, not only the customer attitude change is measured in detail, but also the service professionalism and communication ability of the customer service personnel are quantitatively analyzed. After multi-dimensional data fusion, the output is a quantifiable service quality evaluation result, which provides a solid data foundation for intelligent prediction and optimization decision of conversion rate and renewal rate, thereby effectively helping enterprises improve customer satisfaction, service efficiency and customer value mining ability, realizing high-quality management and continuous optimization of customer relationship. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 The structure diagram of the intelligent system for customer service quality inspection based on multi-source data is given. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0048] Embodiment 1

[0049] Figure 1 The intelligent system for customer service quality inspection based on multi-source data is given, which comprises:

[0050] The data acquisition unit acquires customer service conversation text, product purchase and consumption data in the customer account in real time, and performs data cleaning to output standardized multi-source customer service data;

[0051] Problem labeling unit: based on the standardized multi-source customer service data, a large model automatic labeling algorithm is constructed to perform problem category discrimination and output the category label of customer problems;

[0052] Claim identification unit: based on the category label of customer problems, the problem background and customer claim are identified through context association analysis, and the problem background and customer claim data are output;

[0053] Script evaluation unit: based on the category label of customer problems, the customer attitude and customer service script quality are evaluated through sentiment analysis model and customer service script analysis model, and the customer sentiment tendency data and customer service script quality data are output;

[0054] Quality analysis unit: the problem background and customer claim data, customer sentiment tendency data and customer service script quality data are fused and analyzed to generate the overall quality evaluation result of customer service;

[0055] Strategy optimization unit: based on the overall quality evaluation result of customer service, the customer conversion rate and renewal rate are predicted to generate the customer service optimization strategy.

[0056] Real-time collection of customer service conversation text, product purchase and consumption data in customer account, and data cleaning, output of standardized multi-source customer service data, including:

[0057] Real-time acquisition of customer service conversation text based on transmission control protocol through conversation access interface;

[0058] Real-time acquisition of product purchase and consumption data in customer account through billing data interface;

[0059] Time alignment and data cleaning of customer service conversation text and product purchase and consumption data in customer account;

[0060] The customer service conversation text and product purchase and consumption data in customer account after time alignment and data cleaning are converted into unified field naming and data type format by using field standard mapping table to generate standardized multi-source customer service data.

[0061] Specifically, a session access interface is arranged in the customer service system server, and the session access interface establishes a data communication link with the customer service terminal device according to the standard communication rule of the transmission control protocol; the customer service terminal device includes an intelligent mobile terminal used by a customer, a computer device, and a work order processing terminal used by a customer service personnel, and can send customer service session text generated in a service process to the customer service system server in real time in a data message format conforming to the transmission control protocol standard; the customer service session text refers to dialogue content in a text form generated by interaction between the customer and the customer service personnel; for example, the customer of the customer service terminal device can input consultation information about a function problem of a purchased product in the intelligent mobile terminal, and the customer service personnel can reply to the consultation information of the customer in a text form through the work order processing terminal, so as to form a two-way exchange customer service session text; after the session access interface receives the customer service session text sent by the customer service terminal device in real time, the customer service system server buffers the received customer service session text, and marks the received customer service session text according to a unified precision timestamp, so as to align the customer service session text with other types of data in time.

[0062] The billing data interface is also arranged in the customer service system server, and the billing data interface is connected to a product billing system in real time, and the product billing system is used to store product purchase data corresponding to each customer account and product consumption data; the product purchase data includes product types purchased by the customer, purchase time, purchase quantity, and amount of money at the time of purchase and the like; the product consumption data refers to consumption in an actual use process after the product is purchased by the customer, for example, the number of times of function use of the product, the number of times of service call, the service call duration, the change of the product use state and the like; the billing data interface extracts the product purchase data and the product consumption data corresponding to each customer account from the product billing system in real time, and temporarily buffers the product purchase data and the product consumption data; the billing data interface also marks each piece of product purchase data and product consumption data obtained from the product billing system according to a unified timestamp, so as to ensure that the product purchase data and the product consumption data can be aligned with the customer service session text in the time dimension; for example, the customer purchased a cloud service product at 10:15 on August 14, 2023, the billing data interface records the purchase behavior in real time, and records the product consumption data of the customer on the same day, for example, the number of times of call or the amount of consumed resources of the cloud service product on the same day; and the same millisecond level timestamp is marked.

[0063] The customer service system server obtains customer service session text data, product purchase data and product consumption data with unified precision timestamps from the session access interface and the billing data interface respectively, and aligns the data by timestamp; the time alignment of the data refers to one-to-one correspondence of the customer service session text, product purchase data and product consumption data in the same or similar time interval to form an overall data set based on the same time dimension; the time alignment operation adopts a window sliding algorithm to match and combine the data belonging to the same customer account and having a time interval within a predetermined time threshold with the timestamp as the center; for example, set 5 minutes as the time threshold for alignment, and the product purchase data and consumption data appearing within 5 minutes before and after the specific time of the customer service session are matched accordingly; the data after time alignment is then cleaned; the data cleaning includes filtering of abnormal characters in the aligned data, such as removing special characters that cannot be recognized; desensitizing sensitive information in the data, such as hiding sensitive personal information of the customer's mobile phone number, ID number, etc. in an encrypted manner; uniform conversion of character encoding, such as converting the data encoding format to a standard uniform character set; and filling of missing values or fields in the data, such as completing the missing data by setting a default value or using a historical average value; after time alignment and data cleaning, accurate and consistent high-quality data is formed.

[0064] The field standard mapping table is a pre-set data structure that defines the unified field name and field data type corresponding to the customer service session text, product purchase data and product consumption data in the final standardized data; for example, the product problem content input by the customer in the customer service session text is uniformly mapped to the "problem description" field, the purchase time in the product purchase data is uniformly mapped to the "purchase time" field, and the product usage frequency in the product consumption data is uniformly mapped to the "usage frequency" field; after time alignment and data cleaning of the data, the customer service system server maps each data according to the field standard mapping table, and uniformly converts the customer service session text data, product purchase data and product consumption data into a standardized structured data format; the converted standardized data includes unified field naming and unified data types, such as the "problem description" field being uniformly of string type, the "purchase time" field being uniformly of time type, and the "usage frequency" field being uniformly of integer type; after the above conversion operation, the multi-source customer service data after standardization is obtained, which has uniform format, consistent content structure and can be directly used for subsequent analysis; for example, the data of a certain customer after conversion may be in the form of "problem description" as "unable to log in to the product", "purchase time" as "August 14, 2023, 10:15", "usage frequency" as "0 times", etc. Standard format data records.

[0065] Based on the standardized multi-source customer service data, a large model automatic tagging algorithm is constructed to perform problem category discrimination and output the category label of the customer problem, including:

[0066] Based on the standardized multi-source customer service data, a large model automatic tagging algorithm is constructed, which includes a text feature vectorization unit and a problem category prediction unit.

[0067] The text feature vectorization unit uses text word embedding technology to convert the customer service conversation text in the standardized multi-source customer service data into text feature vector data.

[0068] The problem category prediction unit uses a fully connected neural network structure to perform problem category prediction tasks using text feature vector data and product purchase and consumption data in customer accounts as network input.

[0069] The problem category prediction task outputs the category label of the customer problem according to a pre-set multi-task classification label system.

[0070] Specifically, after the customer service system server receives the standardized multi-source customer service data, it uses the standardized multi-source customer service data to construct a large model automatic tagging algorithm, which is used to automatically identify and label the category of customer problems.

[0071] The large model automatic tagging algorithm includes a text feature vectorization unit and a problem category prediction unit.

[0072] The text feature vectorization unit is responsible for converting customer service conversation text data stored in text form into text feature vector data suitable for computational processing, while the problem category prediction unit uses text feature vector data and customer account product purchase and consumption data to jointly perform automatic prediction of customer problem categories.

[0073] The text feature vectorization unit and the problem category prediction unit cooperate with each other to jointly realize automatic identification and labeling of customer problem categories. For example, if a customer consults a software that cannot start, the large model automatic tagging algorithm can automatically identify the corresponding category of the problem as "software startup exception", thereby facilitating customer service personnel to quickly locate the processing flow corresponding to the customer problem.

[0074] The text feature vectorization unit extracts customer service conversation text from the standardized multi-source customer service data.

[0075] After standardization, the customer service conversation text has been converted into data records with unified structure and clear fields, where each customer service conversation text is a data record stored in standard string type.

[0076] The text feature vectorization unit utilizes a text word embedding technique, i.e., according to the semantic relationship between the words contained in the customer service session text, the customer service session text is converted into a computable feature vector;

[0077] For example, the customer service session text is: "Customer feedback that the product cannot log in, error code 502", the text feature vectorization unit first performs word segmentation processing on the session text to obtain a word set such as "customer", "feedback", "product", "cannot", "log in", "prompt", "error code", and "502"; the text feature vectorization unit maps the words according to the semantic relationship, converts each word into a high-dimensional numerical vector, for example, maps "product" to a word embedding vector containing 256 dimensions, and then the customer service session text is converted into a matrix or text feature vector data composed of multiple word embedding vectors for processing by the problem category prediction unit.

[0078] The problem category prediction unit takes the text feature vector data and the product purchase and consumption data in the customer account as input variables, and uses a fully connected neural network structure to perform a problem category prediction task; the fully connected neural network structure is composed of an input layer, multiple hidden layers, and an output layer;

[0079] The input layer is responsible for receiving the text feature vector data and the product purchase and consumption data in the customer account, and the text feature vector data and the product purchase and consumption data in the customer account are respectively presented in the form of numerical vectors and input into the input layer of the fully connected neural network structure;

[0080] For example, the text feature vector data formed by the customer service session text and the corresponding product purchase and consumption data in the customer account, such as the product purchase record "cloud server advanced version" and the consumption data "service call times totaling 32 times", are synchronously input into the input layer of the network;

[0081] The multiple hidden layers perform nonlinear combination processing on the numerical vectors of the input layer and extract deep feature representations;

[0082] The output layer is responsible for converting the features extracted by the hidden layer into corresponding problem category prediction results;

[0083] For example, after network structure processing, the output layer may represent the probability value of the customer service session text belonging to the "software login problem" category in the form of probability, assuming that the output layer calculation result is 0.92, which indicates that the text highly matches the "software login problem" category characteristics;

[0084] The problem category prediction unit takes the category with the largest probability value as the final predicted customer problem category and generates a problem category label.

[0085] The customer service system server pre-sets a multi-task classification label system, covering various types of service problems that customers may encounter, and each problem category corresponds to a specific classification label, such as "account login problem", "product function abnormality", "recharge and deduction abnormality", "response delay", and different category labels;

[0086] The problem category prediction unit outputs the result directly corresponding to the pre-set multi-task classification label system after completing the calculation of the fully connected neural network structure;

[0087] Each output layer neuron corresponds to a category label, and the output layer calculates the probability value of each category label through the Softmax function;

[0088] The problem category prediction task judges according to the probability value of each category label corresponding to the output layer, and outputs the category label with the largest probability value as the category label of the customer problem;

[0089] For example, the probability value of the "account login problem" category in the output layer of the fully connected neural network is 0.10, the probability value of the "product function abnormality" category is 0.78, the probability value of the "recharge and deduction abnormality" category is 0.08, and the probability value of the "response delay" category is 0.04. The problem category prediction task will select the "product function abnormality" category label with the highest probability value as the final category label of the customer problem;

[0090] The output customer problem category label is stored in the standardized multi-source customer service data record to realize automatic identification and efficient processing of customer problems.

[0091] Based on the category label of the customer problem, the problem background and customer demand identification are performed through context association analysis, and the problem background and customer demand data are output, including:

[0092] Extract the context window text corresponding to the category label of the customer problem in the customer service conversation text;

[0093] Perform entity recognition on the context window text to extract product names, function names, error code information, and time sequence markers;

[0094] Parse the association relationship between named entities to generate an entity dependency relationship graph;

[0095] Extract the problem background field set in combination with the entity dependency relationship graph and the category label of the customer problem;

[0096] Annotate the customer demand field according to the problem background field set and the word slot mapping table;

[0097] Combine the problem background field set and the customer demand field to form the problem background and customer demand data.

[0098] Specifically, after receiving the customer problem category label, the customer service system server will process the customer service session text stored in the standardized multi-source customer service data using the customer problem category label, so as to select the context window text data corresponding to the customer problem category label from the customer service session text data record. The context window text data refers to the continuous text segment of the dialogue content between the customer and the customer service personnel centered on the position of the problem category marked by the customer problem category label, which is used to provide sufficient dialogue background information. The customer service system server uses the pre-set window size rule to extract a certain number of customer service session text sentences forward and backward from the text position of the customer's first question, to ensure that the extracted context window text data can fully present the cause and effect of the customer's problem. For example, assuming that the customer service session text records the communication content between the customer and the customer service personnel from 10:20 to 10:30 on August 14, 2023, and the customer problem category label is "recharge fee deduction anomaly", and the customer raises the question at 10:25, then the server takes the position of the customer's question at 10:25 as the benchmark, and extracts 10 consecutive sentences forward and backward, a total of 20 sentences of text, to form the context window text data corresponding to the customer problem category label, which can more fully understand the cause and background of the customer's problem.

[0099] The customer service system server performs named entity recognition operation on the extracted context window text data, and the purpose of named entity recognition is to accurately extract specific entity types closely related to the customer's problem from the context window text data. A variety of types of entity dictionary or annotation rules are pre-defined in the customer service system server, including product name entity, function name entity, error code information entity, and time sequence mark entity. The server scans and matches the context window text data sentence by sentence, and extracts various entities according to the stored entity dictionary or annotation rules. For example, the context window text data may appear the text sentence "the customer reflects that the use of cloud server advanced version cannot be connected at 10:25 on August 14, 2023, and prompts error code EC500", and the customer service system server will extract "cloud server advanced version" as the product name entity, "cannot be connected" as the function name entity, "EC500" as the error code information entity, and "2023 August 14 10:25" as the time sequence mark entity, which is used for accurate analysis of the background of the customer's problem.

[0100] After completing entity recognition, the customer service system server parses the semantic association relationship between the extracted product name entity, function name entity, error code information entity, and time sequence label entity to construct an entity dependency relationship graph. The entity dependency relationship graph is a structured representation form constituted by taking the extracted entity as a node and the semantic relationship between entities as an edge, which can express the semantic relationship and logical relationship between various entities in the context window text data. The customer service system server uses syntax analysis and semantic relationship recognition rules to parse and identify the possible cause-and-effect relationship, dependency relationship, time sequence relationship, and subordinate relationship between each pair of entities. For example, when the customer raises the question of “the cloud server advanced version cannot be connected at 10:25 on August 14, 2023, and the error code EC500 is prompted”, the server parses the generated entity dependency relationship graph, and the “cloud server advanced version” entity as the main node has a functional subordinate relationship with the “cannot be connected” entity, the “error code EC500” entity has a fault prompt cause-and-effect relationship with the “cannot be connected” entity, and the “2023 August 14 10:25” entity has a time sequence relationship with the “cannot be connected” entity. The entity dependency relationship graph shows the internal logical connection between entities when the customer problem occurs.

[0101] The customer service system server combines the already constructed entity dependency relationship graph and the customer problem category label, and uses a question background field extraction algorithm to determine various problem background fields directly associated with the customer problem category label from the entity dependency relationship graph. The question background field set is predefined by the server and includes information fields closely related to the customer problem category label, such as product type field, function operation field, fault code field, and problem occurrence time field. The customer service system server filters out the entity information corresponding to the customer problem category label by traversing each entity node and entity edge in the entity dependency relationship graph according to the field extraction rules. For example, the customer problem category label is “recharge deduction exception”, the server extracts the “cloud server advanced version” entity information as the product type field from the entity dependency relationship graph, extracts the corresponding “2023 August 14 10:25” entity information as the problem occurrence time field, and includes the “error code EC500” entity information in the fault code field to form the question background field set.

[0102] The customer service system server performs the task of labeling customer request fields based on the extracted problem background field set and a predefined slot mapping table. The slot mapping table is a structured mapping table stored on the customer service system server, used to map the extracted problem background fields to the fields of the customer's request. The customer request field labeling algorithm takes the field values ​​in the problem background field set as input and maps them one by one to the corresponding customer request type fields according to the slot mapping table. For example, if the problem background field set contains the product type field "Cloud Server Advanced Edition", the function operation field "Unable to connect", and the fault code field "EC500", and the slot mapping table pre-maps "Unable to connect" to "Product Fault Report", the server can label the customer request field as "Product Fault Report" based on the mapping relationship, thus expressing the customer's actual request content.

[0103] The customer service system server integrates the problem background field set and the labeled customer request field into a unified structured data record, ultimately generating problem background and customer request data. The server uses a data record merging algorithm to combine the integrated data records of the problem background field set and the labeled customer request field into a unified data format, where each record simultaneously contains both the problem background field and the corresponding customer request field. For example, if a customer reports that "the Cloud Server Advanced Edition could not connect at 10:25 AM on August 14, 2023, error code EC500," the processed problem background field set will be: "Product Type: Cloud Server Advanced Edition, Fault Code: EC500, Problem Time: August 14, 2023, 10:25 AM," and the customer request field will be "Product Fault Report." The final merged problem background and customer request data record will then be: "Product Type: Cloud Server Advanced Edition; Fault Code: EC500; Problem Time: August 14, 2023, 10:25 AM; Customer Request: Product Fault Report," which can be directly used for efficient response and handling of customer issues.

[0104] Based on customer issue category tags, customer attitudes and customer service script quality are evaluated using sentiment analysis and customer service script analysis models. The output includes customer sentiment data and customer service script quality data, including:

[0105] Based on the category tags of customer issues, construct a sentiment analysis model and a customer service script analysis model;

[0106] The sentiment analysis model performs semantic sentiment analysis on customer expressions in customer service conversation texts, classifying customer expressions into positive, neutral, or negative sentiment types, and outputs customer sentiment data.

[0107] The customer service script analysis model performs a semantic content quality analysis task based on the customer service personnel reply statements in the customer service session text, analyzes the professional word accuracy, expression manner standardization, introduction content integrity, and service attitude propriety of the customer service personnel reply statements, and outputs customer service script quality evaluation data.

[0108] Specifically, after obtaining the customer problem category label, the customer service system server constructs a sentiment analysis model and a customer service script analysis model for the standardized multi-source customer service data. Both models are constructed and optimized based on the customer problem category label. The sentiment analysis model is used to analyze the emotional tendency and attitude expressed by the customer in the customer service session text, and the customer service script analysis model is used to analyze the reply statement quality and compliance of the customer service personnel in the customer service session text. The customer service system server constructs two analysis models according to the customer problem category label, so as to accurately analyze the customer sentiment tendency data and the customer service script quality evaluation data. For example, if the customer problem category label is "product function abnormality", the customer service system server will construct a sentiment analysis model and a customer service script analysis model for a large number of historical customer service session text data of the product function abnormality category, in order to capture the customer emotional expression mode and the reply script characteristics of the customer service personnel specific to the product function abnormality category. The sentiment analysis model and the customer service script analysis model are trained based on machine learning algorithms and a large amount of historical data. The selection of training data is directly affected by the customer problem category label, ensuring that the sentiment analysis model and the customer service script analysis model are targeted, accurate and applicable.

[0109] The task of the sentiment analysis model is to analyze the sentiment tendency contained in the customer expression statement in the customer service conversation text. The customer expression statement is the literal content input by the customer through the customer service terminal device during the service process. The input data of the sentiment analysis model is the customer expression statement extracted from the standardized multi-source customer service data. The sentiment analysis model adopts a sentiment classification algorithm based on semantic tendency analysis technology, which includes three specific functional modules: a text segmentation module, a semantic feature extraction module, and a sentiment tendency classification module. First, the text segmentation module performs Chinese word segmentation on the customer expression statement, decomposing the complete sentence into independent lexical units; second, the semantic feature extraction module identifies sentiment-related words based on the segmentation results and extracts corresponding semantic sentiment features, such as extracting sentiment words such as "satisfied", "dissatisfied", "good", "bad", "disappointed", and judging the sentiment intensity in combination with the context before and after the text; the sentiment tendency classification module classifies the extracted semantic sentiment features, and according to the pre-defined sentiment classification rules or the trained classification model, finally classifies the sentiment tendency of the customer expression statement into three sentiment tendency types: positive, neutral, or negative. For example, a customer expresses in the service conversation text: "The product update speed is very fast, the function is easy to use, and I am very satisfied", the sentiment analysis model gets the words "product", "update speed", "fast", "function", "easy to use", "very satisfied" through the text segmentation module, the semantic feature extraction module captures the positive sentiment words "fast", "easy to use", "very satisfied", and the sentiment tendency classification module determines that the sentiment tendency of the customer expression statement is "positive" after comprehensive calculation. The sentiment analysis model analyzes each customer expression statement in turn and generates corresponding customer sentiment tendency data, which includes the original text of the customer expression statement and its corresponding sentiment tendency type.

[0110] The customer service dialogue analysis model performs semantic content quality analysis on the customer service personnel reply statements in the customer service conversation text. The customer service personnel reply statement refers to the text reply content made by the customer service personnel to the customer expression statement in the customer service terminal device. The customer service dialogue analysis model includes four functional modules: professional word accuracy analysis module, expression manner standardization analysis module, introduction content integrity analysis module, and service attitude appropriateness analysis module. The professional word accuracy analysis module is responsible for analyzing the accuracy of the use of product names, function names, and technical terms involved in the customer service personnel reply statement. For example, when the customer service personnel replies to the customer's "cloud server advanced version connection failure", the model automatically evaluates whether the customer service personnel correctly uses the "cloud server advanced version" product name and whether the corresponding technical terms are correctly described or incorrectly described. The expression manner standardization analysis module evaluates the standardization of the grammar, word usage, punctuation usage, and language logical structure of the customer service personnel reply statement. For example, whether the customer service personnel's reply statement contains errors, misspelled words, or inappropriate expression manners. The introduction content integrity analysis module evaluates whether the customer service personnel's reply content is complete and whether necessary explanation information is omitted. For example, when the customer asks about the specific function, whether the customer service personnel's reply covers the function detail description and usage method introduction. The service attitude appropriateness analysis module analyzes the appropriateness of the customer service personnel's reply tone, wording, and polite language. For example, whether the reply is too harsh, mechanical, or obviously lacks polite language. For example, when the customer asks, "The product cannot be connected. What's wrong?", the customer service personnel's reply statement is "Hello, please check the network configuration or contact the customer service personnel for processing". The functional modules of the customer service dialogue analysis model analyze the statement in turn. The professional word accuracy analysis module analyzes whether the customer service reply accurately corresponds to the customer's problem. The expression manner standardization analysis module analyzes whether the reply is language standard and reasonable. The introduction content integrity analysis module analyzes whether the reply covers the required solution steps. The service attitude appropriateness analysis module analyzes whether the customer service reply is polite and appropriate. The above analysis generates customer service dialogue quality evaluation data for the reply statement. The customer service dialogue quality evaluation data contains the score or classification result of each quality evaluation index, which is used to objectively describe and record the quality of the customer service reply.

[0111] The problem background and customer demand data, customer sentiment tendency data, and customer service dialogue quality data are fused and analyzed to generate overall quality evaluation results of customer service, including:

[0112] The problem background and customer demand data, customer sentiment tendency data, and customer service dialogue quality evaluation data are normalized, category field one-hot encoded, and missing field zero-filled to obtain a unified format fusion input matrix;

[0113] generate a weight vector corresponding to the unified format fusion input matrix field according to a preset weight distribution rule;

[0114] perform matrix point multiplication calculation on the unified format fusion input matrix and the weight vector, and output a field weighted score vector;

[0115] perform summation normalization on the field weighted score vector to generate a customer service overall quality evaluation result.

[0116] Specifically, after obtaining the question background and customer demand data, customer sentiment tendency data, and customer service skill quality evaluation data, the customer service system server performs standardization processing to unify the multiple data to the same data scale and format. The question background and customer demand data include multiple different dimension description fields, such as product type, function operation type, fault code information, and customer demand category; the customer sentiment tendency data includes the positive, neutral, or negative classification results corresponding to the customer expression statement; and the customer service skill quality evaluation data includes the professional word accuracy degree, expression manner standard degree, introduction content completeness degree, and service attitude appropriateness degree scores of the customer service personnel reply statement.

[0117] The customer service system server performs numerical normalization on the fields involving continuous numerical value type data to eliminate the differences between different numerical value dimensions. The numerical normalization adopts the minimum-maximum normalization method to normalize all numerical values to the interval between 0 and 1.

[0118] The customer service system server performs category field one-hot encoding on the fields involving category type data in the question background and customer demand data and the customer sentiment tendency data. All possible category values contained in the field are determined, and for each category value, a corresponding binary new field is created, which takes the value 1 when the category appears and 0 when the category does not appear, so that each category type field is converted into a multi-column binary numerical field. For example, the original categories of the customer sentiment tendency data include positive, neutral, and negative three categories, and after one-hot encoding, three new fields of “positive sentiment tendency”, “neutral sentiment tendency”, and “negative sentiment tendency” are formed. If the original data of the customer sentiment tendency of the current record is “positive”, the encoded data is that the “positive sentiment tendency” field is 1 and the other two fields are 0.

[0119] The customer service system server performs zero padding on missing fields, that is, for the data missing situation that may occur in the question background and customer demand data, customer sentiment tendency data, and customer service skill quality evaluation data, the customer service system server fills the missing data positions with zero values to ensure the number of fields of all records is consistent and eliminate the data inconsistency caused by missing values. For example, if a record is missing the “fault code” field in the question background and customer demand data, the server directly sets the value corresponding to this field in the unified fusion input matrix to 0.

[0120] The customer service system server integrates the processed data of various types and forms a unified format fusion input matrix. The unified format fusion input matrix is a structured two-dimensional data array. Each row of data in the matrix represents a customer service record, and each column of data represents a unified processed characteristic field.

[0121] The customer service system server predefines a weight allocation rule for the importance of the customer service characteristics and analysis dimensions represented by each field in the unified format fusion input matrix. The weight allocation rule is obtained by domain experts or based on historical data statistical analysis and stored in a weight rule database in the server. The weight rule database takes the field name as the primary key, and each field name corresponds to a pre-defined weight value to represent the importance difference of the field in the overall customer service quality evaluation process. For example, the weight of the "positive sentiment tendency" field is pre-set to 0.2, the weight of the "negative sentiment tendency" field is pre-set to 0.5, and the weight of the "accuracy of customer service words" field is pre-set to 0.4. The server generates a weight vector corresponding to the unified format fusion input matrix according to this rule database. The data structure of the weight vector is a one-dimensional numerical array, and each numerical element of the weight vector corresponds to each field of the unified format fusion input matrix.

[0122] The customer service system server performs matrix point multiplication calculation on the unified format fusion input matrix and the weight vector, which is the Hadamard product operation. The server performs element-by-element multiplication operation on each record (i.e. each row of the matrix) in the unified format fusion input matrix and the corresponding elements of the weight vector. The original characteristic value of each field is multiplied by the corresponding weight value to generate a field weighted score matrix with the same size as the unified format fusion input matrix. For example, the "positive sentiment tendency" field value of a certain record in the unified format fusion input matrix is 1, and the "accuracy of customer service words" field value is 0.75. The corresponding values of the weight vector are 0.2 and 0.4, respectively. The corresponding record field weighted score calculation result is "positive sentiment tendency" field score 0.2 and "accuracy of customer service words" field score 0.75 x 0.4 = 0.3.

[0123] The customer service system server performs summation operation on all field weighted scores of each record in the field weighted score matrix to obtain the comprehensive score value of each record. The comprehensive score value is normalized by using the minimum-maximum normalization method. The overall quality evaluation result of customer service directly reflects the good or bad of the overall performance of each record in the customer service process, providing data support for customer service improvement.

[0124] Based on the overall quality evaluation result of customer service, the customer conversion rate and renewal rate are predicted, and the customer service optimization strategy is generated, including:

[0125] construct a prediction model of customer conversion rate and customer renewal rate based on the overall quality evaluation result of customer service; the prediction model of customer conversion rate and customer renewal rate is a logistic regression model;

[0126] the input of the logistic regression model is a field weighted score vector in the overall quality evaluation result of customer service;

[0127] the output of the logistic regression model is a probability of predicting customer conversion rate and customer renewal rate;

[0128] set a customer conversion and renewal probability threshold according to the predicted customer conversion rate and customer renewal rate probability;

[0129] when the predicted customer conversion rate and customer renewal rate probability is lower than the customer conversion and renewal probability threshold, generate a corresponding customer service optimization strategy according to a preset customer service quality improvement mapping table.

[0130] Specifically, the customer service system server establishes a prediction model of customer conversion rate and customer renewal rate according to the overall quality evaluation result of customer service; the overall quality evaluation result of customer service is composed of weighted scores of various fields, reflecting the comprehensive satisfaction degree and service experience of customers after receiving customer service. The customer conversion rate refers to the probability that a customer converts into actual purchase, upgrade or subscription service after receiving service; the customer renewal rate refers to the probability that a customer who has purchased or subscribed to a service continues to renew or continues to use the service after the service expires. The customer service system server uses a logistic regression model as the prediction model of customer conversion rate and customer renewal rate. The logistic regression model is a classification model in the field of statistics, which uses historical correlation between specific field weighted score vectors in sample data and customer conversion or renewal behavior for analysis, and establishes a statistical mapping relationship that can predict customer conversion or renewal probability. The customer service system server trains the model through historical overall quality evaluation results of customer service and corresponding actual conversion and renewal behavior data. For example, the customer service system server collects service quality evaluation score vectors of all customers in the past six months, and records whether each customer has realized conversion and whether they have chosen to renew, forming a large-scale training sample set. Based on this training sample set, the customer service system server uses a logistic regression model algorithm to construct a prediction model of customer conversion rate and customer renewal rate.

[0131] The customer service system server sets the input data of the logistic regression model as a field weighted score vector corresponding to each record in the customer service overall quality evaluation result. The field weighted score vector is a comprehensive score set of each customer service quality indicator obtained by standardizing, weighting, and matrix point multiplication calculating each customer service indicator and score. The field weighted score vector includes customer sentiment tendency score, customer service personnel speech quality score, customer complaint and problem background information comprehensive score, and a series of indicators. Each field weighted score vector is a standardized numerical vector, which can be directly used as an input variable of the logistic regression model. For example, the field weighted score vector of a customer may be: "emotional tendency positive score 0.8", "customer service word accuracy score 0.7", "customer complaint resolution satisfaction score 0.6", etc. The above scores constitute the customer service overall quality evaluation result, which is used as an input variable of the logistic regression model.

[0132] The customer service system server uses the logistic regression model to analyze the input field weighted score vector and calculates the probability value of the customer conversion rate and the customer renewal rate. The logistic regression model internally uses the Sigmoid function as the model output conversion function. The calculation process is as follows: the customer service system server applies the Sigmoid function to the linear combination operation of the input field weighted score vector in the logistic regression model, and finally obtains the customer conversion rate and customer renewal rate probability between 0 and 1. For example, after the customer field weighted score vector is input into the logistic regression model, the linear combination output value is 1.5 after internal operation of the model. The logistic regression model uses the Sigmoid function to convert it into a probability, i.e. the Sigmoid function expression is "1 divided by 1 plus the negative 1.5 power of the natural constant e", and the calculation result is about 0.8176, which means that the predicted probability of customer conversion rate or customer renewal rate is 81.76%.

[0133] The customer service system server pre-sets the customer conversion and renewal probability threshold, which is determined according to historical experience, industry standards, and the enterprise's own business objectives. After the customer service system server outputs the probability of the logistic regression model, it compares the predicted customer conversion rate and customer renewal rate probability with the threshold. For example, the customer service system server determines the customer conversion probability threshold to be 0.7 and the customer renewal probability threshold to be 0.65 according to past customer conversion data. When the logistic regression model outputs a customer conversion rate probability of 0.8176, it is higher than the set conversion probability threshold of 0.7, indicating that the customer belongs to a high conversion probability group. If a customer renewal rate probability is 0.55, which is lower than the set renewal probability threshold of 0.65, it indicates that the customer may have a renewal risk and needs customer service intervention.

[0134] The customer service system server internally stores a customer service quality improvement mapping table, which includes the correspondence between customer service quality problems and corresponding improvement strategies. The mapping table content is predefined by enterprise business experts and covers optimization measures for different customer service quality deficiencies. When the customer conversion rate and customer renewal rate predicted by the logistic regression model are lower than the corresponding threshold, the customer service system server selects the corresponding customer service optimization strategy from the mapping table according to the specific score of the field weighted score vector. For example, the customer's renewal rate probability is predicted to be 0.55, which is lower than the threshold 0.65. The customer service system server finds that the customer service skill score is too low through the field weighted score vector. According to the corresponding optimization strategy of "insufficient customer service professional language" in the customer service quality improvement mapping table, the system automatically generates the strategy "additional professional term training for customer service personnel" or "arrange customer service experts to directly communicate with customers and explain relevant product professional details to improve customer recognition and renewal willingness of the service". The customer service optimization strategy is reflected as an automated instruction or a manual processing suggestion to guide the enterprise's customer service action.

[0135] The above formulas are dimensionless numerical calculations. The formula is obtained by collecting a large amount of data to simulate the latest real situation. The preset parameters and threshold values in the formula are set by a person skilled in the art according to the actual situation.

[0136] The above embodiments can be realized wholly or partially by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized wholly or partially in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another by wired (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like containing one or more available medium sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0137] Those skilled in the art can clearly understand that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and module can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0139] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0140] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, which can be located in one place or distributed on multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0141] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically alone, or two or more modules can be integrated in one module.

[0142] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0143] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0144] Finally: the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An intelligent system for customer service quality inspection based on multi-source data, characterized in that, Comprise: Data acquisition unit: real-time acquisition of customer service session text, product purchase and consumption data in customer account, and data cleaning, output standardized multi-source customer service data; Problem labeling unit: based on the standardized multi-source customer service data, a large model automatic labeling algorithm is constructed, the problem category is distinguished, and the category label of customer problem is output; Claim identification unit: based on the category label of customer problem, the problem background and customer claim are identified through context association analysis, and the problem background and customer claim data are output; Speech evaluation unit: based on the category label of customer problem, the customer attitude and customer service speech quality are evaluated through sentiment analysis model and customer service speech analysis model, and the customer emotion tendency data and customer service speech quality data are output; Quality analysis unit: the problem background and customer claim data, customer emotion tendency data and customer service speech quality data are fused and analyzed to generate the overall quality evaluation result of customer service; Strategy optimization unit: based on the overall quality evaluation result of customer service, the customer conversion rate and renewal rate are predicted, and the customer service optimization strategy is generated. 2.The intelligent system based on multi-source data customer service quality inspection according to claim 1, characterized in that, The data acquisition unit is specifically: Real-time acquisition of customer service session text based on transmission control protocol through a session access interface; Real-time acquisition of product purchase and consumption data in customer account through a billing data interface; Time alignment and data cleaning of customer service session text and product purchase and consumption data in customer account; The time-aligned and data-cleaned customer service session text and product purchase and consumption data in customer account are converted into a unified field naming and data type format using a field standard mapping table to generate standardized multi-source customer service data. 3.The intelligent system based on multi-source data customer service quality inspection according to claim 2, characterized in that, The problem labeling unit is specifically: Based on the standardized multi-source customer service data, a large model automatic labeling algorithm is constructed, which includes a text feature vectorization unit and a problem category prediction unit; The text feature vectorization unit uses text word embedding technology to convert the customer service session text in the standardized multi-source customer service data into text feature vector data; The problem category prediction unit uses a fully connected neural network structure to take the text feature vector data and product purchase and consumption data in the customer account as network input to perform problem category prediction task; The problem category prediction task outputs the category label of customer problem according to a pre-set multi-task classification label system. 4.The intelligent system based on multi-source data customer service quality inspection according to claim 3, characterized in that, The claim identification unit is specifically: Extract the context window text corresponding to the category label of customer problem in the customer service session text; Perform entity recognition on the context window text to extract product name, function name, error code information and time sequence mark; Parse the association relationship between named entities to generate an entity dependency relationship graph; Extract a set of problem background fields based on the entity dependency relationship graph and the category label of customer problem; Label customer claim fields according to the set of problem background fields and word slot mapping table; Combine the set of problem background fields and customer claim fields to form problem background and customer claim data. 5.The intelligent system based on multi-source data customer service quality inspection according to claim 4, characterized in that, The speech evaluation unit is specifically: According to the category label of customer problem, construct a sentiment analysis model and a customer service speech analysis model; The emotion analysis model divides the customer expression sentences in the customer service conversation text into positive, neutral or negative sentiment tendency types by performing a semantic tendency analysis task on the customer expression sentences, and outputs customer sentiment tendency data; The customer service skill analysis model performs a semantic content quality analysis task on the customer service personnel reply sentences in the customer service conversation text, analyzes the accuracy of professional words, the standardization of expression methods, the completeness of introduction content and the appropriateness of service attitude of the customer service personnel reply sentences, and outputs customer service skill quality evaluation data. 6.The intelligent system based on multi-source data customer service quality inspection according to claim 5, characterized in that, The quality analysis unit, specifically: The problem background, customer appeal data, customer sentiment tendency data and customer service skill quality evaluation data are subjected to numerical normalization, category field one-hot encoding and missing field zero padding to obtain a unified format fusion input matrix; According to a preset weight distribution rule, a weight vector corresponding to the fields of the unified format fusion input matrix is generated; The unified format fusion input matrix and the weight vector are subjected to matrix point multiplication calculation to output a field weighted score vector; The field weighted score vector is summed and normalized to generate a customer service overall quality evaluation result. 7.The intelligent system based on multi-source data customer service quality inspection according to claim 6, characterized in that, The strategy optimization unit, specifically: A prediction model of customer conversion rate and customer renewal rate is constructed based on the customer service overall quality evaluation result; the prediction model of customer conversion rate and customer renewal rate is a logistic regression model; The input of the logistic regression model is the field weighted score vector in the customer service overall quality evaluation result; The output of the logistic regression model is the probability of predicting the customer conversion rate and the customer renewal rate; According to the predicted customer conversion rate and customer renewal rate probability, customer conversion and renewal probability thresholds are set; When the predicted customer conversion rate and customer renewal rate probability is lower than the customer conversion and renewal probability threshold, a corresponding customer service optimization strategy is generated according to a preset customer service quality improvement mapping table.

Images