Complaint work order processing method and system based on artificial intelligence analysis

Abstract

The invention provides a complaint work order processing method and system based on artificial intelligence analysis, and the method comprises the steps: carrying out the clustering analysis of historical complaint work order data through an intelligent analysis model based on artificial intelligence analysis, and determining a work order category system; and then, identifying and classifying the to-be-processed complaint work order based on the work order category system through the intelligent response model, and determining a work order theme category and a work order grade corresponding to the to-be-processed complaint work order, so as to realize grading processing of the complaint work order, optimize processing resources and improve processing efficiency. And finally, under the condition that the work order level of the to-be-processed complaint work order is the first level, determining a first target solution corresponding to the to-be-processed complaint work order through the intelligent response model, and under the condition that the work order level of the to-be-processed complaint work order is the second level, determining a corresponding second target solution through the intelligent recommendation model. Therefore, intelligent question-answering is realized. Therefore, the complaint work order processing efficiency and accuracy can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer operation and maintenance technology, and specifically to a method and system for processing complaint work orders based on artificial intelligence analysis. Background Art

[0002] With the rapid development of computer technology, new business forms such as microservices and containerized transformation of business support systems have continued to develop and play an important role. However, the increasingly complex businesses and systems have brought challenges to business support service personnel.

[0003] Specifically, with the continuous optimization and upgrading of customer service systems and the rapid development of big data storage and analysis technology, a large number of complaint tickets have been generated. Currently, the large number of complaint tickets generated by the support system every day are manually processed with a large amount of human resources.

[0004] However, this relatively simple processing method has problems such as scattered fault reporting channels, little service experience, difficulty in cross-system collaboration, and untimely warning push. This leads to many problems such as low efficiency and long processing time for complaint tickets, inaccurate processing results, etc., making it impossible to handle complaint tickets efficiently and accurately. Summary of the invention

[0005] The technical problem to be solved by this application is the inability to efficiently and accurately handle complaint tickets.

[0006] In order to solve the above technical problems, this application provides a method and system for handling complaint tickets based on artificial intelligence analysis, which specifically adopts the following technical solutions:

[0007] In the first aspect, the present application provides a method for handling complaint work orders based on artificial intelligence analysis, including: first, clustering analysis of historical complaint work order data is performed through an intelligent analysis model to determine a work order category system; the work order category system includes: multiple work order subject categories and classification standards and work order levels corresponding to the work order subject categories, and the work order levels include at least: first level, second level and third level. Then, the complaint work orders to be processed are identified and classified based on the work order category system through an intelligent response model to determine the work order subject category and work order level corresponding to the complaint work orders to be processed. Finally, when the work order level corresponding to the complaint work order to be processed is the first level, the first target solution corresponding to the complaint work order to be processed is determined from the first solution database based on the work order subject category corresponding to the complaint work order to be processed by the intelligent response model. When the work order level corresponding to the complaint work order to be processed is the second level, the complaint work order to be processed is sent to the business personnel processing system, and the second target solution corresponding to the complaint work order to be processed is determined from the second solution database based on the work order subject category corresponding to the complaint work order to be processed by the intelligent recommendation model. If the work order level corresponding to the pending complaint work order is the second level, the pending complaint work order will be sent to the business personnel processing system. If the work order level corresponding to the pending complaint work order is the third level, the pending complaint work order will be sent to the backend support personnel processing system.

[0008] The method can perform cluster analysis on historical complaint work order data through an intelligent analysis model based on artificial intelligence analysis to determine the work order category system. The intelligent response model can identify and classify the complaint work orders based on the work order category system, determine the work order theme category and work order level corresponding to the complaint work orders to be processed, so as to achieve hierarchical processing of complaint work orders, optimize processing resources, and improve processing efficiency. When the work order level corresponding to the complaint work order to be processed is the first level, the intelligent response model can be used to determine the first target solution corresponding to the complaint work order to be processed from the first solution database based on the work order theme category corresponding to the complaint work order to be processed. When the work order level corresponding to the complaint work order to be processed is the second level, the intelligent recommendation model can be used to determine the second target solution corresponding to the complaint work order to be processed from the second solution database based on the work order theme category corresponding to the complaint work order to be processed, so as to achieve intelligent question and answer. The method can realize three application scenarios: intelligent question and answer based on intelligent operation and maintenance robots, intelligent recommendation with the goal of accurate solution matching, and intelligent analysis with the construction of a work order category system based on hot issue mining, thereby improving the efficiency and accuracy of complaint work order processing.

[0009] In combination with the first aspect, in an optional implementation, the above-mentioned cluster analysis of the historical complaint work order data by the intelligent analysis model to determine the work order category system includes: first, converting each historical complaint work order in the historical complaint work order data into a corresponding text feature representation. Then, clustering is performed based on the text feature representation corresponding to each historical complaint work order using a clustering algorithm through the intelligent analysis model to obtain a work order category system.

[0010] In this implementation, first, each historical complaint work order is converted into a corresponding text feature representation. Then, a work order category system is constructed by clustering based on the text feature representation through an intelligent analysis model, thereby improving the accuracy of constructing the work order category system.

[0011] In combination with the first aspect, in an optional implementation, the above-mentioned identification and classification of the complaint work orders to be processed based on the work order category system by the intelligent response model to determine the work order subject category and work order level corresponding to the complaint work orders to be processed includes: first, the word embedding model is used to perform word embedding feature processing on the complaint work orders to be processed to determine the word feature vector corresponding to the complaint work orders to be processed. Then, the word feature vector corresponding to the complaint work orders to be processed is classified and processed based on the work order category system by the intelligent response model to obtain the work order subject category and work order level corresponding to the complaint work orders to be processed.

[0012] In this implementation, first, word embedding feature processing is performed on the complaint ticket to be processed. Then, the word feature vector is classified based on the ticket category system by the intelligent response model to determine the ticket subject category and ticket level. In this way, the intelligent response model can effectively improve the accuracy of classification by classifying the complaint ticket to be processed based on the word feature vector.

[0013] In combination with the first aspect, in an optional implementation method, the above-mentioned intelligent recommendation model is used to determine the second target solution corresponding to the complaint work order to be processed from the second solution database based on the work order subject category corresponding to the complaint work order to be processed, including: first, the complaint work order to be processed is searched and segmented to obtain the segmentation result. Then, based on the segmentation result and the work order subject category corresponding to the complaint work order to be processed, multiple initial solutions corresponding to the complaint work order to be processed are determined from the second solution database through the intelligent recommendation model. Next, the multiple initial solutions are sorted based on the degree of match between the initial solutions and the complaint work order to be processed through a re-ranking algorithm. Finally, the initial solution with the largest match among the multiple initial solutions is used as the second target solution.

[0014] In this implementation, the accuracy of determining the solution corresponding to the complaint worksheet to be processed can be effectively improved by performing search word segmentation processing on the complaint worksheet to be processed and re-ranking the initial solution based on the matching degree.

[0015] In combination with the first aspect, in an optional implementation, the intelligent analysis model is one of the following models: a term frequency-inverse document frequency model, a latent Dirichlet allocation model, a neural network-based text classification model, and an association rule mining model.

[0016] In combination with the first aspect, in an optional implementation method, the above-mentioned intelligent answering model is one of the following models: a fast text classification FastText model, a text classification convolutional neural network TextCNN model.

[0017] In combination with the first aspect, in an optional implementation, the intelligent recommendation model is one of the following models: a rule-based expert system model, a decision tree model, a naive Bayes model, and a neural network model.

[0018] In combination with the first aspect, in an optional implementation, the above-mentioned word embedding model is one of the following models: Word2Vec model, GloVe model, BERT model, GPT model.

[0019] In combination with the first aspect, in an optional implementation, the above-mentioned reordering algorithm is one of the following algorithms: RE-Rank algorithm, List-wise algorithm, Pair-wise algorithm.

[0020] In the second aspect, the present application provides a complaint work order processing system based on artificial intelligence analysis, including: an intelligent analysis module, an intelligent response module and an intelligent recommendation module. Among them, the intelligent analysis module is used to perform cluster analysis on historical complaint work order data through an intelligent analysis model to determine the work order category system; the work order category system includes: multiple work order subject categories and classification standards and work order levels corresponding to the work order subject categories, and the work order levels include at least: the first level, the second level and the third level; the intelligent response module is used to identify and classify the complaint work orders to be processed based on the work order category system through the intelligent response model, and determine the work order subject category and work order level corresponding to the complaint work orders to be processed. The intelligent response module is also used to determine the first target solution corresponding to the complaint work order to be processed from the first solution database based on the work order subject category corresponding to the complaint work order to be processed through the intelligent response model when the work order level corresponding to the complaint work order to be processed is the first level. The intelligent response module is also used to send the complaint work order to be processed to the business personnel processing system when the work order level corresponding to the complaint work order to be processed is the second level. The intelligent recommendation module is used to determine the second target solution corresponding to the unprocessed complaint work order from the second solution database based on the work order subject category corresponding to the unprocessed complaint work order through the intelligent recommendation model. The intelligent response module is also used to send the unprocessed complaint work order to the backend support personnel processing system when the work order level corresponding to the unprocessed complaint work order is the third level.

[0021] According to a third aspect, an electronic device is provided, comprising: a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the method according to the first aspect and any one of the optional methods thereof.

[0022] According to a fourth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method according to the first aspect and any optional method thereof.

[0023] It can be understood that the beneficial effects that can be achieved by the complaint ticket processing system based on artificial intelligence analysis provided by the second aspect, the electronic device of the third aspect, and the computer-readable storage medium of the fourth aspect can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a method for handling complaint tickets based on artificial intelligence analysis provided in an embodiment of the present application;

[0025] Figure 2A schematic diagram of a method flow for determining a second target solution provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of the structure of the complaint ticket processing system based on artificial intelligence analysis provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0028] With the rapid development of computer technology, new business forms such as microservices and containerized transformation of business support systems have continued to develop and play an important role. However, the increasingly complex businesses and systems have brought challenges to business support service personnel.

[0029] Specifically, with the continuous optimization and upgrading of customer service systems and the rapid development of big data storage and analysis technology, a large number of complaint tickets have been generated. Currently, the large number of complaint tickets generated by the support system every day are manually processed with a large amount of human resources.

[0030] However, this relatively simple handling method has problems such as scattered fault reporting channels, little service experience accumulation, difficulty in cross-system collaboration, and untimely warning push. Specifically, the fault reporting channels are scattered: fault reporting is distributed in various channels such as WeChat, QQ, intranet, and email, which is easy to miss orders and difficult to restore problems. Fault reporting data is scattered in multiple cities and counties, and the provincial end cannot fully collect real information from the front line and handle front-line problems in a timely manner. Less service experience accumulation: simple consulting work orders such as business rules and work order handling account for more than 40%. Most of these problems can be directly answered to customers through simple verification, but there is a lack of effective accumulation and accumulation of problem handling experience. Fault handling experience cannot form knowledge and cannot be shared, and the service personnel for work order processing are seriously consumed. Difficulty in cross-system collaboration: A complaint work order or customer consultation work often requires multiple business systems to verify, which is time-consuming. When verifying complex complaint issues, it is easy to miss them based on personal complaint handling experience alone. Untimely complaint warning push: The system is not online or the warning is not changed in time, and service personnel cannot quickly and accurately know. Fault resolution or recovery cannot be quickly publicized. This leads to many problems such as low efficiency, long time limit and inaccurate processing results in complaint ticket processing, making it impossible to process complaint tickets efficiently and accurately.

[0031] In order to solve the above problems, the embodiments of the present application provide a complaint ticket processing method and system based on artificial intelligence analysis. The method can realize three application scenarios based on artificial intelligence analysis, including intelligent question and answer based on intelligent operation and maintenance robots, intelligent recommendation with the goal of precise solution matching, and intelligent analysis of building a ticket category system with hot issue mining as the main line, through intelligent analysis models, intelligent response models and intelligent recommendation models, so as to achieve the purpose of efficiently and accurately processing complaint tickets.

[0032] The solution provided by the embodiment of the present application is introduced below in conjunction with the accompanying drawings.

[0033] For details, see Figure 1 , which is a flowchart of a method for processing a complaint work order based on artificial intelligence analysis provided in an embodiment of the present application, such as Figure 1 As shown, the embodiment of the present application provides a method for processing a complaint ticket based on artificial intelligence analysis, including the following steps S101-S105:

[0034] S101. Perform cluster analysis on historical complaint ticket data through intelligent analysis model to determine the ticket category system.

[0035] Specifically, this application conducts cluster analysis on historical complaint work order data to mine different work order subject categories in historical complaint work order data, thereby constructing a work order category system, and through continuous clustering calculations, maintains the integrity and timeliness of the work order subject categories, thereby ensuring the subject category system of all work orders as the basis for real-time identification of subsequent work order subjects.

[0036] Among them, the work order category system includes: multiple work order subject categories and classification standards and work order levels corresponding to the work order subject categories. The work order levels include at least: first level, second level and third level. Specifically, the work order category system is a work order classification method. Each category has a clear definition and standard that can be used to define the scope of this type of complaint work order and determine the category to which the complaint work order should belong through unified standards. These standards can be based on keywords in the complaint work order, problem type, products or services involved, etc.

[0037] In some embodiments, the intelligent analysis model is constructed based on the following: Usually, the content of the complaint ticket is short and the description is more colloquial. Different billers may describe the same type of problem differently, so the establishment of the subject category of the complaint ticket can serve as the basis for identifying the subject of the complaint ticket. The clustering algorithm used by the intelligent analysis model can integrate multiple clustering calculation methods such as distance clustering and probability clustering, and also calculate from multiple granularities such as words and characters, avoiding errors caused by short ticket content, inconsistent descriptions, etc., so that clustering analysis of historical complaint ticket data can be more accurately achieved.

[0038] In an embodiment of the present application, an intelligent analysis model is used to form a clustering result (i.e., a work order category system) by fusing multiple clustering algorithms, which has higher stability and accuracy. Through a continuous clustering method, a subject category system can be continuously constructed. In addition, the stability brought by the fusion of multiple clustering algorithms and the ability to sustainably construct a subject category system can make the intelligent analysis model suitable for historical complaint work order data in different business type application scenarios, and can complete the construction of the corresponding work order category system.

[0039] For example, the parameter list of the intelligent analysis model may include: area: the region where the work order comes from. consult_filter: 'y' / 'n', whether to filter non-consulting work orders. model: 'hc' / 'kmeans' / 'gmm', clustering model selection. enc_type: 'onehot' / 'w2v', encoding format. max_iter: int value, maximum number of iterations.

[0040] In some embodiments, the intelligent analysis model is a pre-trained model. The model evaluation indicators of the intelligent analysis model may include: (1) CH index (Caliniski-Harabaz): When the clusters are dense and the separation between clusters is good, the higher the Caliniski-Harabaz score, the better the clustering performance. (2) Silhouette Coefficient: The silhouette coefficient of a set of data sets is equal to the average of the silhouette coefficients of each sample in the data set.

[0041] In some embodiments, the intelligent analysis model may be one of the following models: a term frequency-inverse document frequency model, a latent Dirichlet allocation model, a neural network-based text classification model, or an association rule mining model.

[0042] Among them, the term frequency-inverse document frequency (TF-IDF) model, TF (term frequency) can be used to calculate the frequency of a word appearing in the complaint ticket, and TF (term frequency) can reflect the importance of this word in the complaint ticket. IDF (inverse document frequency) can be used to measure the rarity of a word in the entire complaint ticket data. Multiplying TF and IDF gives the TF-IDF value. Words with higher TF-IDF values ​​are usually important words that can represent the subject of the complaint ticket.

[0043] Latent Dirichlet Allocation (LDA) is a topic model. LDA assumes that a complaint ticket is a mixture of multiple topics, and each topic is a probability distribution of a word. LDA can mine potential topics by modeling the word distribution in the complaint ticket.

[0044] Based on the neural network text classification model (for example: TextCNN, Transformer architecture text classification model), first convert the clustered work order text into a vector representation (for example: word embedding vector). Then, use the TextCNN model to classify the complaint work orders. After the training is completed, the number or proportion of complaint work orders in each subject category is counted. Topic categories with a large number or a high proportion can be considered as hot complaint work orders. For the Transformer architecture (such as BERT), the semantic information of the complaint work order can be better captured. After fine-tuning for complaint work order classification, the hot complaint work orders can also be determined based on the distribution of subject categories to further build a work order category system.

[0045] The association rule mining model (e.g., Apriori algorithm) can convert the problem description in the complaint ticket into a transaction form (i.e., each complaint ticket is regarded as a transaction, and the problem description in the complaint ticket is an item in the transaction). Then, the Apriori algorithm is used to mine frequent item sets and association rules. The support (frequency of occurrence) of frequent item sets can help determine the hot complaint tickets, and the confidence of the association rules can determine the association between hot complaint tickets, thereby further determining the association of hot complaint tickets, so as to further build a ticket category system.

[0046] In some embodiments, S101, cluster analysis is performed on historical complaint work order data through an intelligent analysis model to determine a work order category system, which may specifically include the following steps S1011-S1012:

[0047] S1011. Convert each historical complaint work order in the historical complaint work order data into a corresponding text feature representation.

[0048] Specifically, the text feature representation corresponding to the historical complaint ticket can be represented by a word feature vector. For example, the vectors of all words in the historical complaint ticket can be averaged to obtain a word feature vector representing the historical complaint ticket. Alternatively, the word feature vector of the historical complaint ticket can be determined by weighted averaging according to the importance of the word (for example, TF-IDF weight) as its text feature representation.

[0049] Exemplarily, first, the historical complaint worksheets can be preprocessed and the number of preset topic categories can be determined (the number of preset topics can be preset based on prior knowledge or experimental data). Then, the probability distribution of words under each topic category and the probability distribution of each historical complaint worksheet belonging to each topic category are initialized. Next, the probability distribution of words under the topic category and the probability distribution of historical complaint worksheets belonging to the topic category are updated through iterative calculation based on the occurrence results of words in the historical complaint worksheets. For example, if there are more words related to "product failure" and "maintenance" in the historical complaint worksheet, the probability that the historical complaint worksheet belongs to the "product maintenance" topic category will increase. Finally, the word distribution under each topic category and the topic category distribution corresponding to each historical complaint worksheet can be obtained, and these topic category distributions can be used as feature representations of the text.

[0050] S1012. Clustering is performed based on the text feature representation corresponding to each historical complaint ticket using a clustering algorithm through an intelligent analysis model to obtain a ticket category system.

[0051] Among them, the clustering algorithm can integrate multiple clustering algorithms such as distance clustering and probability clustering, thereby improving the accuracy and stability of clustering.

[0052] In some embodiments, cluster analysis of historical complaint ticket data is performed through an intelligent analysis model, and hot issues and corresponding solutions can also be determined. Among them, hot issues are complaint issues whose frequency of occurrence reaches a preset frequency threshold within a preset time period. This type of hot issue has a high frequency and has a corresponding solution. Therefore, the hot issues and corresponding solutions can be sent to the business personnel system so that the business personnel can quickly resolve the complaint issues and improve the efficiency of complaint ticket processing.

[0053] S102. Identify and classify the complaint work orders to be processed based on the work order category system through the intelligent response model, and determine the work order subject category and work order level corresponding to the complaint work orders to be processed.

[0054] Furthermore, based on the work order category system determined in S101, the complaint work orders to be processed can be identified and classified through the intelligent response model to determine their corresponding work order subject categories and work order levels.

[0055] In some embodiments, the construction basis of the intelligent response model is: using the determined work order category system and the text features of the complaint work order and the machine learning classification algorithm, the work order subject recognition of the real-time work order (i.e., the complaint work order to be processed) can be realized to help determine the category of the complaint work order, thereby reducing labor costs and improving the classification efficiency of the complaint work order. At the same time, the knowledge precipitation under the work order category system can be used to cope with subsequent tasks. The intelligent feature mining algorithm is used to handle the needs of classification feature extraction such as industry word discovery and text feature mapping, and at the same time, the variable comparison is controlled to select the appropriate supervised learning classifier to build an intelligent response model to ensure the accuracy of recognition and classification.

[0056] In the embodiment of the present application, the intelligent response model can be used to predict the classification of complaint work orders (pending complaint work orders) in real time. In addition, the classification accuracy is high, and it has strong robustness and fault tolerance to noise neurons. At the same time, the intelligent response model has the ability to be continuously trained, the performance of the intelligent response model can be continuously optimized, and the prediction accuracy can be continuously improved.

[0057] Exemplarily, the parameter list of the smart answer model may include: area: the region where the work order comes from. stopwords: 'y' / 'n', whether to filter stop words (the default is 'y'. keywords_extract_flag: 'y' / 'n', whether the process needs to output keywords (the default is 'y'). new_words_file: file_path / None, whether to add new industry words (the default is None). iter: int value, number of iterations. The model evaluation indicators of the smart answer model can be: acc, F1-score, etc.

[0058] In some embodiments, the above-mentioned intelligent answering model can be one of the following models: a fast text classification FastText model, a text classification convolutional neural network TextCNN model.

[0059] In some embodiments, S102, identifying and classifying the complaint work orders to be processed based on the work order category system through the intelligent response model, and determining the work order subject category and work order level corresponding to the complaint work orders to be processed, may specifically include the following steps S1021-S1022:

[0060] S1021. Perform word embedding feature processing on the complaint worksheet to be processed through a word embedding model to determine the word feature vector corresponding to the complaint worksheet to be processed.

[0061] First, word embedding feature processing is performed on the complaint worksheet to be processed to determine the word feature vector, so that the complaint worksheet to be processed can be more accurately identified and classified according to the word feature vector.

[0062] In some embodiments, the word embedding model is one of the following models: Word2Vec model, GloVe model, BERT model, GPT model.

[0063] Specifically, the Word2Vec model can be trained on large-scale complaint ticket training data to learn the vector representation of each word in the complaint ticket. For example, the CBOW (Continuous Bag of Words) model or the Skip-Gram model can be used for training. For example, the Word2Vec model can be trained on the complaint ticket training data, and the model can learn the vector representation of words such as "product", "fault", and "repair", and these vectors can reflect the semantic relationship between words.

[0064] The GloVe model is also a word embedding model based on global word frequency statistics. The GloVe model uses the co-occurrence information of words in the corpus during training. The word vectors generated by the GloVe model can also capture the semantic similarity between words, and can map semantically similar words to similar vector space positions when processing complaint tickets.

[0065] The BERT model and the GPT model are pre-trained models. The BERT model and the GPT model are pre-trained on large-scale general complaint ticket texts and can learn rich language knowledge and semantic representations. You can directly use the word vectors output by the BERT model and the GPT model, or fine-tune the BERT model and the GPT model to adapt to complaint tickets in different business fields.

[0066] In some embodiments, the evaluation index of the word embedding model may include: using cosine similarity, Euclidean distance and other indicators to evaluate the quality of the word embedding vector. For example, the vector similarity between words with similar semantics (such as "fault" and "problem") is calculated. The higher the similarity, the better the word embedding vector can capture the semantic relationship.

[0067] The optimization of the word embedding model can include: adjusting the parameters, training data or processing methods of the word embedding model according to the evaluation results to improve the quality of the word embedding vector and the effect of subsequent clustering analysis. For example, if the word vector representation of some specific fields is inaccurate, the proportion of these words in the training data can be increased and the word embedding model can be retrained.

[0068] S1022. Classify the word feature vectors corresponding to the complaint work orders to be processed based on the work order category system through the intelligent response model to obtain the work order subject category and work order level corresponding to the complaint work orders to be processed.

[0069] Furthermore, based on the work order category system, the word feature vectors corresponding to the complaint work orders to be processed can be classified through the intelligent response model to determine the work order subject category and work order level corresponding to the complaint work orders to be processed.

[0070] S103. When the work order level corresponding to the complaint work order to be processed is the first level, the first target solution corresponding to the complaint work order to be processed is determined from the first solution database through the intelligent response model based on the work order subject category corresponding to the complaint work order to be processed.

[0071] In an embodiment of the present application, the work order levels include at least: the first level, the second level, and the third level. Among them, the complaint work order corresponding to the first level is a routine business complaint problem, which can be intelligently answered through an intelligent response model. The complaint work order corresponding to the second level is a difficult business complaint problem. The intelligent recommendation model can determine the second target solution corresponding to the complaint work order to be processed from the second solution database, or the system can be manually answered and processed online by business personnel (for example: business experts). The complaint work order corresponding to the third level is a business system problem, which can be answered and processed online by backend support personnel through the backend support personnel processing system. In this way, the hierarchical processing of complaint work orders is realized, the use of expert resources is optimized, and the efficiency of complaint work order processing is improved.

[0072] Specifically, the intelligent response model retrieves and determines the first target solution corresponding to the pending complaint work order from the first solution database based on the work order subject category corresponding to the pending complaint work order. The first solution database is a preset solution knowledge base, which includes solutions to conventional business complaint problems. In addition, the first solution database is a dynamic database, that is, users can add, delete and modify solutions in the first solution database according to actual application needs.

[0073] In one implementation, the intelligent response model can retrieve the first target solution in the first solution database based on the work order subject category corresponding to the complaint work order to be processed and the word segmentation result of the complaint work order to be processed.

[0074] S104. When the work order level corresponding to the complaint work order to be processed is the second level, the complaint work order to be processed is sent to the business personnel processing system, and the second target solution corresponding to the complaint work order to be processed is determined from the second solution database through the intelligent recommendation model based on the work order subject category corresponding to the complaint work order to be processed.

[0075] Furthermore, when the work order level corresponding to the pending complaint work order is the second level, it can be determined that the pending complaint work order is a difficult business complaint problem, that is, this type of difficult business complaint problem cannot be accurately determined in the first solution database. Solution. In this case, the pending complaint work order can be sent to the business personnel processing system, and the intelligent recommendation model determines the second target solution corresponding to the pending complaint work order from the second solution database based on the work order subject category corresponding to the pending complaint work order. In addition, the business personnel (for example: business experts) processing system can also manually and promptly answer and handle online, thereby improving the accuracy and efficiency of complaint work order processing.

[0076] Specifically, when the work order level corresponding to the pending complaint work order is the second level, the second target solution corresponding to the pending complaint work order can be determined from the second solution database through the intelligent recommendation model. Among them, the second solution database is a preset and continuously updated database, and the second solution database includes solutions corresponding to complaint problems of multiple difficult businesses. The solutions can be accumulated based on the solutions of historical complaint work orders. In one implementation method, the second solution database can be constructed based on the knowledge graph.

[0077] In some embodiments, Figure 2 A schematic diagram of a method flow for determining a second target solution provided in an embodiment of the present application. Figure 2 As shown, in S104, based on the work order subject category corresponding to the complaint work order to be processed, the second target solution corresponding to the complaint work order to be processed is determined from the second solution database through the intelligent recommendation model, which may specifically include the following steps S1041-S1044:

[0078] S1041. Search and segment the complaint work order to be processed to obtain the segmentation result.

[0079] For example, the intelligent word segmentation HanLP tool can be used to search and segment the complaint ticket to be processed. Among them, the intelligent word segmentation HanLP tool supports multiple word segmentation matching modes, such as forward maximum matching, reverse maximum matching, etc. Forward maximum matching starts from the beginning of the text of the complaint ticket and searches backwards for the longest word. Reverse maximum matching starts from the end of the text of the complaint ticket and searches forward for the longest word. By combining these matching modes, the accuracy of word segmentation can be further improved.

[0080] S1042. Based on the word segmentation results and the work order subject categories corresponding to the complaint work orders to be processed, determine multiple initial solutions corresponding to the complaint work orders to be processed from the second solution database through an intelligent recommendation model.

[0081] Specifically, first, the intelligent recommendation model can determine all solutions corresponding to the subject category of the work order in the second solution database based on the subject category of the work order corresponding to the complaint work order to be processed. Then, the intelligent recommendation model can match the word segmentation results of the complaint work order to be processed with the word segmentations corresponding to all solutions corresponding to the subject category of the work order, and determine the matching degree of each solution with the complaint work order to be processed. Furthermore, a solution whose matching degree is within a preset matching degree threshold can be selected as the initial solution. Among them, the preset matching degree threshold can be preset based on prior knowledge and actual application requirements, and this application does not make specific limitations on this.

[0082] S1043. Sort multiple initial solutions by a re-sorting algorithm based on the matching degree between the initial solutions and the complaint work orders to be processed.

[0083] Furthermore, the intelligent recommendation model can also sort multiple initial solutions through a re-ranking algorithm according to the degree of match between the initial solutions and the complaint tickets to be processed.

[0084] In some embodiments, the reordering algorithm is one of the following algorithms: RE-Rank algorithm, List-wise algorithm, Pair-wise algorithm.

[0085] Among them, the RE-Rank algorithm will automatically search and sort according to the matching degree, and support manual intervention in sorting knowledge to the front (the top knowledge is placed at the front) - custom scoring formula.

[0086] List-wise algorithms are usually transformed into classification of document pairs, and the classification results are initial solutions with better matching. The goal of List-wise algorithm learning is to reduce the number of incorrectly classified initial solution pairs so that all the sequences of initial solution pairs can be correctly classified. List-wise algorithms are calculated from the perspective of the classification score of a single initial solution, without considering the order relationship between the initial solutions. The training process and training goal of this machine learning method, for example, is to determine the composition of the document pair<D0C1,D0C2> Whether the order relationship is satisfied, that is, whether D0C1 should be placed before DOC2.

[0087] S1044. The initial solution with the greatest matching degree among the multiple initial solutions is used as the second target solution.

[0088] Finally, the initial solution with the highest matching degree with the pending complaint work order indicates that the solution can match the pending complaint work order, that is, the initial solution can be determined as the solution corresponding to the pending complaint work order. In this way, the intelligent recommendation model uses the initial solution with the highest matching degree with the pending complaint work order as the second target solution, which can improve the accuracy of handling complaint work orders for difficult business types.

[0089] In the embodiment of the present application, the intelligent recommendation model can be used to determine the rationality of the order relationship of the initial solution, so as to accurately perform secondary fusion sorting on the initial solution. In addition, the intelligent recommendation model can support custom scoring (i.e., an evaluation method for determining the degree of matching), and can also sort related knowledge according to the sorting rules preset by the user.

[0090] Exemplarily, the parameter list of the intelligent recommendation model may include: area: the region where the work order comes from. n_trees: the number of trees in the model. Learning_rate: the strength of updating parameters of each tree. tree_type: the type of tree used. The model evaluation indicators of the intelligent recommendation model may be: mean average precision (MAP), normalized discounted cumulative gain (NDCG).

[0091] In some embodiments, the intelligent recommendation model may be one of the following models: a rule-based expert system model, a decision tree model, a naive Bayes model, or a neural network model.

[0092] Specifically, the rule-based expert system model can rely on predefined rules. A large number of rules are stored in the knowledge base, and these rules can be determined based on prior knowledge.

[0093] The decision tree model is a tree structure model, in which each internal node represents a test on an attribute, each branch represents a test output, and each leaf node represents a category or a decision result. In the complaint ticket processing, the attributes can be keywords in the complaint ticket, problem type, etc. For example, in a decision tree of a software problem complaint ticket, the root node may be "whether the software can be started normally", if the answer is "no", then go to the next layer of nodes.

[0094] The Naive Bayes model is based on the Bayesian theorem, which assumes that each feature (such as keywords in a complaint ticket) is independent of each other. The Naive Bayesian model selects the most likely solution by calculating the probability of each solution given the complaint ticket features. For example, it is known that there are solutions for complaint ticket A and complaint ticket B in the knowledge base. For a pending complaint ticket, if there are more keywords that mention complaint ticket A, then the probability of calculating the solution of the pending complaint ticket corresponding to complaint ticket A based on the Bayesian theorem is higher.

[0095] Neural network model (e.g., multi-layer perceptron, Transformer architecture model), which consists of an input layer, multiple hidden layers, and an output layer. The input layer receives the feature-extracted vectors of the complaint ticket (e.g., word embedding vectors), the hidden layer performs nonlinear transformation on these vectors, and the output layer outputs the category or specific content of the solution. For example, the text feature vector of the product function failure in the complaint ticket is input into the multi-layer perceptron, and after being processed by the hidden layer, the category or specific content of the corresponding product function repair solution is obtained in the output layer.

[0096] S105. When the work order level corresponding to the unprocessed complaint work order is the third level, the unprocessed complaint work order is sent to the backend support personnel processing system.

[0097] Specifically, when the work order level corresponding to the pending complaint work order is the third level, it can be determined that the pending complaint work order is a business system problem, that is, this type of business system problem cannot be accurately determined in the first solution database and the second solution database. In this case, the pending complaint work order can be sent to the backend support personnel processing system, and the backend support personnel will promptly answer and handle it online through the backend support personnel processing system, thereby improving the accuracy and efficiency of complaint work order processing.

[0098] The method for handling complaint work orders based on artificial intelligence analysis provided in the above-mentioned embodiment of the present application is adopted. This method can perform cluster analysis on historical complaint work order data based on artificial intelligence analysis through an intelligent analysis model to determine a work order category system. The complaint work orders to be processed are identified and classified based on the work order category system through an intelligent response model, and the work order subject category and work order level corresponding to the complaint work orders to be processed are determined to achieve hierarchical processing of complaint work orders, optimize processing resources, and improve processing efficiency. In the case where the work order level corresponding to the complaint work order to be processed is the first level, the first target solution corresponding to the complaint work order to be processed is determined from the first solution database based on the work order subject category corresponding to the complaint work order to be processed through the intelligent response model. In the case where the work order level corresponding to the complaint work order to be processed is the second level, the second target solution corresponding to the complaint work order to be processed is determined from the second solution database based on the work order subject category corresponding to the complaint work order to be processed through the intelligent recommendation model to achieve intelligent question and answer. This method can realize three application scenarios: intelligent question and answer based on intelligent operation and maintenance robots, intelligent recommendation with the goal of accurate solution matching, and intelligent analysis of work order category system based on hot issue mining, thereby improving the efficiency and accuracy of complaint work order processing.

[0099] The present application also provides a complaint ticket processing system based on artificial intelligence analysis. Specifically, Figure 3 A schematic diagram of the structure of the complaint ticket processing system based on artificial intelligence analysis provided in the embodiment of the present application is shown in FIG. Figure 3 As shown, the complaint ticket processing system 300 based on artificial intelligence analysis includes: an intelligent analysis module 310, an intelligent response module 320 and an intelligent recommendation module 330.

[0100] Intelligent analysis module 310 is used to perform cluster analysis on historical complaint work order data through an intelligent analysis model to determine a work order category system; the work order category system includes: multiple work order subject categories and classification standards and work order levels corresponding to the work order subject categories, and the work order levels include at least: first level, second level and third level.

[0101] The intelligent response module 320 is used to identify and classify the complaint work orders to be processed based on the work order category system through the intelligent response model, and determine the work order subject category and work order level corresponding to the complaint work orders to be processed.

[0102] The intelligent response module 320 is also used to determine the first target solution corresponding to the complaint work order to be processed from the first solution database through the intelligent response model based on the work order subject category corresponding to the complaint work order to be processed, when the work order level corresponding to the complaint work order to be processed is the first level.

[0103] The intelligent recommendation module 320 is used to send the unprocessed complaint work order to the business personnel processing system when the work order level corresponding to the unprocessed complaint work order is the second level.

[0104] The intelligent recommendation module 330 is used to determine the second target solution corresponding to the complaint work order to be processed from the second solution database based on the work order subject category corresponding to the complaint work order to be processed through the intelligent recommendation model.

[0105] The intelligent response module 320 is also used to send the unprocessed complaint work order to the backend support personnel processing system when the work order level corresponding to the unprocessed complaint work order is the third level.

[0106] The complaint work order processing system based on artificial intelligence analysis provided by the above-mentioned embodiment of the present application is adopted. The system can perform cluster analysis on historical complaint work order data based on artificial intelligence analysis through an intelligent analysis module using an intelligent analysis model to determine the work order category system. The intelligent response module uses an intelligent response model to identify and classify the complaint work orders to be processed based on the work order category system, and determine the work order theme category and work order level corresponding to the complaint work orders to be processed, so as to realize the hierarchical processing of complaint work orders, optimize processing resources, and improve processing efficiency. In the case where the work order level corresponding to the complaint work order to be processed is the first level, the first target solution corresponding to the complaint work order to be processed is determined from the first solution database based on the work order theme category corresponding to the complaint work order to be processed by the intelligent response model. In the case where the work order level corresponding to the complaint work order to be processed is the second level, the second target solution corresponding to the complaint work order to be processed is determined from the second solution database based on the work order theme category corresponding to the complaint work order to be processed by the intelligent recommendation model, so as to realize intelligent question and answer. This method can effectively improve the efficiency and accuracy of complaint work order processing.

[0107] The embodiment of the present application also provides an electronic device, which may include: a display screen, a memory, and one or more processors. The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device may execute the various methods or steps executed in the above-mentioned complaint work order processing method embodiment based on artificial intelligence analysis. Of course, the electronic device includes but is not limited to the above-mentioned display screen, memory, and one or more processors.

[0108] An embodiment of the present application also provides a computer-readable storage medium for storing computer instructions for running the above-mentioned complaint ticket processing method based on artificial intelligence analysis.

[0109] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0110] In the description of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0111] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0112] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0113] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A method for handling complaint work orders based on artificial intelligence analysis, characterized in that: include: Perform cluster analysis on historical complaint ticket data through intelligent analysis models to determine the ticket category system; The work order category system includes: a plurality of work order subject categories and classification standards and work order levels corresponding to the work order subject categories, and the work order levels include at least: a first level, a second level and a third level; Identify and classify the complaint work orders to be processed based on the work order category system through the intelligent response model, and determine the work order subject category and work order level corresponding to the complaint work orders to be processed; When the work order level corresponding to the to-be-processed complaint work order is the first level, determining a first target solution corresponding to the to-be-processed complaint work order from a first solution database based on the work order subject category corresponding to the to-be-processed complaint work order by using an intelligent response model; When the work order level corresponding to the to-be-processed complaint work order is the second level, the to-be-processed complaint work order is sent to the business personnel processing system, and a second target solution corresponding to the to-be-processed complaint work order is determined from a second solution database based on the work order subject category corresponding to the to-be-processed complaint work order through an intelligent recommendation model; When the work order level corresponding to the unprocessed complaint work order is the third level, the unprocessed complaint work order is sent to the backend support personnel processing system.

2. The method according to claim 1, characterized in that The intelligent analysis model is used to perform cluster analysis on historical complaint ticket data to determine the ticket category system, including: Convert each historical complaint work order in the historical complaint work order data into a corresponding text feature representation; The intelligent analysis model uses a clustering algorithm to perform clustering based on the text feature representation corresponding to each historical complaint work order to obtain the work order category system.

3. The method according to claim 1, characterized in that: The intelligent response model is used to identify and classify the complaint work orders to be processed based on the work order category system, and determine the work order subject category and work order level corresponding to the complaint work orders to be processed, including: Perform word embedding feature processing on the complaint work order to be processed by using a word embedding model to determine a word feature vector corresponding to the complaint work order to be processed; The intelligent response model classifies the word feature vectors corresponding to the complaint work order to be processed based on the work order category system to obtain the work order subject category and work order level corresponding to the complaint work order to be processed.

4. The method according to any one of claims 1 to 3, characterized in that: The determining, from a second solution database, a second target solution corresponding to the complaint work order to be processed based on the work order subject category corresponding to the complaint work order to be processed by the intelligent recommendation model includes: Perform search and segmentation processing on the complaint work order to be processed to obtain the segmentation result; Based on the word segmentation result and the work order subject category corresponding to the unprocessed complaint work order, determine multiple initial solutions corresponding to the unprocessed complaint work order from the second solution database through the intelligent recommendation model; Sorting the multiple initial solutions based on the matching degree between the initial solutions and the complaint work orders to be processed by a re-ranking algorithm; The initial solution with the greatest matching degree among the multiple initial solutions is used as the second target solution.

5. The method according to claim 1, characterized in that The intelligent analysis model is one of the following models: a term frequency-inverse document frequency model, a latent Dirichlet allocation model, a neural network-based text classification model, and an association rule mining model.

6. The method according to claim 1, characterized in that The intelligent answering model is one of the following models: a fast text classification FastText model and a text classification convolutional neural network TextCNN model.

7. The method according to claim 1, characterized in that The intelligent recommendation model is one of the following models: a rule-based expert system model, a decision tree model, a naive Bayes model, and a neural network model.

8. The method according to claim 3, characterized in that The word embedding model is one of the following models: Word2Vec model, GloVe model, BERT model, GPT model.

9. The method according to claim 4, characterized in that The reordering algorithm is one of the following algorithms: RE-Rank algorithm, List-wise algorithm, Pair-wise algorithm.

10. A complaint ticket processing system based on artificial intelligence analysis, characterized in that: include: Intelligent analysis module, intelligent response module and intelligent recommendation module; among them, The intelligent analysis module is used to perform cluster analysis on historical complaint work order data through an intelligent analysis model to determine a work order category system; the work order category system includes: a plurality of work order subject categories and classification standards and work order levels corresponding to the work order subject categories, and the work order levels include at least: a first level, a second level, and a third level; The intelligent response module is used to identify and classify the complaint work orders to be processed based on the work order category system through the intelligent response model, and determine the work order subject category and work order level corresponding to the complaint work orders to be processed; The intelligent response module is further configured to determine, from a first solution database, a first target solution corresponding to the complaint work order to be processed based on the work order subject category corresponding to the complaint work order to be processed by the intelligent response model when the work order level corresponding to the complaint work order to be processed is the first level; The intelligent response module is further configured to send the complaint work order to be processed to a business personnel processing system when the work order level corresponding to the complaint work order to be processed is the second level; The intelligent recommendation module is used to determine the second target solution corresponding to the unprocessed complaint work order from the second solution database based on the work order subject category corresponding to the unprocessed complaint work order through the intelligent recommendation model; The intelligent response module is also used to send the unprocessed complaint work order to the backend support personnel processing system when the work order level corresponding to the unprocessed complaint work order is the third level.

Images