Methods and systems for pinpointing the root causes of mobile internet user complaints

Abstract

This invention discloses a method and system for root cause localization of mobile internet user complaints, belonging to the field of network communication technology. The technical problem it addresses is how to fully utilize multi-dimensional complaint-related data to accurately pinpoint the root cause of user complaints end-to-end. The method includes the following steps: mapping a work order knowledge base to an expert experience base to form a standardized complaint knowledge base; obtaining associated data by linking indicator data, economic attribute data, and the complaint knowledge base, and performing feature and label segmentation on the associated data; constructing a training dataset based on the processed feature data and labels; constructing a root cause complaint prediction model based on the LightGBM machine learning algorithm; training the root cause complaint prediction model based on the training dataset; daily linking of feature data and user mobile phone numbers, predicting and outputting the root cause of the complaint through the trained root cause prediction model, obtaining the root cause of the complaint corresponding to the user's mobile phone number, and analyzing the root cause of the complaint.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, specifically to a method and system for locating the root cause of complaints from mobile internet users. Background Technology

[0002] Currently, user-level problem localization and analysis methods mainly fall into two categories: those based on user signaling data tracing and those based on user metric segmentation and horizontal comparison. Both methods rely on the experience of business experts to identify network anomalies. The difference lies in that segmentation and horizontal comparison based on user metrics bring the experience of business experts online, solidifying specific rules within the IT system. In contrast, tracing based on user signaling data relies on offline analysis by business experts, manually analyzing the signaling content one by one within the time period of the user problem.

[0003] User signaling data tracing is primarily used to trace back user business behavior signaling after a complaint. It relies on signaling error code information to analyze the complaint from multiple levels, including the basic information layer, control plane layer, business plane layer, domain layer, and protocol handshake layer. The basic information layer analysis checks whether the complaining user is infected with a virus, accessed an illegal address, generated detailed billing records, used an abnormal terminal, or used high-traffic services to determine the reason for the complaint. Further, it analyzes the complaint from multiple levels based on signaling error code information.

[0004] The segmentation and horizontal comparison based on user metrics mainly start with user complaints. For different problem types, on the one hand, business experts sort out the set of metrics that are strongly related to the problem and confirm the network nodes where the metrics converge. Based on experience, they sort out the horizontal and vertical comparison rules of the metrics and output the problem location conclusions on the user side, network side, and business side based on the rules. On the other hand, they analyze the metrics on the user side, network side, and business side based on experience and directly output the conclusion that there is a problem with the node based on the degradation of the metrics.

[0005] Existing methods primarily rely on the accumulation of expert experience, and rules need to be summarized and supplemented manually. They lack standardization, automated learning, and rapid iteration capabilities, and also suffer from inaccurate positioning. Furthermore, they cannot quickly acquire data for automatic updates and escalation of complaints related to new businesses. Current problem location and analysis methods require specialized technical personnel for signaling and data analysis, which is time-consuming and unsuitable for time-sensitive scenarios such as frontline handling of online complaints. Moreover, they rely mainly on signaling data, failing to fully utilize the multi-dimensional data related to complaints and thus failing to achieve accurate end-to-end positioning.

[0006] How to fully utilize multi-dimensional data related to complaints to accurately pinpoint the root causes of user complaints end-to-end is a technical problem that needs to be solved. Summary of the Invention

[0007] The technical objective of this invention is to address the above-mentioned shortcomings by providing a method and system for root cause localization of mobile internet user complaints, thereby solving the technical problem of how to fully utilize multi-dimensional data related to complaints to accurately locate the root cause of user complaints end-to-end.

[0008] In a first aspect, the present invention provides a method for root cause localization of complaints from mobile internet users, comprising the following steps:

[0009] Obtain historical complaint work order data, identify and standardize the root causes of complaints hidden in the historical complaint work order data, and build a work order knowledge base based on the extracted historical processing root cause features;

[0010] For each type of complaint, based on the experience of business experts and online complaint experts, the problem phenomenon indicators of the complaint phenomenon under each dimension are analyzed, and the judgment rules and degradation thresholds of the problem phenomenon indicators are defined by experience. Based on the complaint phenomenon, the problem phenomenon indicators of the complaint phenomenon under each dimension, the judgment rules and degradation thresholds corresponding to each problem phenomenon indicator, an expert experience database is constructed.

[0011] By mapping the work order knowledge base to the expert experience base, a standardized complaint knowledge base is formed. The complaint knowledge base includes problem phenomenon indicators under each dimension and the root cause of the complaint under each problem phenomenon indicator.

[0012] The associated data is obtained by linking indicator data, economic attribute data and complaint knowledge base. The associated data is then divided into features and labels. The indicator data and economic attribute data are used as feature data, and the root causes of complaints are used as labels. Feature processing is performed on the feature data to obtain processed feature data. A training dataset is constructed based on the processed feature data and labels.

[0013] A root cause complaint prediction model is constructed based on the LightGBM machine learning algorithm. The root cause complaint prediction model is used to predict user complaints at the daily level.

[0014] The root cause complaint prediction model is trained based on the training dataset to obtain the trained root cause prediction model.

[0015] Every day, feature data and user mobile phone numbers are associated. Feature data is processed to obtain processed feature data. Using the processed feature data as input, a root cause prediction model is trained to predict and output the root cause of the complaint, obtaining the root cause of the complaint corresponding to the user's mobile phone number. The root cause of the complaint is then analyzed to determine the main affected areas and main business operations.

[0016] As a preferred approach, the root causes of complaints implicit in historical complaint work order data are identified and standardized, including the following steps:

[0017] Based on historical complaint work order data, the complaint reasons, handling measures and other complaint handling information are standardized to obtain historical complaint information.

[0018] For historical complaint information, the root cause features of historical processing are extracted using keyword extraction algorithms and short text similarity matching algorithms, and a work order knowledge base is constructed based on the root cause features of historical processing.

[0019] Preferably, feature processing is performed on the feature data, including the following operations:

[0020] The feature data is reduced in dimensionality and noise is removed by using the PCA method to obtain the dimensionality-reduced feature data.

[0021] Data completion is performed on the dimensionality-reduced feature data to fill in missing and outlier values, resulting in completed feature data.

[0022] The completed feature data is then discretized to obtain the discretized feature data.

[0023] The discretized feature data is undersampled / oversampled to obtain balanced feature data.

[0024] As a preferred method, the Lagrange interpolation method is used to complete the feature data after dimensionality reduction.

[0025] As a preferred approach, when training the root cause complaint prediction model based on sample data, a random forest model is constructed. The training dataset is then trained and optimized using the random forest model to obtain the F1 score of the root cause complaint prediction model. Cross-validation of the root cause prediction model is then performed until the F1 score of the root cause complaint prediction model stabilizes, thus obtaining the trained root cause prediction model.

[0026] In a second aspect, the present invention provides a root cause localization system for mobile internet user complaints, used to perform a root cause localization method for mobile internet user complaints as described in any of the first aspects, the system comprising:

[0027] The work order knowledge base construction module is used to acquire historical complaint work order data, identify and standardize the root causes of complaints hidden in the historical complaint work order data, and construct a work order knowledge base based on the extracted historical root cause features.

[0028] The expert experience base construction module is used to analyze the problem phenomenon indicators of each type of complaint phenomenon based on the experience of business experts and network complaint experts, and to define the judgment rules and degradation thresholds of the problem phenomenon indicators through experience. The expert experience base is constructed based on the complaint phenomenon, the problem phenomenon indicators of the complaint phenomenon in each dimension, the judgment rules and degradation thresholds corresponding to each problem phenomenon indicator.

[0029] The complaint knowledge base construction module is used to map the work order knowledge base to the expert experience base to form a standardized complaint knowledge base. The complaint knowledge base includes problem phenomenon indicators under each dimension and the root cause of the complaint under each problem phenomenon indicator.

[0030] The training dataset construction module is used to associate indicator data, economic attribute data and complaint knowledge base to obtain associated data, and to divide the associated data into features and labels. The indicator data and economic attribute data are used as feature data, and the root causes of complaints are used as labels. Feature processing is performed on the feature data to obtain processed feature data. The training dataset is constructed based on the processed feature data and labels.

[0031] The model building and training module is used to build a root cause complaint prediction model based on the LightGBM machine learning algorithm. The root cause complaint prediction model is used to predict user complaints at the daily level. It is also used to train the root cause complaint prediction model based on the training dataset to obtain the trained root cause prediction model.

[0032] The root cause prediction module is used to associate feature data and user mobile phone numbers daily, perform feature processing on the associated feature data to obtain processed feature data, use the processed feature data as input, and predict and output the root cause of the complaint through the trained root cause prediction model to obtain the root cause of the complaint corresponding to the user's mobile phone number. The root cause of the complaint is then analyzed to determine the main affected area and the main affected business of the complaint.

[0033] Preferably, the work order knowledge base construction module is used to perform the following process to identify and standardize the root causes of complaints hidden in historical complaint work order data:

[0034] Based on historical complaint work order data, the complaint reasons, handling measures and other complaint handling information are standardized to obtain historical complaint information.

[0035] For historical complaint information, the root cause features of historical processing are extracted using keyword extraction algorithms and short text similarity matching algorithms, and a work order knowledge base is constructed based on the root cause features of historical processing.

[0036] Preferably, the model building and training module and the root cause prediction module are configured to perform the following feature processing on the feature data, including the following operations:

[0037] The feature data is reduced in dimensionality and noise is removed by using the PCA method to obtain the dimensionality-reduced feature data.

[0038] Data completion is performed on the dimensionality-reduced feature data to fill in missing and outlier values, resulting in completed feature data.

[0039] The completed feature data is then discretized to obtain the discretized feature data.

[0040] The discretized feature data is undersampled / oversampled to obtain balanced feature data.

[0041] Preferably, the model building and training module and the root cause prediction module are used to complete the dimensionality-reduced feature data using the Lagrange interpolation method.

[0042] Preferably, when training the root cause complaint prediction model based on sample data, the model building and training module performs the following: constructing a random forest model, training and optimizing the training dataset using the random forest model to obtain the model F1 value of the root cause complaint prediction model, and performing cross-validation on the root cause prediction model until the model F1 value of the root cause complaint prediction model is stable, thus obtaining the trained root cause prediction model.

[0043] The root cause localization method and system for mobile internet user complaints of the present invention have the following advantages:

[0044] 1. The work order knowledge base and expert experience base are mapped to form a complaint instruction base. Correlation data is obtained by linking indicator data, economic attribute data and complaint knowledge base. Based on the correlation data, a training dataset is constructed. The root cause prediction model is trained on the constructed dataset. The trained root cause prediction model is used to predict and analyze the daily associated feature data to obtain the root cause of the complaint. This can quickly and accurately locate the complaint problem, determine the cause of the complaint, and improve the analysis and processing efficiency and versatility of maintenance personnel.

[0045] 2. A complaint knowledge base is formed by mapping the work order knowledge base and the expert experience base. It combines credibility methods with traditional experience to form an instantiated operation by utilizing the set of various mapping results that experts' experience and the work order knowledge base may make on the complaint factors.

[0046] 3. Data processing was performed on the associated feature data, which reduced the dimensionality of the feature data and filled in missing values, thereby improving the accuracy and speed of subsequent model training.

[0047] 4. When training the root cause prediction model, the training dataset is optimized using a random forest model, and cross-validation is performed on the root cause prediction model to improve the accuracy of the trained root cause prediction model. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] The invention will be further described below with reference to the accompanying drawings.

[0050] Figure 1 This is a schematic diagram of root cause analysis based on signaling data backtracking;

[0051] Figure 2 This is a schematic diagram illustrating segmentation and horizontal comparison based on user metrics;

[0052] Figure 3 This is a flowchart of a method for locating the root cause of user complaints on mobile internet, as described in Example 1.

[0053] Figure 4 This is the construction and analysis process of the expert experience base in the root cause localization method for mobile internet user complaints in Example 1;

[0054] Figure 5 This is a schematic diagram of the Skip-gram and CBOW model structure in Example 1, a method for locating the root cause of complaints from mobile internet users.

[0055] Figure 6 This is a schematic diagram of the Transformer Encoder and BERT structure in a root cause localization method for mobile internet user complaints, as shown in Example 1.

[0056] Figure 7 This is a flowchart illustrating the training process of the root cause prediction model in a root cause localization method for mobile internet user complaints, as described in Example 1.

[0057] Figure 8 This is a schematic diagram of oversampling in a root cause localization method for mobile internet user complaints in Example 1;

[0058] Figure 9 This is a flowchart of the model training based on random forest and cross-validation in Example 1, a method for root cause localization of complaints from mobile internet users.

[0059] Figure 10This is a schematic diagram of cross-validation in a root cause localization method for mobile internet user complaints, as described in Example 1. Detailed Implementation

[0060] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0061] This invention provides a method and system for root cause localization of mobile internet user complaints, which addresses the technical problem of how to fully utilize multi-dimensional data related to complaints to accurately locate the root cause of user complaints end-to-end.

[0062] Example 1:

[0063] This invention provides a method for root cause localization of complaints from mobile internet users, comprising the following steps:

[0064] S100. Obtain historical complaint work order data, identify and standardize the root causes of complaints hidden in the historical complaint work order data, and build a work order knowledge base based on the extracted historical processing root cause features.

[0065] S200. For each type of complaint, based on the experience of business experts and network complaint experts, analyze the problem phenomenon indicators of the complaint in each dimension, and define the judgment rules and degradation thresholds of the problem phenomenon indicators through experience. Construct an expert experience database based on the complaint, the problem phenomenon indicators of the complaint in each dimension, the judgment rules and degradation thresholds corresponding to each problem phenomenon indicator.

[0066] S300. Map the work order knowledge base to the expert experience base to form a standardized complaint knowledge base. The complaint knowledge base includes problem phenomenon indicators under each dimension and the root cause of the complaint under each problem phenomenon indicator.

[0067] S400, associated indicator data, economic attribute data and complaint knowledge base are used to obtain associated data, and the associated data is divided into features and labels. The indicator data and economic attribute data are used as feature data, and the root cause of the complaint is used as the label. Feature processing is performed on the feature data to obtain processed feature data. A training dataset is constructed based on the processed feature data and labels.

[0068] S500. A root cause complaint prediction model is constructed based on the LightGBM machine learning algorithm. The root cause complaint prediction model is used to predict user complaints at the daily level.

[0069] The root cause complaint prediction model is trained based on the training dataset to obtain the trained root cause prediction model.

[0070] S600 associates daily feature data with user mobile phone numbers, performs feature processing on the associated feature data to obtain processed feature data, uses the processed feature data as input, and predicts and outputs the root cause of the complaint through a trained root cause prediction model to obtain the root cause of the complaint corresponding to the user's mobile phone number, and analyzes the root cause of the complaint to analyze the main affected areas and main affected businesses.

[0071] In this embodiment, step S100 identifies and standardizes the root causes of complaints hidden in historical complaint work order data, including the following steps:

[0072] (1) Historical complaint work order data: The complaint reasons, handling measures and other complaint handling information are standardized to obtain historical complaint information;

[0073] (2) For historical complaint information, the root cause features of historical processing are extracted by keyword extraction algorithm and short text similarity matching algorithm, and a work order knowledge base is constructed based on the root cause features of historical processing.

[0074] Among them, the short text similarity matching uses Word2Vec-based similarity calculation.

[0075] In NLP, words are abstract summaries of human thought, and they are in symbolic form (such as Chinese, English, Latin, etc.). They need to be converted into numerical form and embedded into a mathematical space. This embedding method is called word embedding, and Word2vec is a model of word embedding.

[0076] Word2Vec includes two language models: Skip-gram and CBOW. The Skip-gram model uses a word as input to predict its surrounding context; the CBOW model uses the context of a word as input to predict the word itself. The model structure is as follows: Figure 5 As shown.

[0077] The hidden variables obtained after training a language model are word vectors.

[0078] Generally, the word vectors we use are pre-trained. In text similarity calculation, the average of the word vectors of all words in a sentence can be used as the sentence's vector representation. Then, methods such as cosine similarity are used to calculate the similarity between two different sentences in the vector space, representing the semantic similarity of the texts.

[0079] Essentially, BERT learns a good semantic representation for text through self-supervised learning on a massive corpus. Unlike previous methods that used traditional one-way language models or shallow concatenation of two one-way language models for pre-training, it uses a new masked language model (MLM) that can generate deep bidirectional language representations.

[0080] BERT is made up of multiple Transformer Encoders stacked one on top of another, and Transformer is a self-attention network.

[0081] The main input to the BERT model is the original word vectors of each character / word in the text. These vectors can be randomly initialized or pre-trained using algorithms such as Word2Vector as initial values. In addition to the word vectors, the model input includes two other parts: text vectors and position vectors. The values ​​of the text vectors are automatically learned during model training and are used to characterize the global semantic information of the text, which is then fused with the semantic information of individual characters / words. Because the semantic information carried by characters / words appearing in different positions in the text differs (e.g., "I love you" and "you love me"), the BERT model attaches a different position vector to characters / words at different positions for differentiation.

[0082] The output of the BERT model is a vector representation of each character / word in the text after incorporating the semantic information of the entire text.

[0083] Similar to Word2Vec, when calculating text similarity, methods such as cosine similarity are used to calculate the similarity between two different sentences in the vector space, which represents the semantic similarity of the text.

[0084] TextRank, a keyword extraction algorithm, is an extractive unsupervised text summarization method that can extract keywords and keyword phrases from a given text and use extractive automatic summarization methods to extract key sentences from the text.

[0085] The TextRank algorithm process is as follows:

[0086] (1) Divide the given text T into complete sentences, i.e., T = [S1, S2, ..., S...]. m ]

[0087] (2) For each sentence S i ∈T, perform word segmentation and part-of-speech tagging, filter out stop words, and retain only words with the specified part of speech, such as nouns, verbs, and adjectives, i.e., S. i =[t i，1 , t i，2 , ..., tt，n ], where t i，j These are the retained candidate keywords.

[0088] (3) Construct a candidate keyword graph G = (V, E), where V is a set of nodes composed of candidate keywords generated in the previous step. Then, construct an edge between any two nodes using co-occurrence relations. An edge exists between two nodes only if their corresponding words co-occur in a window of length k, where k represents the window size, i.e., at most k words co-occur.

[0089] (4) According to the following formula, iteratively propagate the weights of each node until convergence. Where wji is the keyword similarity.

[0090]

[0091] (5) The node weights are sorted in reverse order to obtain the T most important words as candidate keywords.

[0092] (6) The most important T words obtained from the previous step are marked in the original text. If they form adjacent word groups, they are combined into multi-word keywords.

[0093] Most broadly defined complaint ticket data records the complaint handling process in detail, containing key information about the root causes of user complaints, such as weak coverage, interference, and base station failures. The first step is to identify and standardize the root causes of complaints implicit in these tickets. By leveraging expert experience and using keyword extraction and short text similarity matching algorithms, historical processing root cause features are extracted to form a ticket knowledge base. The specific structure of the ticket knowledge base is shown in Table 1.

[0094] Table 1

[0095]

[0096]

[0097] Step S200, leveraging the experience of business experts and network complaint handling experts, identifies key indicators of complaint phenomena. For each complaint type, including "no signal on the terminal," "unable to access the internet," "able to access the internet but at a slow speed," and "frequent disconnections," it outputs problem localization rules across terminal, network, and cloud dimensions. This includes the possible dimensions involved in the user-reported problem and the indicators causing each dimension. Degradation thresholds for these indicators are defined based on experience. Furthermore, each dimension's internal elements are further refined; the network side can be further subdivided into wireless and core sides, etc. Specifically, as follows... Figure 2 As shown.

[0098] Based on the user's Internet access signaling process, the analysis is performed step by step according to the six stages of Internet access (network attachment / bearer establishment / TAU update / domain name query / connection establishment / service usage).

[0099] This study delves into key performance indicators (KQIs) for user internet access from both the control and business perspectives, extracting key indicators from the signaling process. Ultimately, it translates user perception into metrics such as "success rate," "latency," and "speed," pinpointing issues to terminal problems, wireless problems, bearer problems, core problems, and service provider (SP) problems, thereby achieving the goal of accurately locating fault points. The resulting expert experience database is shown in Table 2.

[0100] Table 2

[0101]

[0102]

[0103] The work order knowledge base itself is a standardized accumulation of online complaint processing results, which has certain deviations in the accuracy of root causes and needs to be corrected. At the same time, the classification of root causes of online complaints is relatively messy and needs to be integrated. The expert experience base is a root cause positioning rule based on actual business logic. Step S300 maps the work order knowledge base to the expert experience base. On the one hand, it can correct the root causes of the work order knowledge base, and on the other hand, it can reduce the dimensionality of the complaint root cause classification, and finally form a standardized complaint knowledge base, as shown in Table 3.

[0104] Table 3

[0105]

[0106]

[0107]

[0108] After the complaint feature database is built, the root cause standardization processing of batch complaint work orders needs to be carried out to form sample data for the complaint root cause analysis model. At the same time, for the content of user complaints, it is necessary to sort out user-level impact indicators, involving the entire process of user business, including network registration, business initiation, business operation, and business termination. It is also necessary to consider customer service, billing and other data that affect user satisfaction. The specific scope includes XDR data, network management data, network optimization data, dial-up test data, alarm data, business analysis data, etc.

[0109] In this embodiment, step S400 involves associating indicator data, economic attribute data, and the complaint knowledge base. The associated data is then divided into features and tags; that is, the indicator and economic attribute data are feature data, and the root causes of complaints are tag data. The acquired feature data is then processed, including feature discretization, component analysis, and difference processing. The indicator data includes daily HTTP data, daily MME data, and network management data, while the economic attribute data consists of monthly user economic data.

[0110] In this embodiment, feature processing is performed on the feature data, including the following operations:

[0111] (1) The feature data is reduced in dimensionality and noise is removed by PCA to obtain the reduced feature data;

[0112] (2) Perform data completion on the dimension-reduced feature data. The missing values ​​and outliers are filled in by data completion to obtain the completed feature data.

[0113] (3) Perform feature discretization on the completed feature data to obtain the discretized feature data;

[0114] (4) Perform undersampling / oversampling on the discretized feature data to obtain balanced feature data.

[0115] By reducing the dimensionality of high-dimensional features, noise is removed, and patterns in the data are discovered. PCA replaces the original n features with a smaller number of m features. The new features are linear combinations of the old features, and these linear combinations maximize the sample variance and make the new m features as uncorrelated as possible. The mapping from old features to new features captures the inherent variability in the data. The transformation process is shown in the figure below. While achieving dimensionality reduction, it also reduces the influence of noise information and the probability of model overfitting.

[0116] The original data contains a large amount of incomplete and outlier data, which seriously affects the model results. Therefore, it is necessary to impute missing and outlier values ​​before inputting the data into the model. This solution mainly uses the Lagrange interpolation method to impute missing values. The imputation formula is:

[0117] P n-1 (x)=y1L1(x)+…+y n L n (x)

[0118] Such a polynomial P n-1 (x), satisfying P n-1 (x i )=y i i = 1, 2, 3, ..., n

[0119] Continuous data is discretized before being fed into the model. Discretization can be understood as the process of extracting features. Since the expressive power of a generalized linear model is limited, feature discretization is used to improve nonlinear learning capabilities.

[0120] Because complaining users represent a small percentage of all users, the associated data exhibits data skew, necessitating data distribution balancing. This solution primarily employs oversampling of complaining users to address this issue. The oversampling method is as follows: Figure 8 As shown.

[0121] The oversampling formula is: xnew=x+rand(0,1)*|x-xn|.

[0122] After performing feature processing on the associated feature data, a training dataset is constructed based on the processed feature data and labels.

[0123] In step S500, when training the root cause prediction model using the above training dataset, a random forest model is constructed. The training dataset is then trained and optimized using the random forest model to obtain the F1 score of the root cause complaint prediction model. Cross-validation is then performed on the root cause prediction model until the F1 score of the root cause complaint prediction model stabilizes, thus obtaining the trained root cause prediction model.

[0124] like Figure 9 As shown, a random forest model is constructed, the training dataset is trained and optimized, and the corresponding model file and feature importance ranking for training metrics are output.

[0125] The model needs to be optimized by adjusting the parameters to address the underfitting problem during model training. The main optimization parameters are shown in Table 4.

[0126] Table 4

[0127]

[0128] By iterating through each parameter value, the F1 score of the model is calculated, and the optimal F1 score model is obtained as the training output.

[0129] Cross-validation is necessary for root cause prediction models to address overfitting during training. The main steps of cross-validation are: dividing the training dataset into k parts, selecting one part without replacement as the validation dataset, and conducting multiple experiments. If the model's F1 score remains stable, it indicates that the model is not overfitting. The cross-validation process is as follows: Figure 10 As shown.

[0130] After training, the system outputs two parts: a post-training prediction model and an indicator importance analysis. The prediction model is responsible for predicting the access data and outputting predicted data for the root cause classification of user complaints. The indicator analysis provides support for indicator selection in subsequent indicator analysis. The importance of features is shown in Table 5.

[0131] Table 5

[0132]

[0133]

[0134] Finally, complaint prediction and application are performed. The daily-level correlation data is input into the LightGBM model, which outputs the user's phone number and the predicted root cause label for the complaint. Before analyzing the daily-level correlation data using the trained root cause prediction model, feature processing is performed on the daily-level correlation data, in the same manner as the feature processing performed on the training dataset during model building and training.

[0135] The method in this embodiment is based on artificial intelligence technology. It automatically learns the content features of the root causes of user complaints, updates them dynamically, and iterates rapidly to locate the root causes. It performs root cause location for user internet-related complaints by region. The method mainly involves the construction of a work order knowledge base, the construction of an expert experience base, the construction of a complaint knowledge base, root cause feature analysis, training of an intelligent recommendation model for root cause location, and real-time root cause location of complaints.

[0136] By leveraging artificial intelligence technology, historical work order data is automatically learned, and the complaint reasons, handling measures, and other complaint information are standardized. Based on keyword extraction and short text similarity matching algorithms, historical handling measures features are extracted to form category features and construct fault labels. This enables AI-based keyword extraction of complaint data without relying on manual sorting, allowing for rapid iteration and dynamic updates. It can also perform efficient, comprehensive, and collaborative detection on text-based data.

[0137] The complaint knowledge base is formed by mapping the work order knowledge base and the expert experience base. It combines credibility methods with traditional experience to form an instantiated operation by utilizing the set of various mapping results that experts and the work order knowledge base may make on the complaint factors.

[0138] By applying artificial intelligence technology and introducing machine learning feature engineering, this method effectively distinguishes key indicators affecting complaints from massive and multidimensional XDR data based on different business types, eliminating irrelevant or redundant features. This improves problem convergence and, through an intelligent analysis process, allows complaint handlers to focus on key indicators that negatively impact user perception. The model input combines the LGBM algorithm with manually labeled data to automatically learn key features of complaint data, outputting the top root causes of complaints and their probabilities, thus increasing the diversity of model training samples.

[0139] Example 2:

[0140] The present invention provides a root cause localization system for mobile internet user complaints, comprising a work order knowledge base construction module, an expert experience base construction module, a complaint knowledge base construction module, a training dataset construction module, a model construction and training module, and a root cause prediction module. The system can execute the method disclosed in Embodiment 1.

[0141] The work order knowledge base construction module is used to obtain historical complaint work order data, identify and standardize the root causes of complaints hidden in the historical complaint work order data, and construct a work order knowledge base based on the extracted historical processing root cause features.

[0142] The work order knowledge base construction module is used to perform the following tasks: identifying and standardizing the root causes of complaints hidden in historical complaint work order data:

[0143] (1) Historical complaint work order data: The complaint reasons, handling measures and other complaint handling information are standardized to obtain historical complaint information;

[0144] (2) For historical complaint information, the root cause features of historical processing are extracted by keyword extraction algorithm and short text similarity matching algorithm, and a work order knowledge base is constructed based on the root cause features of historical processing.

[0145] Among them, the short text similarity matching uses Word2Vec-based similarity calculation.

[0146] In NLP, words are abstract summaries of human thought, and they are in symbolic form (such as Chinese, English, Latin, etc.). They need to be converted into numerical form and embedded into a mathematical space. This embedding method is called word embedding, and Word2vec is a model of word embedding.

[0147] Word2Vec includes two language models: Skip-gram and CBOW. The Skip-gram model uses a word as input to predict its surrounding context; the CBOW model uses the context of a word as input to predict the word itself. The model structure is as follows: Figure 5 As shown.

[0148] The hidden variables obtained after training a language model are word vectors.

[0149] Generally, the word vectors we use are pre-trained. In text similarity calculation, the average of the word vectors of all words in a sentence can be used as the sentence's vector representation. Then, methods such as cosine similarity are used to calculate the similarity between two different sentences in the vector space, representing the semantic similarity of the texts.

[0150] Essentially, BERT learns a good semantic representation for text through self-supervised learning on a massive corpus. Unlike previous methods that used traditional one-way language models or shallow concatenation of two one-way language models for pre-training, it uses a new masked language model (MLM) that can generate deep bidirectional language representations.

[0151] BERT is made up of multiple Transformer Encoders stacked one on top of another, and Transformer is a self-attention network.

[0152] The main input to the BERT model is the original word vectors of each character / word in the text. These vectors can be randomly initialized or pre-trained using algorithms such as Word2Vector as initial values. In addition to the word vectors, the model input includes two other parts: text vectors and position vectors. The values ​​of the text vectors are automatically learned during model training and are used to characterize the global semantic information of the text, which is then fused with the semantic information of individual characters / words. Because the semantic information carried by characters / words appearing in different positions in the text differs (e.g., "I love you" and "you love me"), the BERT model attaches a different position vector to characters / words at different positions for differentiation.

[0153] The output of the BERT model is a vector representation of each character / word in the text after incorporating the semantic information of the entire text.

[0154] Similar to Word2Vec, when calculating text similarity, methods such as cosine similarity are used to calculate the similarity between two different sentences in the vector space, which represents the semantic similarity of the text.

[0155] TextRank, a keyword extraction algorithm, is an extractive unsupervised text summarization method that can extract keywords and keyword phrases from a given text and use extractive automatic summarization methods to extract key sentences from the text.

[0156] The TextRank algorithm process is as follows:

[0157] (1) Divide the given text T into complete sentences, i.e., T = [S1, S2, ..., S...]. m ]

[0158] (2) For each sentence S i ∈T, perform word segmentation and part-of-speech tagging, filter out stop words, and retain only words with the specified part of speech, such as nouns, verbs, and adjectives, i.e., S. i =[t i，1, t i，2 , ..., t t，n ], where t i，j These are the retained candidate keywords.

[0159] (3) Construct a candidate keyword graph G = (V, E), where V is a set of nodes composed of candidate keywords generated in the previous step. Then, construct an edge between any two nodes using co-occurrence relations. An edge exists between two nodes only if their corresponding words co-occur in a window of length k, where k represents the window size, i.e., at most k words co-occur.

[0160] (4) According to the following formula, iteratively propagate the weights of each node until convergence. Where wji is the keyword similarity.

[0161]

[0162] (5) The node weights are sorted in reverse order to obtain the T most important words as candidate keywords.

[0163] (6) The most important T words obtained from the previous step are marked in the original text. If they form adjacent word groups, they are combined into multi-word keywords.

[0164] Most of the broad complaint ticket data records the complaint handling process in detail and contains key information about the root causes of user complaints, such as weak coverage, interference, base station failure, etc. The first step is to identify and standardize the root causes of complaints hidden in these tickets. With the help of expert experience, based on keyword extraction and short text similarity matching algorithms, the characteristics of historical processing root causes are extracted to form a ticket knowledge base.

[0165] The expert experience base construction module is used to analyze the problem phenomenon indicators of each type of complaint phenomenon under each dimension based on the experience of business experts and network complaint experts. It also defines the judgment rules and degradation thresholds of the problem phenomenon indicators through experience, and constructs an expert experience base based on the complaint phenomenon, the problem phenomenon indicators of the complaint phenomenon under each dimension, the judgment rules and degradation thresholds corresponding to each problem phenomenon indicator.

[0166] In this embodiment, the expert experience base construction module, through the accumulated experience of business experts and network complaint handling experts, sorts out the obvious indicator characteristics of complaint phenomena. For each type of complaint, including "no signal on the terminal", "cannot access the Internet", "can access the Internet but the speed is slow", "frequent Internet disconnection", etc., it outputs the problem location rules according to the dimensions of terminal, network, and cloud. This includes the possible dimensions involved in the problem phenomenon reported by users, as well as the indicators that cause the phenomenon in each dimension. The degradation threshold of the indicators is defined by experience. At the same time, the elements within each dimension are also further refined. The network side can be further subdivided into wireless side, core side, etc.

[0167] Based on the user's Internet access signaling process, the analysis is performed step by step according to the six stages of Internet access (network attachment / bearer establishment / TAU update / domain name query / connection establishment / service usage).

[0168] By drilling down into the control plane and business plane, we inject research into the key KQI indicators of users' Internet access, extract key indicators of signaling process, and finally put the user's perception into related indicators such as "success rate", "latency" and "speed", and locate terminal problems, wireless problems, bearer problems, core problems and SP problems, so as to achieve the goal of accurately locating the fault point.

[0169] The complaint knowledge base construction module is used to map the work order knowledge base to the expert experience base to form a standardized complaint knowledge base. The complaint knowledge base includes problem phenomenon indicators under each dimension and the root cause of the complaint under each problem phenomenon indicator.

[0170] The work order knowledge base itself is a standardized accumulation of front-line online complaint processing results, which has certain deviations in the accuracy of root causes and needs to be corrected. At the same time, the classification of root causes of online complaints is relatively disorganized and needs to be integrated. The expert experience base, on the other hand, is a root cause positioning rule based on actual business logic. The complaint knowledge base construction module maps the work order knowledge base to the expert experience base. On the one hand, it can correct the root causes of the work order knowledge base, and on the other hand, it can reduce the dimensionality of the complaint root cause classification, ultimately forming a standardized complaint knowledge base.

[0171] The training dataset construction module is used to associate indicator data, economic attribute data, and complaint knowledge base to obtain associated data. It then divides the associated data into features and labels, using indicator data and economic attribute data as feature data and complaint root causes as labels. The feature data is then processed to obtain processed feature data, and the training dataset is constructed based on the processed feature data and labels.

[0172] After the complaint feature database is built, the root cause standardization processing of batch complaint work orders needs to be carried out to form sample data for the complaint root cause analysis model. At the same time, for the content of user complaints, it is necessary to sort out user-level impact indicators, involving the entire process of user business, including network registration, business initiation, business operation, and business termination. It is also necessary to consider customer service, billing and other data that affect user satisfaction. The specific scope includes XDR data, network management data, network optimization data, dial-up test data, alarm data, business analysis data, etc.

[0173] In this embodiment, the training dataset construction module is used to associate indicator data, economic attribute data, and the complaint knowledge base. The associated data is then divided into features and labels; that is, the indicator and economic attribute data are feature data, and the root causes of complaints are label data. The acquired feature data is then processed, including feature discretization, component analysis, and difference processing. The indicator data includes daily HTTP data, daily MME data, and network management data, while the economic attribute data consists of monthly user economic data.

[0174] In this embodiment, the training dataset construction module is used to perform the following feature processing on the feature data:

[0175] (1) The feature data is reduced in dimensionality and noise is removed by PCA to obtain the reduced feature data;

[0176] (2) Perform data completion on the dimension-reduced feature data. The missing values ​​and outliers are filled in by data completion to obtain the completed feature data.

[0177] (3) Perform feature discretization on the completed feature data to obtain the discretized feature data;

[0178] (4) Perform undersampling / oversampling on the discretized feature data to obtain balanced feature data.

[0179] By reducing the dimensionality of high-dimensional features, noise is removed, and patterns in the data are discovered. PCA replaces the original n features with a smaller number of m features. The new features are linear combinations of the old features, and these linear combinations maximize the sample variance and make the new m features as uncorrelated as possible. The mapping from old features to new features captures the inherent variability in the data. The transformation process is shown in the figure below. While achieving dimensionality reduction, it also reduces the influence of noise information and the probability of model overfitting.

[0180] The original data contains a large amount of incomplete and outlier data, which seriously affects the model results. Therefore, it is necessary to impute missing and outlier values ​​before inputting the data into the model. This solution mainly uses the Lagrange interpolation method to impute missing values. The imputation formula is:

[0181] P n-1 (x)=y1L1(x)+…+y n L n (x)

[0182] Such a polynomial P n-1 (x), satisfying P n-1 (x i )=y i i = 1, 2, 3, ..., n

[0183] Continuous data is discretized before being fed into the model. Discretization can be understood as the process of extracting features. Since the expressive power of a generalized linear model is limited, feature discretization is used to improve nonlinear learning capabilities.

[0184] Because complaining users represent a small percentage of all users, the associated data exhibits data skew, necessitating data distribution balancing. This solution primarily employs oversampling of complaining users to address this issue. The oversampling method is as follows: Figure 8 As shown.

[0185] The oversampling formula is: xnew=x+rand(0,1)*|x-xn|.

[0186] After performing feature processing on the associated feature data, a training dataset is constructed based on the processed feature data and labels.

[0187] The model building and training module is used to build a root cause complaint prediction model based on the LightGBM machine learning algorithm. The root cause complaint prediction model is used to predict user complaints at the daily level. It is also used to train the root cause complaint prediction model based on the training dataset to obtain the trained root cause prediction model.

[0188] When training the root cause prediction model using the above training dataset, the model building and training module is used to perform the following: build a random forest model, train and optimize the training dataset using the random forest model to obtain the F1 score of the root cause complaint prediction model, and perform cross-validation on the root cause prediction model until the F1 score of the root cause complaint prediction model is stable, thus obtaining the trained root cause prediction model.

[0189] In this embodiment, a random forest model is constructed, the training dataset is trained and optimized, and the corresponding model file and feature importance ranking for training metrics are output.

[0190] The model needs parameter tuning to address the underfitting problem during model training.

[0191] By iterating through each parameter value, the F1 score of the model is calculated, and the optimal F1 score model is obtained as the training output.

[0192] Cross-validation is required for root cause prediction models to address overfitting during model training. The main steps of cross-validation are to divide the training dataset into k parts, select one part without replacement as the validation dataset, and conduct multiple experiments. If the F1 score of the model is stable, it indicates that the model does not have an overfitting problem.

[0193] After training, there are two outputs: a post-training prediction model and an indicator importance analysis. The prediction model is responsible for predicting the access data and outputting predicted data for the root cause classification of user complaints. The indicator analysis is responsible for providing indicator selection support for subsequent indicator analysis.

[0194] The root cause prediction module is used to associate feature data and user mobile phone numbers daily, perform feature processing on the associated feature data to obtain processed feature data, use the processed feature data as input, and predict and output the root cause of the complaint through the trained root cause prediction model to obtain the root cause of the complaint corresponding to the user's mobile phone number. The root cause of the complaint is then analyzed to determine the main affected area and the main business affected by the complaint.

[0195] In this embodiment, the root cause prediction module is used to input daily-level correlation data into the LightGBM model and output the user's mobile phone number and the predicted root cause label of the complaint. Specifically, the root cause prediction module performs feature processing on the daily-level correlation data, and the processing method is consistent with the feature processing of the training dataset by the model building and training module.

[0196] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments. These embodiments are also within the protection scope of the present invention.

Claims

1. A method for root cause localization of mobile internet user complaints, characterized in that, Includes the following steps: Obtain historical complaint work order data, identify and standardize the root causes of complaints hidden in the historical complaint work order data, and build a work order knowledge base based on the extracted historical processing root cause features; For each type of complaint, based on the experience of business experts and online complaint experts, the problem phenomenon indicators of the complaint phenomenon under each dimension are analyzed, and the judgment rules and degradation thresholds of the problem phenomenon indicators are defined by experience. Based on the complaint phenomenon, the problem phenomenon indicators of the complaint phenomenon under each dimension, the judgment rules and degradation thresholds corresponding to each problem phenomenon indicator, an expert experience database is constructed. By mapping the work order knowledge base to the expert experience base, a standardized complaint knowledge base is formed. The complaint knowledge base includes problem phenomenon indicators under each dimension and the root cause of the complaint under each problem phenomenon indicator. The associated data is obtained by linking indicator data, economic attribute data and complaint knowledge base. The associated data is then divided into features and labels. The indicator data and economic attribute data are used as feature data, and the root causes of complaints are used as labels. Feature processing is performed on the feature data to obtain processed feature data. A training dataset is constructed based on the processed feature data and labels. A root cause complaint prediction model is constructed based on the LightGBM machine learning algorithm. The root cause complaint prediction model is used to predict user complaints at the daily level. The root cause complaint prediction model is trained based on the training dataset to obtain the trained root cause prediction model. Every day, feature data and user mobile phone numbers are associated. Feature data is processed to obtain processed feature data. Using the processed feature data as input, a root cause prediction model is trained to predict and output the root cause of the complaint, obtaining the root cause of the complaint corresponding to the user's mobile phone number. The root cause of the complaint is then analyzed to determine the main affected areas and main business operations.

2. The root cause localization method for mobile internet user complaints according to claim 1, characterized in that, Identifying and standardizing the root causes of complaints hidden in historical complaint work order data includes the following steps: Based on historical complaint work order data, the complaint reasons, handling measures and other complaint handling information are standardized to obtain historical complaint information. For historical complaint information, the root cause features of historical processing are extracted using keyword extraction algorithms and short text similarity matching algorithms, and a work order knowledge base is constructed based on the root cause features of historical processing.

3. The root cause localization method for mobile internet user complaints according to claim 1, characterized in that, Feature processing of feature data includes the following operations: The feature data is reduced in dimensionality and noise is removed by using the PCA method to obtain the dimensionality-reduced feature data. Data completion is performed on the dimensionality-reduced feature data to fill in missing and outlier values, resulting in completed feature data. The completed feature data is then discretized to obtain the discretized feature data. The discretized feature data is undersampled / oversampled to obtain balanced feature data.

4. The root cause localization method for mobile internet user complaints according to claim 3, characterized in that, The Lagrange interpolation method is used to complete the feature data after dimensionality reduction.

5. The root cause localization method for mobile internet user complaints according to any one of claims 1-4, characterized in that, When training the root cause complaint prediction model based on sample data, a random forest model is constructed. The training dataset is then trained and optimized using the random forest model to obtain the F1 score of the root cause complaint prediction model. Cross-validation of the root cause prediction model is then performed until the F1 score of the root cause complaint prediction model stabilizes, resulting in the trained root cause prediction model.

6. A root cause localization system for mobile internet user complaints, characterized in that, The system is used to perform a root cause localization method for mobile internet user complaints as described in any one of claims 1-5, the system comprising: The work order knowledge base construction module is used to acquire historical complaint work order data, identify and standardize the root causes of complaints hidden in the historical complaint work order data, and construct a work order knowledge base based on the extracted historical root cause features. The expert experience base construction module is used to analyze the problem phenomenon indicators of each type of complaint phenomenon based on the experience of business experts and network complaint experts, and to define the judgment rules and degradation thresholds of the problem phenomenon indicators through experience. The expert experience base is constructed based on the complaint phenomenon, the problem phenomenon indicators of the complaint phenomenon in each dimension, the judgment rules and degradation thresholds corresponding to each problem phenomenon indicator. The complaint knowledge base construction module is used to map the work order knowledge base to the expert experience base to form a standardized complaint knowledge base. The complaint knowledge base includes problem phenomenon indicators under each dimension and the root cause of the complaint under each problem phenomenon indicator. The training dataset construction module is used to associate indicator data, economic attribute data and complaint knowledge base to obtain associated data, and to divide the associated data into features and labels. The indicator data and economic attribute data are used as feature data, and the root causes of complaints are used as labels. Feature processing is performed on the feature data to obtain processed feature data. The training dataset is constructed based on the processed feature data and labels. The model building and training module is used to build a root cause complaint prediction model based on the LightGBM machine learning algorithm. The root cause complaint prediction model is used to predict user complaints at the daily level. It is also used to train the root cause complaint prediction model based on the training dataset to obtain the trained root cause prediction model. The root cause prediction module is used to associate feature data and user mobile phone numbers daily, perform feature processing on the associated feature data to obtain processed feature data, use the processed feature data as input, and predict and output the root cause of the complaint through the trained root cause prediction model to obtain the root cause of the complaint corresponding to the user's mobile phone number. The root cause of the complaint is then analyzed to determine the main affected area and the main affected business of the complaint.

7. The root cause localization system for mobile internet user complaints according to claim 6, characterized in that, The work order knowledge base construction module is used to perform the following tasks: identifying and standardizing the root causes of complaints hidden in historical complaint work order data: Based on historical complaint work order data, the complaint reasons, handling measures and other complaint handling information are standardized to obtain historical complaint information. For historical complaint information, the root cause features of historical processing are extracted using keyword extraction algorithms and short text similarity matching algorithms, and a work order knowledge base is constructed based on the root cause features of historical processing.

8. The root cause localization system for mobile internet user complaints according to claim 6, characterized in that, The model building and training module and the root cause prediction module are used to perform the following feature processing on the feature data, including the following operations: The feature data is reduced in dimensionality and noise is removed by using the PCA method to obtain the dimensionality-reduced feature data. Data completion is performed on the dimensionality-reduced feature data to fill in missing and outlier values, resulting in completed feature data. The completed feature data is then discretized to obtain the discretized feature data. The discretized feature data is undersampled / oversampled to obtain balanced feature data.

9. The root cause localization system for mobile internet user complaints according to claim 8, characterized in that, The model building and training module and the root cause prediction module are used to complete the feature data after dimensionality reduction by using the Lagrange difference method.

10. The root cause localization system for mobile internet user complaints according to any one of claims 6-9, characterized in that, When training the root cause complaint prediction model based on sample data, the model building and training module performs the following: constructing a random forest model, training and optimizing the training dataset using the random forest model to obtain the model F1 value of the root cause complaint prediction model, and performing cross-validation on the root cause prediction model until the model F1 value of the root cause complaint prediction model is stable, thus obtaining the trained root cause prediction model.

Images