System fault solution generation modeling method, generation method, device and storage medium

Abstract

The present invention discloses a system fault solution generation modeling method, generation method, device and storage medium. The modeling method includes obtaining processed fault information and its solution; performing data cleaning on first error information and constructing a sample data set based on the cleaned first error information; using the sample data set to train, verify and test an initial entity extraction model to obtain a target entity extraction model; using the target entity extraction model to extract entities from the cleaned first error information to obtain a first key entity; constructing a corpus based on the first context information, the first key entity and its solution, and constructing a knowledge graph and a fine-tuning data set based on the corpus; and fine-tuning the pre-trained model using the fine-tuning data set to obtain a solution generation model. The present invention can achieve rapid location, accurate analysis and efficient solution matching for complex fault problems, thereby improving fault handling efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of system failure analysis, and in particular relates to a system failure solution generation modeling method based on RAG architecture, a generation method, a device and a storage medium. Background Art

[0002] In today's digital age, system operation and maintenance face numerous challenges, the most prominent of which is how to accurately locate, effectively analyze, and rationally resolve system failures. Failures manifest in a variety of complex ways, primarily in the following areas:

[0003] (1) Error message descriptions vary, making it difficult to quickly and accurately understand;

[0004] (2) Multiple subsystems are interdependent, and a single failure may trigger a chain reaction;

[0005] (3) Newly emerged errors are uncontrollable and difficult to predict and respond to in advance.

[0006] These factors together make troubleshooting for operations personnel complex, time-consuming, and extremely challenging. Currently, system troubleshooting mostly relies on manual operations, which has many shortcomings:

[0007] First, the expertise required of operations and maintenance personnel is extremely high, making this knowledge-intensive work. Second, problem location and classification are inaccurate, making it difficult to quickly identify the root cause of a problem. Third, the accuracy of matching solutions is low, often requiring multiple attempts. Finally, the low level of automation makes it difficult to improve efficiency. For example, in the case of carrier service ticket activation, companies are eager to improve the efficiency of service ticket activation to increase customer satisfaction, reduce churn, reduce the workload of front-line customer service personnel, and promote cross-selling and upgrades. Therefore, resolving current system failures and eliminating inefficiencies have become urgent needs for business development.

[0008] Retrieve Augmented Generation (RAG) is a hybrid architecture that combines information retrieval (IR) and natural language generation (NLG). It aims to enhance the generative capabilities of large language models (LLMs) by retrieving relevant information from external knowledge bases. The core concept of RAG is to address the limitations of purely generative models, such as hallucinations, timeliness issues, and data security concerns. The RAG workflow consists of three main steps: retrieval, augmentation, and generation. Specifically, the retriever first retrieves relevant documents or information from an external knowledge base based on the user's original query. The retrieved information is then combined with the user's original query to form an augmented input. Finally, the generator uses this augmented input to produce the final output. This architecture enables the RAG model to not only generate fluent text but also ensure that the output is based on real-world, up-to-date data. RAG technology has a wide range of applications, covering multiple natural language processing tasks, including question-answering systems, document generation, intelligent assistants, information retrieval, and knowledge graph population.

[0009] System failure refers to abnormal operation or interruption of a computer or network system due to hardware failure, software defects, configuration errors, human error, or external environmental interference. It may manifest as program crashes, data loss, network connection failures, or slow system responses, seriously impacting the normal operation of the system and user experience.

[0010] Taking the system for facilitating the service work order process as an example, the customer relationship management (CRM) system serves as the user's front-end interface. It generates service orders based on user requirements and transmits order information to the orchestration center. As the central coordination hub, the orchestration center manages the service order flow and automatically generates activation processes for different services, such as mobile phone card activation. It monitors each step of the process and coordinates allocation with the resource capability center and procurement and control center. Network elements execute specific commands, such as port activation and configuration. After completing all service delivery steps, the orchestration center notifies the operations and dispatch center for billing. If an error occurs, the user reports it through the CRM system. Upon receiving the error report, the orchestration center calls the operations and dispatch center to dispatch operations and maintenance personnel for manual troubleshooting. Summary of the Invention

[0011] The purpose of the present invention is to provide a system fault solution generation modeling method, generation method, device and storage medium to solve the problem of low system fault troubleshooting efficiency due to differences in error information descriptions, interdependence between subsystems and uncontrollable new errors.

[0012] The present invention solves the above technical problems through the following technical solutions: a system fault solution generation modeling method, comprising:

[0013] Obtaining processed fault information and its solution; wherein the processed fault information includes first error information and first context information;

[0014] Performing data cleaning on the first error message, and constructing a sample data set based on the cleaned first error message;

[0015] Deploying an initial entity extraction model, and using the sample data set to train, verify, and test the initial entity extraction model to obtain a target entity extraction model;

[0016] Using the target entity extraction model to perform entity extraction on the first error message after data cleaning to obtain a first key entity;

[0017] Constructing a corpus based on the first context information, the first key entity and its solution, and constructing a knowledge graph and a fine-tuning dataset based on the corpus; wherein each sample in the fine-tuning dataset includes the first context information, the first key entity and its solution;

[0018] A pre-trained model is obtained, and the pre-trained model is fine-tuned using the fine-tuning dataset to obtain a solution generation model.

[0019] Furthermore, a sample data set is constructed based on the first error message after data cleaning, specifically including:

[0020] Perform word segmentation on the first error message after data cleaning to obtain potential entities;

[0021] Reviewing and revising the word segmentation boundaries of the potential entities to form standard entities;

[0022] Convert the standard entity into a BIO tag, and perform vector embedding on the characters in the standard entity to obtain a character embedding vector;

[0023] The character embedding vector and the BIO tag constitute a sample in the sample data set.

[0024] Furthermore, the initial entity extraction model adopts a BERT-BiLSTM-CRF model, which includes a BERT layer, a BiLSTM layer and a CRF layer; the BERT layer is used to extract deep contextual semantic features through a bidirectional Transformer structure to generate a word vector representation containing contextual information; the BiLSTM layer is used to perform a bidirectional temporal scanning on the word vector representation containing contextual information, and capture the long-distance dependency of the word vector representation from the forward and backward directions through long short-term memory units; the CRF layer is used to generate a predicted label based on the output of the BiLSTM layer.

[0025] Based on the same concept, the present invention also provides a method for generating a system fault solution, comprising:

[0026] Calling a target entity extraction model, a knowledge graph, and a solution generation model; wherein the target entity extraction model, the knowledge graph, and the solution generation model are constructed using the system fault solution generation modeling method described above;

[0027] Obtaining pending fault information; wherein the pending fault information includes second error information and second context information;

[0028] performing data cleaning on the second error information;

[0029] Using the target entity extraction model to perform entity extraction on the second error message after data cleaning to obtain a second key entity;

[0030] Matching the second key entity and the second context information using the knowledge graph to obtain a corpus list sorted by overall similarity;

[0031] Determine whether the maximum overall similarity is greater than a set threshold. If so, the solution in the corpus corresponding to the maximum overall similarity is the final solution; if not, input the top several corpora in terms of overall similarity into the solution generation model to obtain a predicted solution;

[0032] The prediction solution is evaluated, and the prediction solution that passes the evaluation is used as the final solution, and the prediction solution that passes the evaluation is added to the knowledge graph.

[0033] Furthermore, performing data cleaning on the second error information specifically includes:

[0034] Obtaining a regular expression library; wherein the regular expression library is pre-built according to the first error message;

[0035] Based on the second error message, traverse the regular expression library to obtain a regular expression that matches the second error message;

[0036] The second error message is cleaned using a matching regular expression.

[0037] Furthermore, the specific construction process of the regular expression library includes:

[0038] Performing dynamic parameter identification on the first error message;

[0039] Based on the identified dynamic parameters, extract prefix features and / or suffix features from the first error message;

[0040] A regular expression is generated according to the identified dynamic parameters and the prefix feature and / or the suffix feature, thereby forming a regular expression library.

[0041] Furthermore, matching the second key entity and the second context information using the knowledge graph specifically includes:

[0042] Using the knowledge graph to match each second context information, obtaining a matching degree for each second context information;

[0043] Match the second key entity using the knowledge graph to obtain the number of first key entities that match the second key entity;

[0044] Calculating a ratio of the number of first key entities that match the second key entity to the total number of second key entities;

[0045] A weighted sum is performed on the matching degree of each second context information and the ratio to obtain the overall similarity and its corpus; wherein the corpus includes the second context information, the second key entity and its solution.

[0046] Furthermore, the prediction solution is evaluated, specifically including:

[0047] Performing machine evaluation on the prediction solution using evaluation metrics, wherein the evaluation metrics include Jaccard coefficient, BLEU, EM, and ROUGE-L;

[0048] If the machine evaluation is passed, the predicted solution that has passed the machine evaluation is used as the final solution, and the predicted solution that has passed the machine evaluation is added to the knowledge graph;

[0049] If the machine evaluation fails, the prediction solution is evaluated and adjusted by experts to obtain a final solution, and the final solution is added to the knowledge graph.

[0050] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the system fault solution generation modeling method or the system fault solution generation method as described above.

[0051] Based on the same concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the system fault solution generation modeling method or system fault solution generation method as described above.

[0052] Compared with the prior art, the advantages of the present invention are:

[0053] This invention performs preliminary processing on fault information through data cleaning and entity extraction, effectively removing redundant information and accurately extracting the core fault information, laying a solid foundation for subsequent analysis. It cleverly integrates the advantages of knowledge graphs and pre-trained models to design a retrieval generation enhancement system framework based on the RAG architecture. This not only fully utilizes the structured knowledge resources in the knowledge graph, but also leverages the powerful generation capabilities of pre-trained models to achieve rapid location, accurate analysis, and efficient solution matching for complex fault problems, thereby improving fault handling efficiency and reducing the burden on operation and maintenance personnel.

[0054] Through data cleaning and entity extraction, the present invention solves the problem that anomalies in error information descriptions make it difficult to quickly and accurately understand them, laying the foundation for generating system fault solutions; the present invention can automatically and quickly generate solutions based on new fault information, improving efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 This is a flow chart of a method for generating a system fault solution model in an embodiment of the present invention;

[0057] Figure 2 This is a flow chart of a method for generating a system fault solution in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following is a clear and complete description of the technical solutions of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0059] The following specific embodiments are used to describe the technical solution of the present application in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0060] Example 1

[0061] Figure 1 The flowchart of the system fault solution generation modeling method provided by the present invention is shown. Figure 1As shown, the system fault solution generation modeling method includes the following steps:

[0062] Step A1: Obtain information about the resolved fault and its solution.

[0063] Collect processed fault information and solutions to the processed fault information from technical literature, production data, and user feedback data, wherein the solutions include specific solutions and root cause analysis, and the processed fault information includes first error information and context information.

[0064] Taking the activation of an operator work order as an example, the context information includes product type, service type, and error reporting link. The first error reporting information is the interface call log, card order exception description, etc.

[0065] Step A2: performing data cleaning on the first error message, and constructing a sample data set based on the cleaned first error message.

[0066] Error messages often contain a large amount of sensitive private content and irrelevant information, such as the user's phone number, home address, order number, etc. The presence of this information can interfere with subsequent processing, so removing this content is crucial. To remove sensitive private content and irrelevant information from error messages, regular expressions are used to clean the first error message. Before data cleaning, a regular expression library is first constructed. The specific construction process includes:

[0067] Step A2.11: Identify dynamic parameters of the first error message; dynamic parameters refer to parameters that are easily changed;

[0068] Step A2.12: Extract prefix features and / or suffix features from the first error message based on the identified dynamic parameters;

[0069] Step A2.13: Generate a regular expression based on the identified dynamic parameters and prefix features and / or suffix features, thereby forming a regular expression library.

[0070] The regular expression template is: Rule ID: ("prefix feature + dynamic parameter regular expression", retaining the number of characters). Taking an operation fault ticket as an example, dynamic parameters include IP address, timestamp, home address, etc. Prefix features include "IP:", "IPWG:", etc., and suffix features include "existed" and "failed". The example regular expression is as follows:

[0071] 18: ("IPWG:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}", 5);

[0072] Among them, 18 represents the number of the regular expression in the regular expression library; "IPWG:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" is the regular expression string, and "IPWG:" is a fixed prefix, indicating that the matching string must start with "IPWG: " begins; [0-9]{1,3} is a character set matching pattern, which means matching any number (0 to 9), and the character set (number) here can appear 1 to 3 times, so [0-9]{1,3} matches a value in the range of 0 to 999; "\." is an escaped dot (.). In regular expressions, the dot (.) is a special character that matches any single character; by adding a backslash (\) in front of it, the dot is escaped to an ordinary character, indicating a literal dot. In short, the regular expression string is a repeated pattern, representing four numeric parts separated by dots, and the numeric range of each part is 0 to 999, which conforms to the format of IPv4 addresses; 5 means that the first 5 characters are retained in the data cleaning result.

[0073] During the data cleaning process, based on the first error message, a regular expression library is traversed to obtain a regular expression that matches the first error message; and then the first error message is cleaned using the matched regular expression.

[0074] Taking the first error message "(Please reallocate an idle IP) IPWG:220.170.78.9mse pre-check failed" as an example, the execution process of rule #18 is as follows:

[0075] Identify the "IPWG:" prefix + IP address pattern; retain the first five characters of "IPWG:" ("IPWG"); and delete the dynamic IP address "220.170.78.9".

[0076] The result after data cleaning is: (Please reallocate an idle IP address) IPWG: mse pre-check failed.

[0077] To verify the effectiveness of regular expressions in removing interference from dynamic parameters, we compared the initial error messages before and after data cleaning, established a backtracking mechanism for incorrect cleaning, and continuously optimized the regular expression library. Based on the specific error messages, we can flexibly add, delete, modify, and query regular expressions, thereby removing as much redundant information as possible and ensuring data accuracy and security.

[0078] In order to realize entity extraction of error information after data cleaning, it is necessary to build an entity extraction model. Before building the entity extraction model, it is necessary to build a corresponding sample data set. In a specific embodiment of the present invention, the sample data set is built according to the first error information after data cleaning, specifically including:

[0079] Step A2.21: Use the Jieba word segmentation algorithm to segment the first error message after data cleaning to obtain potential entities.

[0080] For example, "idle IP" is segmented into "idle" and "IP".

[0081] Step A2.22: Experts review and correct the word segmentation boundaries of potential entities to form standard entities.

[0082] This approach merges fragmented descriptions referring to the same physical entity. For example, "Mse equipment" is treated as a complete entity rather than a single segmented word. Through segmentation processing, review, and correction, a high-confidence annotated corpus is constructed, reducing noise interference in subsequent models.

[0083] Step A2.23: Use the BIO (B-begin, I-inside, O-outside) annotation method to convert the standard entity into BIO tags, perform vector embedding on the characters in the standard entity, and obtain the character embedding vector.

[0084] The mark "O" indicates a non-entity, and the marks "B-INFO" and "I-INFO" both indicate entities. For example, the error message after data cleaning is "(Please reallocate an idle IP) IPWG: mse pre-check failed", and the standard entity is "[reallocate, idle IP, IPWG, mse pre-check failed]", and the BIO mark of each part is: "(" BIO mark is O, "please" BIO mark is O, "re-" BIO mark is B-INFO, "new" BIO mark is I-INFO, "divide" BIO mark is I-INFO, "allocate" BIO mark is I-INFO, "empty" BIO mark is B-INFO, "idle" BIO mark is I-INFO, "I" BIO mark is I-INFO, and "P" BIO mark is I-INFO. The BIO mark of “)” is O, the BIO mark of “I” is B-INFO, the BIO mark of “P” is I-INFO, the BIO mark of “W” is I-INFO, the BIO mark of “G” is I-INFO, the BIO mark of “:” is O, the BIO mark of “m” is B-INFO, the BIO mark of “s” is I-INFO, the BIO mark of “e” is I-INFO, the BIO mark of “pre” is I-INFO, the BIO mark of “check” is I-INFO, the BIO mark of “no” is B-INFO, the BIO mark of “pass” is I-INFO, and the BIO mark of “pass” is I-INFO.

[0085] The Word2Vec model pre-trained on the Chinese Wiki corpus is used to embed the characters in the standard entities into vectors to obtain character embedding vectors.

[0086] Step A2.24: The character embedding vector and the BIO tag constitute a sample in the sample dataset.

[0087] According to the chronological order of the fault occurrence, the sample data set is divided into training set, validation set and test set with a division ratio of 6:2:2, so as to facilitate the training, verification and testing of subsequent models.

[0088] Step A3: Deploy the initial entity extraction model, use the sample data set to train, verify and test the initial entity extraction model, and obtain the target entity extraction model.

[0089] Taking the character embedding vector as input and the corresponding BIO tag as the expected output, the initial entity extraction model is trained, verified and tested to obtain the target entity extraction model.

[0090] In a specific embodiment of the present invention, the initial entity extraction model adopts a BERT-BiLSTM-CRF model, which includes a BERT (Bidirectional Encoder Representations from Transformers) layer, a BiLSTM layer (bidirectional long short-term memory network), and a CRF layer (conditional random field). The BERT layer acts as a semantic encoder, deeply encoding character embedding vectors through a pre-training mechanism and dynamically generating word vector representations that incorporate global context information using the bidirectional self-attention mechanism of the Transformer architecture. The BiLSTM layer acts as a sequence modeler, performing temporal modeling on the word vector representations output by the BERT layer. The forward LSTM captures historical information (such as the context before the current word) and the backward LSTM captures future information (such as the context after the current word), concatenating the hidden states of the two to form a comprehensive temporal feature. The CRF layer acts as a label decoder. Based on the state probability matrix output by the BiLSTM layer, it calculates the global optimal label path using the Viterbi algorithm. It then uses a preset label transfer matrix to constrain the logical relationships between entity labels, thereby eliminating unreasonable entity segment combinations. Through the collaboration of the BERT layer, BiLSTM layer, and CRF layer, the accuracy and robustness of entity extraction are improved.

[0091] The training parameters are set as: batch size 20, learning rate 1e -3 The number of iterations was 100, and an early stopping strategy on the validation set was used to prevent overfitting. Finally, on the test set, the F1 value reached 98.3%, meeting the accuracy requirement.

[0092] Step A4: Use the target entity extraction model to perform entity extraction on the first error message after data cleaning to obtain the first key entity.

[0093] For example, the input is: "(Please reallocate an idle IP) IPWG: MSE pre-check failed: 1. A static VPN user already exists on the MSE device (IP:)!" The entity extraction result is: [reallocate, idle IP, IPWG, MSE pre-check, failed, MSE device, IP, static VPN user, already exists].

[0094] Step A5: Build a corpus based on the first context information, the first key entity and its solution, and build a knowledge graph and a fine-tuning dataset based on the corpus.

[0095] To address the complex challenge of different solutions required for different types of errors in different contexts, a system fault knowledge graph is constructed. This knowledge graph explicitly represents the relationships between contextual information, the first key entity, and the solution. In this embodiment, the knowledge graph is a directed acyclic graph G = (V, E), where each node V represents an entity related to system fault operation and maintenance. Specifically, it is divided into three types: error nodes (indicating error type or error information), context nodes (covering contextual information such as product type, service type, and error reporting process), and solution nodes (corresponding to solutions and root cause analysis for specific errors). Directed edges E represent the relationships between nodes. Key relationships include error information-context information (reflecting the dependency of error information on context), error information-solution relationships (linking error information to its solutions and root cause analysis), and hierarchical relationships (reflecting the hierarchy or correlation between context information and solution nodes). In the knowledge graph matching or retrieval stage, matching or retrieval is performed based on the node type. The user's query Q is input, and the top three corpora ranked by overall similarity are output, including Rank (ranking sorted by overall similarity), CorpusID (the number of the retrieved information in the knowledge graph), context information, error information and solution. The corpus is passed to the large language model (LLM) to enhance the reasoning ability of the large language model.

[0096] Before building the knowledge graph, we first deduplicate the first key entity obtained in step A4. Then, we build a corpus based on the first context information, the first key entity, and its solution. We then create a new graph database using Neo4j, traverse the corpus, and sequentially connect each node according to the knowledge graph ontology to create the knowledge graph.

[0097] When knowledge graph matching or retrieval fails, a large language model is used for reasoning to generate a solution. The large language model is a pre-trained model. To apply the pre-trained model to solution reasoning, it is necessary to fine-tune the pre-trained model using a fine-tuning dataset. The fine-tuning dataset of this embodiment is constructed based on a corpus. Each sample in the fine-tuning dataset includes first context information, a first key entity, and its solution.

[0098] Step A6: Obtain a pre-trained model, and fine-tune the pre-trained model using the fine-tuning dataset to obtain a solution generation model.

[0099] In a specific embodiment of the present invention, a LoRA (Low-Rank Adaptation) fine-tuning method is adopted. By adding LoRA modules to the key layers of the pre-trained model (including the self-attention projection layer (query / key / value matrix) in the Transformer module and the fully connected layer in the feedforward network), the low-rank matrix parameters are adjusted, and the basic layers related to general semantic understanding in the pre-trained model (such as the underlying Transformer module) are frozen. While retaining the basic language capabilities of the model, the video memory consumption and computing resources are reduced.

[0100] In this embodiment, the pre-training model uses the Qwen-7B-Chat model. The fine-tuning parameter configuration includes LoRA configuration and training parameter configuration. LoRA configuration: the rank r of the low-rank matrix is ​​set to 8, the LoRA scaling factor α is set to 32, and the Dropout rate is set to 0.1; training parameter configuration: the learning rate is set to 10 -4 ,The batch size is adjusted according to the hardware resources, and the number of training rounds is set to 2-3 rounds.

[0101] Example 2

[0102] Figure 2 The method for generating a system fault solution provided by the present invention is shown. Figure 2 As shown, the method for generating a system fault solution includes the following steps:

[0103] Step B1: Call the target entity extraction model, knowledge graph, and solution generation model.

[0104] The target entity extraction model, knowledge graph and solution generation model are constructed using the system fault solution generation modeling method in Example 1 of the present application.

[0105] Step B2: Obtain information about the fault to be processed.

[0106] The fault information to be processed includes the second error information and the second context information.

[0107] Step B3: Perform data cleaning on the second error message.

[0108] In this embodiment of the present invention, the regular expression library constructed in the first embodiment is used to clean the second error message to remove sensitive privacy content and irrelevant information. The specific data cleaning process includes:

[0109] Based on the second error message, traverse the regular expression library to obtain a regular expression that matches the second error message;

[0110] The second error message is cleaned using a matching regular expression.

[0111] Step B4: Use the target entity extraction model to perform entity extraction on the second error message after data cleaning to obtain the second key entity.

[0112] The second error information after data cleaning is input into the target entity extraction model, and multiple second key entities of the second error information are output.

[0113] Step B5: Use the knowledge graph to match the second key entity and the second context information to obtain a corpus list sorted by overall similarity.

[0114] In a specific embodiment of the present invention, matching the second key entity and the second context information using the knowledge graph specifically includes:

[0115] Step B5.1: Use the knowledge graph to match each second context information to obtain the matching degree of each second context information;

[0116] Step B5.2: Use the knowledge graph to match the second key entity and obtain the number of first key entities that match the second key entity;

[0117] Step B5.3: Calculate the ratio of the number of first key entities matching the second key entity to the total number of second key entities;

[0118] Step B5.4: Perform weighted summation on the matching degree and ratio of each second context information to obtain the overall similarity and its corpus; wherein the corpus includes the second context information, the second key entity and its solution.

[0119] Taking the stuck order phenomenon during the operator work order activation process as an example, inputting <node type, node specific information> into the knowledge graph returns CorpusIDlist (a numbered list of the retrieved information in the knowledge graph); for example, inputting <product type, broadband> returns CorpusIDlist [2,101,890], which means that the product type of 2,101,890 is also exactly the same broadband.

[0120] According to the user query Q (including the second key entity and the second context information, the second context information includes product type, service type and error reporting link), the relevant information is retrieved layer by layer in the knowledge graph, and the matching degree is calculated layer by layer:

[0121] First, search the product type layer to obtain Q's product type; call the node type-based search algorithm, input <product type, Q's product type>, and obtain the CorpusIDlist of the product type layer hits;

[0122] Search the service type layer to obtain the service type of Q; call the node type-based search algorithm, input <service type, Q's service type>, and obtain the CorpusIDlist of the service type layer hits;

[0123] Search the error reporting link layer to obtain Q's error reporting link; call the node type-based search algorithm, input <error reporting link, Q's error reporting link>, and obtain the CorpusIDlist of the error reporting link layer hits;

[0124] Search the error information entity layer to obtain Q's error information entity; call the node type-based search algorithm, input <error information, Q's error information entity>, and obtain the CorpusIDlist of the error information entity layer hits;

[0125] Calculate the overall similarity based on the CorpusIDlist hits at each layer:

[0126] (1)

[0127] (2)

[0128] in, Indicates the overall similarity of the fault information to be processed, Indicates the retrieved or matched information. The product type of Q is The matching or similarity of product types, Indicates the service type of Q The matching or similarity of the service types, Indicates the error reporting link of Q and The matching degree or similarity of the error reporting link, express The number of key entities that match Q in represents the total number of key entities in Q, 、 、 、 They represent the weight coefficients of the product type layer, service type layer, error reporting link layer, and error reporting information entity layer respectively.

[0129] The overall similarity of the fault information to be processed is equal to the weighted sum of the matching degree of each context information and the proportion of the number of matched key entities in the total number of key entities, as shown in formula (1) and formula (2).

[0130] Step B6: Determine whether the maximum overall similarity is greater than a set threshold. If so, the solution in the corpus corresponding to the maximum overall similarity is the final solution; if not, input the first few corpora with the highest overall similarity into the solution generation model to obtain a predicted solution.

[0131] All the results retrieved for query Q Sort by overall similarity. If the maximum overall similarity is greater than the set threshold, the corresponding The solution in the list is the final solution and is fed back to the user; if the maximum overall similarity is less than or equal to the set threshold, the top 3 solutions in terms of overall similarity are selected. The context information, key entities and solutions in the solution generation model are input into the solution generation model, which performs reasoning and generates a prediction solution.

[0132] Step B7: Evaluate the prediction solution, take the prediction solution that passes the evaluation as the final solution, and add the prediction solution that passes the evaluation to the knowledge graph.

[0133] In a specific embodiment of the present invention, the prediction solution is evaluated, specifically including:

[0134] Machine evaluation of prediction solutions using evaluation metrics such as Jaccard coefficient, Bilingual Evaluation Understudy (BLEU), Exact Match (EM), and ROUGE-L (where L is LCS (Longest Common Subsequence)).

[0135] If the machine evaluation is passed, the predicted solution that passes the machine evaluation is fed back to the user as the final solution, and the predicted solution that passes the machine evaluation is added to the knowledge graph;

[0136] If the machine evaluation fails, the prediction solution is evaluated and adjusted by experts to obtain the final solution, which is then added to the knowledge graph.

[0137] Example 3

[0138] An embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored on the memory, and the processor executes the computer program / instructions to implement the system fault solution generation modeling method or the system fault solution generation method in the embodiment of the present application.

[0139] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes based on programs and / or data stored in a read-only memory (ROM) or programs and / or data loaded from a storage portion into a random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. Various programs and data required for device operation are also stored in the RAM. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0140] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0141] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the system fault solution generation modeling method or the system fault solution generation method in the embodiment of the present application.

[0142] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0143] The above disclosure is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, and they should all be covered by the scope of protection of the present invention.

Claims

1. A method for generating a system fault solution, characterized in that: The generation method comprises: Call the target entity extraction model, knowledge graph, and solution generation model; Obtaining pending fault information; wherein the pending fault information includes second error information and second context information; performing data cleaning on the second error information; Using the target entity extraction model to perform entity extraction on the second error message after data cleaning to obtain a second key entity; Matching the second key entity and the second context information using the knowledge graph to obtain a corpus list sorted by overall similarity; Determine whether the maximum overall similarity is greater than a set threshold. If so, the solution in the corpus corresponding to the maximum overall similarity is the final solution; if not, input the top several corpora in terms of overall similarity into the solution generation model to obtain a predicted solution; Evaluate the prediction solutions, and add prediction solutions that pass the evaluation or prediction solutions that fail the evaluation but have been evaluated and adjusted by experts to the knowledge graph; The specific modeling process of the target entity extraction model, knowledge graph and solution generation model includes: Obtaining processed fault information and its solution; wherein the processed fault information includes first error information and first context information; Performing data cleaning on the first error message, and constructing a sample data set based on the cleaned first error message; Deploying an initial entity extraction model, and using the sample data set to train, verify, and test the initial entity extraction model to obtain a target entity extraction model; Using the target entity extraction model to perform entity extraction on the first error message after data cleaning to obtain a first key entity; Constructing a corpus based on the first context information, the first key entity and its solution, and constructing a knowledge graph and a fine-tuning dataset based on the corpus; wherein each sample in the fine-tuning dataset includes the first context information, the first key entity and its solution; Obtaining a pre-trained model, and fine-tuning the pre-trained model using the fine-tuning dataset to obtain a solution generation model; The method of matching the second key entity and the second context information using the knowledge graph specifically includes: Using the knowledge graph to match each second context information, obtaining a matching degree for each second context information; Match the second key entity using the knowledge graph to obtain the number of first key entities that match the second key entity; Calculating a ratio of the number of first key entities that match the second key entity to the total number of second key entities; A weighted sum is performed on the matching degree of each second context information and the ratio to obtain the overall similarity and its corpus; wherein the corpus includes the second context information, the second key entity and its solution.

2. The method for generating a system fault solution according to claim 1, wherein: Construct a sample data set based on the first error message after data cleaning, specifically including: Perform word segmentation on the first error message after data cleaning to obtain potential entities; Reviewing and revising the word segmentation boundaries of the potential entities to form standard entities; Convert the standard entity into a BIO tag, and perform vector embedding on the characters in the standard entity to obtain a character embedding vector; The character embedding vector and the BIO tag constitute a sample in the sample data set.

3. The method for generating a system fault solution according to claim 1, wherein: The initial entity extraction model adopts a BERT-BiLSTM-CRF model, which includes a BERT layer, a BiLSTM layer, and a CRF layer. The BERT layer is used to extract deep contextual semantic features through a bidirectional Transformer structure to generate word vector representations containing contextual information. The BiLSTM layer is used to perform bidirectional temporal scanning on the word vector representations containing contextual information, and capture the long-distance dependencies of the word vector representations from both the forward and backward directions through long short-term memory units. The CRF layer is used to generate predicted labels based on the output of the BiLSTM layer.

4. The method for generating a system fault solution according to claim 1, wherein: The data cleaning of the second error message specifically includes: Obtaining a regular expression library; wherein the regular expression library is pre-built according to the first error message; Based on the second error message, traverse the regular expression library to obtain a regular expression that matches the second error message; The second error message is cleaned using a matching regular expression.

5. The method for generating a system fault solution according to claim 4, wherein: The specific construction process of the regular expression library includes: Performing dynamic parameter identification on the first error message; Based on the identified dynamic parameters, extract prefix features and / or suffix features from the first error message; A regular expression is generated according to the identified dynamic parameters and the prefix feature and / or the suffix feature, thereby forming a regular expression library.

6. The method for generating a system failure solution according to any one of claims 1 to 5, characterized in that: Evaluate the forecasting solution, including: The prediction solution is machine evaluated using evaluation metrics, wherein the evaluation metrics include Jaccard coefficient, BLEU, EM, and ROUGE-L.

7. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the method for generating a system fault solution according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the method for generating a system fault solution according to any one of claims 1 to 6 is implemented.

Images