Root cause analysis method and device based on large language model and storage medium

Abstract

The invention provides a root cause analysis method and device based on a large language model and a storage medium, and belongs to the field of industrial fault diagnosis and artificial intelligence. The root cause analysis method comprises the following steps: obtaining a user question; based on the user question, performing retrieval from a knowledge base constructed through a large language model in advance to obtain a first retrieval result; obtaining a corresponding expected question based on the user question through a pre-constructed query agent; retrieving from a knowledge base based on the expected problem to obtain a second retrieval result; and optimizing based on the first retrieval result and the second retrieval result to obtain a root cause analysis result. According to the method, the retrieval range is expanded from the aspects of problems and contents through a double-path retrieval mechanism, and retrieval omission caused by user expression deviation is avoided; the optimization processing process ensures that the analysis result not only retains core information, but also supplements associated content, and the credibility of the result is improved while the universality of the method is maintained; the finally obtained model is low in cost and higher in generalization ability.

Description

Technical Field

[0001] This invention relates to the fields of industrial fault diagnosis and artificial intelligence, specifically to a root cause analysis method, device, and storage medium based on a large language model. Background Technology

[0002] In industrial production, unpredictable failures impact production efficiency and reduce product quality. If these failures are not corrected promptly, they can lead to significant economic losses or equipment damage. As modern industrial technology evolves towards larger scales and greater complexity, the number of devices involved in product manufacturing increases, making the interconnection and interoperability between these devices more complex. While data-driven and artificial intelligence technologies have been applied to various aspects of automated industrial production, such as fault detection and classification, the complex coupling relationships between event factors still require maintenance engineers to expend considerable time and effort, engaging in multiple rounds of communication to determine the root cause of failures and mitigation measures. To maintain the efficiency and stability of industrial processes, a reliable Root Cause Analysis (RCA) method is needed to identify the root causes of failures and ensure system reliability.

[0003] Traditional RCA methods primarily rely on statistical correlation analysis techniques, such as root cause localization strategies based on causal discovery. Other approaches include using metric data to construct graph neural networks to capture fault propagation paths, or leveraging causal Bayesian networks and tracking data for root cause localization. However, while these methods perform well in static environments, they struggle to adapt to dynamic dependencies.

[0004] Large Language Models (LLMs), capable of understanding and generating human-like responses to various types of questions, have been applied to RCA tasks. Microsoft conducted the first large-scale study in 2023 to evaluate the effectiveness of LLMs in helping engineers identify and mitigate production incidents. Other researchers have also proposed a tool-enhanced LLM autonomous agent framework, RCAgent, for practical and privacy-conscious RCA tasks in industrial applications.

[0005] Existing RCA methods based on large language models rely heavily on computationally expensive LLM fine-tuning schemes to generate texts of event summaries, root causes, and mitigation measures. Therefore, these LLM-based RCA methods suffer from high fine-tuning costs and insufficient generalization ability in root cause analysis. Summary of the Invention

[0006] The purpose of this invention is to solve the problems existing in the prior art and provide a root cause analysis method, device and storage medium based on a large language model, which effectively solves the problems of high fine-tuning cost and insufficient generalization ability when the existing large language model is applied to root cause analysis.

[0007] This invention is achieved through the following technical solution:

[0008] The first aspect of the invention provides a root cause analysis method based on a large language model, the root cause analysis method comprising:

[0009] Obtaining user questions;

[0010] Based on the user's question, the first search result is obtained from the knowledge base pre-constructed using a large language model;

[0011] Based on the user's question, the corresponding expected question is obtained through a pre-built query agent;

[0012] Based on the desired question, a second search result is obtained from the knowledge base;

[0013] Based on the first search result and the second search result, the root cause analysis result is obtained through optimization.

[0014] Preferably, before obtaining user issues, the root cause analysis method further includes:

[0015] Obtain relevant documents for the target industrial sector;

[0016] Data preprocessing is performed on relevant documents in the target industrial field, and semantic blocks in a unified format are generated through a large language model. Each semantic block includes the query question and context-related information.

[0017] The semantic blocks are segmented, and a vector database of the knowledge base is constructed using text vectorization operations.

[0018] Based on the semantic block, a corresponding preset question is generated, and a query agent is constructed.

[0019] Preferably, the relevant documents include academic articles, books, and reports, record forms, and QA documents from industrial production. The format of the semantic block includes: the number of words in the question is less than 200, and the number of words in the context-related information is less than 2000.

[0020] Preferably, the method for segmenting the semantic block includes:

[0021] Set the proportion of the actual length-based segmentation method;

[0022] Calculate the segmentation length when the semantic block is segmented into a fixed size;

[0023] If the actual length of the semantic block is greater than the segmentation length, the semantic block is segmented into multiple fixed-size text blocks according to the segmentation length, wherein there is a preset proportion of repeated data between the text blocks;

[0024] If the actual length of the semantic block is less than or equal to the segmentation length, the entire semantic block is treated as a single text block.

[0025] Preferably, the method for retrieving a first search result from the knowledge base based on the user question includes:

[0026] Based on the user question, a set of text blocks consisting of one or more related text blocks is obtained by matching from the knowledge base;

[0027] Obtain the semantic block corresponding to the set of text blocks;

[0028] The first context-related information related to the user's question is obtained by matching from the corresponding semantic block and used as the first retrieval result.

[0029] Preferably, the second search result includes second context-related information. Based on the first search result and the second search result, the root cause analysis result is optimized and obtained, including:

[0030] The first and second context-related information are deduplicated and merged to obtain semantically complete context information;

[0031] Assess the relevance of the contextual information to the user's question and assign relevant scores;

[0032] The context information is reordered based on the relevant scores, and the top K results with the highest relevance to the user's question are retained first, where K is a positive integer and K≥1.

[0033] Preferably, the method for generating corresponding preset questions based on the semantic blocks and constructing a query proxy includes:

[0034] For each semantic block, a large language model is used to generate multiple pre-defined questions corresponding to the context-related information within it.

[0035] Text embedding is performed on all generated preset questions to obtain preset question embedding vectors;

[0036] The preset question embedding vector and the index relationship between the preset question and the corresponding semantic block are saved to the query proxy database to form a query proxy.

[0037] Preferably, the method for obtaining the corresponding expected question through a pre-built query agent based on the user question includes:

[0038] The user question is embedded into text to obtain a user question embedding vector;

[0039] By comparing the cosine similarity between the user question embedding vector and the preset question embedding vector stored in the query agent, one or more preset questions whose similarity satisfies a preset similarity threshold are obtained.

[0040] One or more preset questions are obtained as expected questions corresponding to the user questions.

[0041] A second aspect of the present invention provides an electronic device comprising a memory and a processor, the memory storing a computer program executed by the processor, the computer program, when executed by the processor, causing a device equipped with the processor to perform the root cause analysis method based on a large language model as described in any of the preceding claims.

[0042] A third aspect of the invention provides a storage medium storing a computer program that runs on a computer and, when running, causes the computer to perform the root cause analysis method based on a large language model as described in any of the preceding claims.

[0043] Compared with the prior art, the beneficial effects of the present invention are:

[0044] 1. No fine-tuning of LLM is required. Using LLM directly will reconstruct the original document according to the predefined pattern, thereby reorganizing the document into semantic blocks related to the question, reducing possible invalid or redundant information to improve the signal-to-noise ratio.

[0045] 2. Generate preset questions for semantic block content, and enhance the model's ability to understand user queries by matching preset questions, thereby improving the success rate of contextual retrieval;

[0046] 3. A retrieval agent based on preset questions was established, which solved the problem of insufficient relevance between the query question and the correct context content;

[0047] 4. A two-stage retrieval optimization approach is adopted to expand the retrieval scope from both the question and content perspectives, and the retrieval results are merged for precise ranking, which can obtain closely matching results; the final model has lower cost and stronger generalization ability. Attached Figure Description

[0048] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0049] Figure 1 This is a flowchart illustrating the root cause analysis method based on a large language model according to some embodiments of the present invention;

[0050] Figure 2 This is a flowchart illustrating a method for constructing a knowledge base and query agent according to an embodiment of the present invention.

[0051] Figure 3 This is a flowchart illustrating another embodiment of the root cause analysis method based on a large language model according to the present invention.

[0052] Figure 4 This is an example diagram of user questions, document content, and preset questions according to another embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram illustrating the principle of a root cause analysis method based on a large language model, according to another embodiment of the present invention.

[0054] Figure 6 This is an overall framework diagram of the root cause analysis model of another embodiment of the present invention;

[0055] Figure 7a This is a reference document content and a sample diagram of the first report generated in another embodiment of the present invention;

[0056] Figure 7b This is a reference document content and a sample diagram of the generated second report in yet another embodiment of the present invention;

[0057] Figure 8 This is a schematic diagram of the structure of an electronic device according to some embodiments of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0059] Retrieval Augmentation (RAG) is a method for answering user questions on a dataset, aiming to enhance the performance of large language models on specific tasks, such as question answering and summarizing, by retrieving documents. However, when applied to RCA tasks, the process involves a large amount of specialized knowledge and expression, making it difficult for general reasoning models to generate convincing answers without sufficient information. In this case, the ability of the RAG model to retrieve the correct external data becomes crucial. The main limitations encountered are: First, the external formats available in real-world production scenarios are diverse, including equipment manuals, engineers' maintenance records, and opinions from relevant domain experts. These documents are often recorded in their own convenient formats, resulting in low density of effective information, which greatly complicates the extraction of effective knowledge and the construction of a knowledge base. Second, most of this external data consists of results of questions without corresponding questions. Furthermore, non-experts cannot accurately describe the fault itself or provide accurate query questions; the questions they raise lack sufficient relevance to the context required for answering, leading to the inability to retrieve correct content. This further increases the difficulty for the model to provide accurate fault causes and mitigation measures information. Existing large language models, when applied to root cause analysis, suffer from high fine-tuning costs and insufficient generalization ability. Therefore, this invention proposes a root cause analysis method based on large language models, which can quickly determine the causes of failures in industrial production processes. The invention will now be described in further detail with reference to the accompanying drawings.

[0060] To facilitate understanding of this invention, a detailed description of a root cause analysis method based on a large language model, as disclosed in the embodiments of this invention, is provided first. The execution entity of this root cause analysis method based on a large language model is generally a computer device with a certain computing power. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, or a terminal. In some possible implementations, this root cause analysis method based on a large language model can be implemented by a processor calling computer-readable instructions stored in memory. For example, the execution entity of the root cause analysis method based on a large language model may be an industrial production event management system installed on a product production line.

[0061] The following is for reference Figure 1 This describes a root cause analysis method based on a large language model, which is used in some embodiments of the present invention.

[0062] Figure 1 This is a flowchart illustrating the root cause analysis method based on a large language model according to some embodiments of the present invention, such as... Figure 1As shown, the root cause analysis method based on a large language model in this invention includes at least the following steps S100 to S500.

[0063] Step S100: Obtain user questions.

[0064] User questions refer to user-input descriptions of fault phenomena or analysis requests, which can be collected in the form of natural language text to trigger the initial retrieval process.

[0065] Step S200: Based on the user's question, retrieve the first search result from the knowledge base pre-built using a large language model.

[0066] The first search result refers to the context information that directly matches the user's original query. Specifically, it can be implemented using a vector similarity retrieval algorithm, preserving the core content of the user's explicit needs.

[0067] A knowledge base can refer to a structured database containing failure cases in the industrial field. Specifically, it can use vectorization techniques to build semantic indexes, supporting rapid matching of relevant contextual information.

[0068] By using large language models to simplify and extract this external data, structured information highly relevant to the causes of failures and mitigation measures can be generated, thereby building a knowledge base for root cause analysis.

[0069] Step S300: Based on the user's question, obtain the corresponding expected question through a pre-built query agent.

[0070] Here, the expectation question refers to an extended question that is potentially related to the user's query. Specifically, results can be filtered and generated through semantic similarity calculations to ensure the effectiveness of secondary retrieval. The query proxy refers to a generative model that can expand the user's query intent. Specifically, it can use a pre-trained language model to automatically generate potential related questions to supplement the retrieval dimensions that the user has not explicitly expressed.

[0071] Step S400: Based on the expected question, retrieve the second search result from the knowledge base.

[0072] The second search result refers to supplementary information obtained based on the extended question, which can be obtained through a multi-round search strategy to cover implicit association scenarios.

[0073] Step S500: Based on the first search result and the second search result, optimize to obtain the root cause analysis result.

[0074] Optimization processing refers to the integration and analysis of the results of two searches. Specifically, deduplication and sorting algorithms can be used to fuse information and form a multi-dimensional analytical basis.

[0075] Specifically, when a user submits a description of an abnormal device shutdown, the system first matches directly related fault cases from the knowledge base as the initial search results. Simultaneously, the query agent generates multiple potential fault-causing questions based on the user's question, such as power fluctuations or program logic errors. These extended questions are then used to search the knowledge base again, yielding a second set of search results related to the status of the associated device. Finally, the fault cases obtained from both searches are deduplicated and sorted according to their relevance to the user's question, prioritizing cases involving power supply module malfunctions as the root cause analysis results.

[0076] It should be noted that the method of the present invention can be applied not only to the root cause analysis of industrial production events, but also to other fields, such as biomedical fields. The application scenarios of the present invention are not specifically limited.

[0077] This invention provides a root cause analysis method based on a large language model. It expands the search scope from both the problem and content perspectives through a dual-path retrieval mechanism, avoiding omissions due to user-defined biases. The extended questions generated by the query agent can uncover potential related factors, solving the problem of insufficient relevance between the query question and the correct context, thus improving the accuracy of root cause localization in complex fault scenarios. The optimized processing ensures that the analysis results retain core information while supplementing related content, improving the reliability of the results while maintaining the method's universality. The final model has lower costs and stronger generalization capabilities.

[0078] The following uses failure cause analysis in industrial production as an example to illustrate the root cause analysis method of the present invention through different embodiments.

[0079] First refer to Figure 2 This invention describes a root cause analysis method based on a large language model, according to one embodiment of the present invention.

[0080] like Figure 2 As shown, in one embodiment of the present invention, before obtaining user problems, the root cause analysis method further includes:

[0081] Step S001: Obtain relevant documents for the target industrial sector.

[0082] Specifically, the relevant documents include documents from various sources in industrial production, such as academic articles, books, as well as reports, record forms, and question-and-answer (QA) documents in industrial production.

[0083] The collected document set D can include, but is not limited to, various file formats, such as: html, pdf, xml, docx, xlsx, etc. This embodiment does not specifically limit the document format.

[0084] Step S002: Perform data preprocessing on relevant documents in the target industrial field, and generate semantic blocks in a unified format through a large language model. Each semantic block includes the query question and context-related information.

[0085] Data preprocessing refers to format conversion and noise removal of the original document. This can be achieved through text cleaning, paragraph segmentation, and keyword extraction to ensure the standardization and usability of the input data. Data preprocessing can be performed using various existing open-source tools for format conversion, which will not be elaborated upon here.

[0086] After preprocessing, LLM is used to rewrite the input document into semantic blocks in a uniform format. A semantic block is a text unit with complete semantic logic; each semantic block includes the query question and context-related information. This reduces interference from irrelevant or redundant information in the document, providing a more reliable context for subsequent modeling.

[0087] The predefined pattern also specifies the word count requirements for each part, thus forming a structured knowledge representation. For example, the question should not exceed 200 words, and the contextual information should not exceed 2000 words. Therefore, the transformed document is divided into a set S of multiple independent semantic blocks, each containing a query question of less than 200 words and contextual information of less than 2000 words.

[0088] By using LLM, the original document is reconstructed according to a predefined pattern, thereby reorganizing the document into question-related semantic blocks, reducing possible invalid or redundant information and improving the signal-to-noise ratio.

[0089] Step S003: Segment the semantic blocks and construct the vector database of the knowledge base using text vectorization operations.

[0090] Segmentation refers to dividing a semantic block into one or more text blocks. Specifically, it can be segmented according to the actual length or by a fixed size. For example, when the length of a semantic block exceeds a set threshold, it can be segmented by a fixed size while retaining a preset proportion of overlapping content to ensure contextual coherence; when the length of a semantic block does not exceed the set threshold, it can be segmented according to the actual length.

[0091] When constructing the vector database, the first step is text vectorization. Text vectorization involves converting text into numerical vectors, providing a computational foundation for semantic matching. Existing methods can be used for text vectorization, such as BERT or Sentence-BERT models for text embedding representation, which will not be elaborated upon here. Next, the segmented text chunks are saved, along with their attributes and their correspondence with semantic blocks, allowing for mapping to the location of semantic blocks. This facilitates finding all question-related chunks during subsequent retrieval, ensuring the integrity of the context and providing reliable reference content for the generation process, thus enhancing the reliability of the generated answers.

[0092] By using a vectorized storage and retrieval mechanism, the computational overhead of traditional methods that rely on complex model reasoning is reduced, enabling the system to quickly respond to the fault analysis needs of different industrial scenarios.

[0093] Step S004: Generate corresponding preset questions based on semantic blocks and construct a query agent.

[0094] Preset questions refer to potential query expressions related to the content of semantic blocks. Specifically, they can be generated from multiple perspectives by using a large language model to generate questions based on contextual information in the semantic blocks, covering the diverse query methods that users may use.

[0095] A query proxy is a mapping system between questions and contextual information. Specifically, it can be constructed by storing the embedding vectors of preset questions and their index relationships with semantic blocks, enabling question expansion and precise matching. By automatically generating diverse preset questions to build query proxies, the deficiency in retrieval coverage caused by the uniformity of user query expressions is overcome.

[0096] By generating pre-defined questions for semantic block content, the model's ability to understand user queries can be enhanced by matching these pre-defined questions, thereby improving the recall rate of retrieved correct answers and the accuracy of generated question answers.

[0097] Existing technologies require repeated model updates and fine-tuning when dealing with dynamic industrial scenarios. However, this embodiment directly utilizes the sample generation capabilities of a large language model by constructing a structured knowledge base and an automated query agent. This transforms domain knowledge into standardized semantic blocks and generates extended query questions, avoiding the model fine-tuning process. Only the database needs to be maintained to adapt to the needs of new domains.

[0098] Since the semantic chunks obtained by the Semantic Chunk Algorithm contain contextual information that constitutes a complete answer to a specific question, to ensure content integrity and reduce the risk of information loss, a segmentation strategy combining actual length and fixed size is preferably employed when dividing the semantic chunks. Two principles are followed when segmenting semantic chunks: first, relevant content from different questions should not be in the same chunk; second, the chunk length should comprehensively contain the relevant content for the vast majority of questions to better facilitate understanding and generating high-quality answers.

[0099] Specifically, methods for segmenting semantic blocks include:

[0100] Set the proportion of the actual length-based segmentation method;

[0101] The method for calculating the segmentation length of semantic blocks when they are divided into fixed-size segments, i.e., the text block length, is as follows: calculate the length of context-related information in all semantic blocks, sort the obtained length values ​​from largest to smallest, divide them into two parts according to a set ratio, and then use the length value at the boundary point between the two parts as the segmentation length when dividing into fixed-size segments.

[0102] If the actual length of the semantic block is greater than the segmentation length, the semantic block is segmented into multiple fixed-size text blocks according to the segmentation length. Among them, there is a preset proportion of repeated data between the text blocks to maintain the integrity of the information.

[0103] If the actual length of a semantic block is less than or equal to the segmentation length, the entire semantic block is divided into a single text block. In this case, the length of the chunk is equal to the actual length of the semantic block.

[0104] The preset proportion of duplicate data refers to the amount of information overlap between adjacent segments. By setting duplicate data, the contextual association across segments can be preserved.

[0105] Specifically, the hybrid segmentation strategy dynamically adjusts the combination ratio of two segmentation methods to achieve standardized processing of long texts while preserving the integrity of short texts. The calculation of fixed segmentation block lengths is automatically optimized based on document features, avoiding subjective biases caused by manual intervention. When processing long semantic blocks, a fixed segmentation method with overlapping windows is adopted. By setting a reasonable repetition ratio, an information buffer is formed at the block boundaries, effectively preventing the severing of key contextual information. Phrase semantic blocks directly retain their complete structure, avoiding unnecessary segmentation operations. This hierarchical processing mechanism ensures both information integrity and processing efficiency.

[0106] In one embodiment of the present invention, the method for generating corresponding preset questions based on semantic blocks and constructing a query proxy includes: generating multiple preset questions corresponding to the context-related information in each semantic block using a large language model; embedding text into all generated preset questions to obtain preset question embedding vectors; and saving the preset question embedding vectors and the index relationship between the preset questions and the corresponding semantic blocks into a query proxy database to form a query proxy.

[0107] Because there are often many ways to describe and ask questions about the same fault, multiple possible questions can be pre-constructed based on the answer, thereby enabling more accurate questions to be matched during the retrieval process, and thus improving the feasibility of finding the required answer.

[0108] Specifically, a question-based approach can be used to guide the LLM to generate questions. That is, the LLM is prompted to generate a set of questions whose answers are based on the context of the semantic block and are similar to the question content. This approach provides some control over the structure of the input and output, thereby improving the accuracy of language model generation; it not only makes the output more precise but also exhibits stronger coherence and context relevance.

[0109] Because the preset question maintains a strong correlation with the question and context of the semantic block, the model can improve its understanding of user questions by matching them with the preset question.

[0110] All generated preset questions are embedded with text, and the preset questions, their corresponding embedding vectors, and the index relationship between the preset questions and their corresponding semantic blocks are saved to the query agent database to form a query agent. This allows the location of the semantic block to be easily mapped through the index relationship.

[0111] The query agent provides additional query paths for the content retrieval process, enabling more convenient and accurate finding of context related to the question. Furthermore, these pre-defined questions are set to a shorter length during production, focusing more on the question information and effectively improving text recall during the retrieval process.

[0112] Next, refer to Figures 3-5 To describe another embodiment of the present invention, a root cause analysis method based on a large language model is described.

[0113] like Figure 3 As shown, the root cause analysis method based on a large language model in this embodiment includes steps S310-S340. The following describes each step.

[0114] Step S310: Obtain user questions.

[0115] For example, please refer to Figure 4 The user's problem is that the vacuum condition is not met during mold changing. When searching for answers to the input problem, the user's problem only describes the state and not the faulty equipment. When the user's question is not precise enough, the existing retrieval model often cannot understand the question. The relevance between the question and the answer is only 0.6091, and the existing retrieval model cannot determine whether the retrieved answer is accurate.

[0116] Step S320: Based on the user's question, retrieve the first search result from the knowledge base using a contextual retrieval tool. Specifically, this includes:

[0117] Based on the user's question, a set of text blocks consisting of one or more related text blocks is obtained by matching from the knowledge base;

[0118] Retrieve the semantic blocks corresponding to the text block set;

[0119] The first context-related information related to the user's question is obtained from the corresponding semantic block and used as the first search result.

[0120] A semantic block contains complete semantic information, and a semantic block may be divided into multiple text blocks and stored in a knowledge base. Since the text blocks store the correspondence between the text blocks and the semantic blocks, it is easy to find all the text blocks that are divided from the same semantic block, so as to ensure the integrity of the information.

[0121] Please see Figure 5 Given an input query q, i.e., a user question, the model first calculates the similarity between the user question and text blocks stored in the knowledge base for matching. Specifically, the user question is vectorized through text embedding. The cosine similarity between this vector and the text block vectors in the knowledge base is calculated. Chunks whose similarity meets a preset similarity threshold are used as the matching results, i.e., the chunk set C. Then, the semantic block identifiers sids (where sids represents multiple sids) from the metadata of the matched chunks are passed to the retrieval model Γ. c The model then matches the first search result E from the context database. c This process can be described as follows:

[0122] C=Γ c (q,CDC)

[0123]

[0124] Wherein, CDC represents the Context Database Collection, which serves as a knowledge base. x This represents the x-th chunk in the matching chunk set C. This represents the semantic block identifier stored in the x-th matched chunk.

[0125] Step S330: Obtain the second search result from the knowledge base by querying the proxy search engine.

[0126] Please continue reading. Figure 5 When a user's query reaches the query agent retrieval server, the query agent first retrieves preset questions that match the user's query to obtain the user's expected question. Specifically, this is done by calculating the similarity between the user's question and the preset questions stored in the query agent. The final matched preset questions can be one or more, i.e., the user's expected questions. Then, it retrieves the context associated with the preset questions. Because the query agent database stores the index relationship between preset questions and corresponding semantic blocks, it can match the semantic block identifier (sid) stored in the preset questions and retrieve all chunks segmented from the semantic block associated with the preset questions, returning them as the user's search results.

[0127] Here, when matching by calculating the similarity between the user question and the preset questions stored in the query agent, the user question can first be vectorized through text embedding. Then, the cosine similarity between the user question embedding vector and the preset question embedding vector stored in the query agent is compared, and one or more preset questions whose similarity meets the preset similarity threshold are taken as the expected questions corresponding to the user question.

[0128] Based on the expected question, second contextual information related to the expected question is retrieved from the knowledge base and used as the second retrieval result.

[0129] Specifically, the method involves retrieving semantic blocks associated with the preset question from the context database set using the semantic block identifiers stored in the preset question, and using these as the second retrieval result.

[0130] Please continue reading. Figure 5 The problem retrieval model is Γ a The context retrieval model is Γ c Given an input query q, the model first matches a predefined set of questions Q from the question database, and then passes the semantic block identifiers sids from the metadata of the matched predefined questions to the retrieval model Γ. c The model then matches the second search result E from the context database. q This process can be described as follows:

[0131] Q = Γ a (q,QAC)

[0132]

[0133] Where QAC represents the Query Agent Collection, CDC represents the Context Database Collection, and q x This represents the x-th preset question in the set of preset questions that were matched. This represents the semantic block identifier saved in the xth preset question that was matched.

[0134] In both of the above retrieval processes, a two-stage retrieval approach is used to obtain the context relevant to the user's query. Both require two database accesses, each serving a different purpose: the proxy retrieval mechanism provides a path to first match possible questions and then find the answer. The context retrieval mechanism, on the other hand, ensures that no semantic information is overlooked.

[0135] Step S340: Based on the first and second search results, the root cause analysis results are obtained by optimizing the results through the result optimizer.

[0136] The specific process includes:

[0137] Using the metadata stored in the first search result Ec and the second search result Eq, specifically the semantic block identifiers in the metadata, the first search result E... c Second search result E q This involves deduplicating and merging the first and second context-related information to obtain semantically complete context information; for example, if semantic blocks have the same identifier, they need to be merged.

[0138] The relevance of contextual information to the user's question is evaluated and a relevant score is assigned using an existing custom re-rating algorithm called BgeRerank. Specifically, the similarity between the two is calculated using metrics such as cosine similarity to measure the semantic closeness between the contextual information and the user's question.

[0139] The context information is reordered based on the relevant scores, and the top K results that are most relevant to the user's question are retained first. Here, K is a pre-defined hyperparameter, K is a positive integer, and K≥1.

[0140] Reordering ensures that the provided query semantics closely match the user's question.

[0141] Next, refer to Figure 6 This invention describes a root cause analysis method based on a large language model, according to one embodiment of the present invention.

[0142] To verify the effectiveness of the method in this embodiment, the RAGAS framework was used to evaluate the dataset, the preset question, and the model performance. The specific evaluation metrics used included recall, relevance, fidelity, and accuracy.

[0143] Recall rate refers to the ability to retrieve all relevant information about the answer. The formula for calculating recall is:

[0144]

[0145] Here, TP (True Positives) represents the number of correctly retrieved relevant documents, and FN (False Negatives) represents the number of unretrieved relevant documents.

[0146] Fidelity refers to evaluating the semantic similarity between texts using the embedding vectors themselves (cosine vector distance); this application uses cosine similarity for measurement, and the calculation formula is as follows:

[0147]

[0148] Where A = (a1, a2, ..., an) and B = (b1, b2, ..., bn) represent two n-dimensional infeed vectors.

[0149] Fidelity measures whether the claims in an answer can be inferred from the context. The formula for calculation is:

[0150] score=inferred_statements / generated_statements

[0151] Here, inferred_statements represents the number of statements that can be inferred from the given context, and generated_statements represents the number of statements that can be extracted from the generated answer.

[0152] Accuracy is used to measure whether an answer is the correct answer to a question, and the formula is as follows:

[0153]

[0154] in, E is the embedded representation of the generated i-th question. o It is the embedding of the original problem, and N is the number of problems generated.

[0155] Figure 6 This diagram illustrates the overall framework of a root cause analysis model according to yet another embodiment of the present invention, as shown below. Figure 6 As shown, the root cause analysis model includes a document preprocessing module, a vector library construction module, a query proxy construction module, and a retrieval optimization module.

[0156] The document preprocessing module rewrites the collected documents into semantic blocks in a uniform format. This module integrates multiple open-source tools to achieve format conversion. Based on a predefined schema, it uses LLM to rewrite the input documents into semantic blocks in a uniform format.

[0157] The documents in this embodiment are a collection of actual factory production process documents, including: daily maintenance records of equipment and production lines by workers; documents introducing equipment, product manufacturing processes, and production techniques; and Q&A records from relevant experts. The significant differences in document formats across different types hinder knowledge base construction and contextual content retrieval. Therefore, the three types of original documents undergo format conversion. A large language model is used for preprocessing and extraction of semantic blocks specific to certain questions from the text, generating three datasets: an Industrial Knowledge (IK) dataset, a Question-Answer (QA) dataset, and a Product Records (PR) dataset. These datasets consist of 150, 239, and 1070 text segments, respectively. To ensure the effectiveness of subsequent processing, the document preprocessing module assigns a unique identifier to each semantic block.

[0158] The document preprocessing process can be represented as:

[0159] S=PreprocessWithLLM(d1,d2,…,d m )={s1,s2,…,s n}

[0160] Where d1, d2, ..., d m Let s1, s2, ..., s be the documents in document set D. n Let S and S represent semantic blocks in the semantic block set S, respectively. n The unique identifier is sid n express.

[0161] Table 1 is an example table of semantic blocks in one embodiment of the present invention, which includes two pieces of information: query and context. “Query” serves as a reference example for generating a preset question, and “Context” refers to the text content related to the question's answer.

[0162] Table 1

[0163]

[0164] Table 2 shows the statistics of the number of semantic blocks in the dataset and the evaluation results of the relevance and accuracy of Query and Context. As shown in Table 2, 150, 239 and 1070 semantic blocks were extracted from the knowledge base documents, QA records and production maintenance records, respectively. The cosine similarity between Query and Context in the semantic blocks all exceeded 0.72, and the accuracy reached more than 0.82, indicating that the Context in the extracted semantic blocks can be used as the true answer to the Query.

[0165] Table 2

[0166]

[0167] The vector library building module segments semantic blocks and builds a vector database.

[0168] The Semantic Chunk Algorithm can employ a hybrid approach of semantic-based and fixed-size chunking. The following is the algorithm's code:

[0169]

[0170]

[0171] During the vector library construction process, the module uses the Context Database Constructor to build the vector database. While saving the segmented text, it also saves the chunk attributes and their correspondence with semantic blocks through metadata. Specifically, the metadata is a triple containing a unique identifier for the chunk, its data type (indicating that this data is used for content retrieval), and the semantic block name. n The identifier. For example, semantic blocks s n The content that needs to be saved to the database for the k-th chunk is:

[0172]

[0173] in, and These represent the chunk and its corresponding embedding vector, respectively. This is a unique identifier for the chunk. The string "context" indicates the type, signifying that this data is used for content retrieval. n For the corresponding semantic block s n Unique identifier, This refers to the Context Database Collection (CDC).

[0174] The query agent building module includes an Expect Query Generator and a Query Agent Constructor, which are used to generate multiple Expect Query corresponding to the rewritten Semantic Block and build a query agent.

[0175] The purpose of the preset question generator is to generate multiple high-quality questions for each semantic block, with the context as the answer. To accurately reflect the core content of the context, the generator generates questions for each semantic block. n The process of generating k preset questions can be represented as:

[0176]

[0177] Where q and c represent semantic blocks s, respectively. n The query question and related context information are given, where θ represents the knowledge model parameters that the LLM possesses for understanding and generating knowledge, and the preset question EQ is given. sn There are several different ways to express this, namely

[0178] In this embodiment, the preset questions are generated using a large language model. Five questions that are highly relevant to the queries in the dataset and use semantic blocks as the standard answers are selected as preset questions. Table 3 shows an example of a semantic block preset question.

[0179] Table 3

[0180]

[0181] Among them, the questions in the Expect Query are all associated with a context when constructing the knowledge base. When a user's query question matches any of these Expect Query questions, the most likely answer can be determined directly, thus bypassing the search for potential answers.

[0182] Table 4 lists the correlation between the preset questions and the query questions in the dataset, as well as the accuracy of the answers.

[0183] Table 4

[0184]

[0185] Among these, a relevance score of 0.7 or higher indicates that the pre-defined question in this paper is similar to the query question in the dataset; an accuracy score exceeding or approaching 0.8 is similar to the accuracy of the query question and answer in the dataset. This demonstrates that the text block can indeed serve as the answer to the pre-defined question constructed in this paper.

[0186] The query proxy builder uses metadata to store the relevant attributes of the preset question and its links to semantic blocks. The metadata is a triple that includes: the identifier of the preset question, its data type, and the semantic block name. n The identifier. For example, semantic blocks s n The content that needs to be saved to the database for the k-th preset question, QAC (QueryAgent Collection), is:

[0187]

[0188] in, and These represent the preset question and the corresponding embedding vector, respectively. This is a unique identifier for the preset question. The string "query" indicates the type, signifying that this data is used for question retrieval. sid n For the corresponding semantic block s n Unique identifier, Represents semantic block s n The content that needs to be saved to the database for the kth preset question.

[0189] The search optimization module retrieves relevant content from both the question and the context, and then selects the most relevant results by reordering them.

[0190] When a user query reaches the Query Agent Retriever, the Query Agent first retrieves a preset question that matches the user query, and then retrieves the context associated with the preset question, which is returned as the user's search result.

[0191] When a user query reaches the Contexts Retriever, the retriever directly searches for context information that matches the user query. At this point, the search results may contain text blocks with incomplete semantics, so it is also necessary to retrieve other text blocks with the same semantic meaning.

[0192] The result optimizer deduplicates and merges the retrieved results to obtain a series of semantically complete contextual information.

[0193] This embodiment proposes a root cause analysis method for industrial production failures based on a large language model. This method uses predefined patterns to reconstruct the original document into semantic blocks to handle massive and complex industrial production data. In addition, a series of preset questions are generated for the content of the semantic blocks, establishing a mapping relationship between questions and answers, and constructing a root cause analysis vector knowledge base. Finally, a retrieval agent based on preset questions is set in the model to enhance the model's understanding of user queries by matching preset questions, thereby improving the success rate of contextual retrieval.

[0194] The effectiveness of the method of the present invention will be verified through experiments below.

[0195] Table 5 compares the performance of the model in this embodiment with the MemoRAG method on three datasets, showing a comparison of the RAG retrieval and generation results with zero samples.

[0196] Table 5

[0197]

[0198] In Table 5, the recall scores of the model in this embodiment are 0.9896, 0.8049, and 0.7945, respectively, which are 35.54%, 18.07%, and 26.62% higher than the recall scores of MemoRAG. This indicates that the method of the present invention has good potential to improve the text recall rate of language models in specific domain contexts. The improvements in fidelity are 11.61%, 7.84%, and 7.52%, indicating that the answers generated by the model used in this invention are closer to the retrieved content.

[0199] In the ablation experiments, the effectiveness of the proposed modules was investigated. Specifically, these models were evaluated on the collected datasets. Table 6 shows the ablation experiment evaluation scores. As shown in Table 6, the method "w / o preprocessing" indicates the absence of a document preprocessing module. In the method "w / o presupposition problem," the query agent construction module and query agent retrieval module are inactive. The results in Table 6 show that the proposed methods significantly improve text recall, increasing by 50.74%, 42.07%, and 36.70% respectively compared to "w / o preprocessing," and by 31.34%, 17.25%, and 16.88% respectively compared to "w / o presupposition problem." Furthermore, they perform best in fidelity and accuracy, confirming their effectiveness.

[0200] Table 6

[0201]

[0202]

[0203] Figure 7a and Figure 7bThe document showcases the content of the reference document and a sample report, including fault analysis, causes, and maintenance recommendations. It can be seen that the answers to questions regarding abnormal situations are quite similar to those in the reference document. The output fault type, "external," is consistent with the maintenance records. The causes, "cable interference with the moving mechanism, wear and breakage," and the mitigation measures, "check other trolleys for cable interference with the running mechanism," can both be found within the relevant context of the dataset. However, the answers generated by Memoreg (a memory-driven retrieval enhancement generation framework) are more general. For example, mitigation measures include "posting warning signs" and "conducting comprehensive quality control and maintenance on relevant components throughout the production line." These descriptions fail to pinpoint the specific faulty equipment and propose corresponding corrective actions.

[0204] Therefore, the root cause analysis method provided by this invention is crucial in offering users appropriate explanations of causes and suggestions for subsequent improvements. Compared with existing methods, the method of this invention exhibits excellent performance, with significant improvements in retrieval recall and the fidelity of generated answers.

[0205] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program that is executed by the processor. When the computer program is executed by the processor, it causes the device equipped with the processor to perform the root cause analysis method based on a large language model as described in any of the above embodiments.

[0206] Next, refer to Figure 8 This describes an example electronic device 100 for implementing the root cause analysis method based on a large language model according to embodiments of the present invention.

[0207] like Figure 8 As shown, the electronic device 100 includes a processor 110, a memory 120, and a communication interface 130. The processor 110, the memory 120, and the communication interface 130 can be interconnected and communicate via a communication bus 140 and / or other forms of connection mechanisms (not shown).

[0208] It should be noted that Figure 8 The components and structure of the electronic device 100 shown are merely exemplary and not limiting; the electronic device may have other components and structures as needed.

[0209] Optionally, the communication interface 130 may also include a transmitter and / or a receiver.

[0210] The processor 110 may be a microcontroller unit (MCU), a central processing unit (CPU), a digital signal processor (DSP), a microcontroller, an embedded device, or other processing units with data processing and / or instruction execution capabilities.

[0211] The memory 120 can be various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM), cache memory, synchronous dynamic random access memory (SDRAM), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may also be stored on the computer-readable storage medium, and the memory 120 can execute these program instructions to implement the root cause analysis method based on a large language model in the embodiments of the present invention described above.

[0212] This application also provides a storage medium storing a computer program that runs on a computer. When the computer program runs, it causes the computer to execute the root cause analysis method based on a large language model as described in any of the above embodiments.

[0213] Finally, it should be noted that the above technical solution is only one embodiment of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the methods described in the above specific embodiments of the present invention. Therefore, the methods described above are only preferred and have no limiting significance.

Claims

1. A root cause analysis method based on a large language model, characterized in that: The root cause analysis method includes: Obtaining user questions; Based on the user's question, the first search result is obtained from the knowledge base pre-constructed using a large language model; Based on the user's question, the corresponding expected question is obtained through a pre-built query agent; Based on the desired question, a second search result is obtained from the knowledge base; Based on the first search result and the second search result, the root cause analysis result is obtained through optimization.

2. The root cause analysis method based on a large language model according to claim 1, characterized in that: Before obtaining user issues, the root cause analysis method also includes: Obtain relevant documents for the target industrial sector; Data preprocessing is performed on relevant documents in the target industrial field, and semantic blocks in a unified format are generated through a large language model. Each semantic block includes the query question and context-related information. The semantic blocks are segmented, and a vector database of the knowledge base is constructed using text vectorization operations. Based on the semantic block, a corresponding preset question is generated, and a query agent is constructed.

3. The root cause analysis method based on a large language model according to claim 2, characterized in that: The relevant documents include academic articles, books, and reports, record forms, and QA documents from industrial production. The format of the semantic blocks includes: the number of words in the question is less than 200, and the number of words in the context-related information is less than 2000.

4. The root cause analysis method based on a large language model according to claim 2, characterized in that: The method for segmenting the semantic block includes: Set the proportion of the actual length-based segmentation method; Calculate the segmentation length when the semantic block is segmented into a fixed size; If the actual length of the semantic block is greater than the segmentation length, the semantic block is segmented into multiple fixed-size text blocks according to the segmentation length, wherein there is a preset proportion of repeated data between the text blocks; If the actual length of the semantic block is less than or equal to the segmentation length, the entire semantic block is treated as a single text block.

5. The root cause analysis method based on a large language model according to claim 4, characterized in that: A method for retrieving a first search result from the knowledge base based on the user question includes: Based on the user question, a set of text blocks consisting of one or more related text blocks is obtained by matching from the knowledge base; Obtain the semantic block corresponding to the set of text blocks; The first context-related information related to the user's question is obtained by matching from the corresponding semantic block and used as the first retrieval result.

6. The root cause analysis method based on a large language model according to claim 5, characterized in that: The second search result includes second context-related information. Based on the first search result and the second search result, the root cause analysis result is optimized and obtained, including: The first and second context-related information are deduplicated and merged to obtain semantically complete context information; Assess the relevance of the contextual information to the user's question and assign relevant scores; The context information is reordered based on the relevant scores, and the top K results with the highest relevance to the user's question are retained first, where K is a positive integer and K≥1.

7. The root cause analysis method based on a large language model according to claim 2, characterized in that: The method for generating corresponding preset questions based on the semantic blocks and constructing a query proxy includes: For each semantic block, a large language model is used to generate multiple pre-defined questions corresponding to the context-related information within it. Text embedding is performed on all generated preset questions to obtain preset question embedding vectors; The preset question embedding vector and the index relationship between the preset question and the corresponding semantic block are saved to the query proxy database to form a query proxy.

8. The root cause analysis method based on a large language model according to claim 2, characterized in that: The method for obtaining the corresponding expected question through a pre-built query agent based on the user question includes: embedding the user question into text to obtain a user question embedding vector; By comparing the cosine similarity between the user question embedding vector and the preset question embedding vector stored in the query agent, one or more preset questions whose similarity satisfies a preset similarity threshold are obtained. One or more preset questions are obtained as expected questions corresponding to the user questions.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program executed by the processor, the computer program, when executed by the processor, causing the device on which the processor is installed to perform the root cause analysis method based on a large language model as described in any one of claims 1-8.

10. A storage medium, characterized in that, The storage medium stores a computer program that runs on a computer and, when running, causes the computer to perform the root cause analysis method based on a large language model as described in any one of claims 1-8.

Images