Chaotic engineering atomic fault library question-answering method and system based on retrieval enhancement generation

Abstract

The invention discloses a chaos engineering atomic fault library question-answering method and system based on retrieval enhancement generation, and the method comprises the steps: firstly obtaining an atomic fault library file, storing the atomic fault library file in a JSON format, carrying out the slicing through a JSON slicing algorithm, and converting a node into a vector through an embedded model; then, three retrievers are constructed based on the atomic fault library, the three retrievers are used for retrieval according to user questions, and matched information nodes are obtained; deduplication and rearrangement are carried out on the obtained information nodes to obtain a text block set, and the text block set is spliced into complete atomic fault information; and finally, inputting the text blocks and the user questions into the large model, constructing cue words, and guiding answering according to the types of the user questions. According to the method, the integrity of single atomic fault information is greatly reserved through the JSON format and the JSON slicing algorithm, meanwhile, similar atomic faults can be matched through the mixed retriever, and the integrity, reliability and usability of a question answering system are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and data retrieval technology, and in particular to a chaos engineering atomic fault library question-answering method and system based on retrieval enhancement generation. Background Art

[0002] The atomic fault library contains a variety of complex fault types. In order to help users quickly understand detailed fault information, it is crucial to find an efficient and accurate question-and-answer method.

[0003] Retrieval-Augmented Generation (RAG) technology can significantly improve the quality of question-answering and greatly reduce the hallucination problem of large models. There are three challenges in building a chaos engineering atomic fault library question-answering system based on retrieval-augmented generation: First, the existing data node slicing method relies on the artificially set slice length, which causes the atomic fault information to be truncated, greatly affecting the integrity of the large model's answers. Second, if the recalled atomic faults are not related to the user's questions, the reliability of the atomic fault library question-answering system will be greatly reduced. Therefore, it is crucial to accurately recall the atomic faults that are closely related to the user's questions. Third, the questions raised by users may not accurately include the names or keywords of the atomic faults, but rather fuzzy searches. The atomic fault library question-answering system should also be able to give reasonable answers to broad questions to improve the availability of the question-answering system.

[0004] It can be seen that the integrity, reliability and availability of the existing chaos engineering atomic fault library question-answering system based on retrieval enhancement generation need to be further improved and studied. Summary of the invention

[0005] In view of the above problems, the present invention provides a chaos engineering atomic fault library question and answer method and system based on retrieval enhanced generation, which is used to improve the integrity, reliability and availability of the existing question and answer system.

[0006] The present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for answering a question of a chaos engineering atomic fault library based on retrieval enhancement generation, comprising the following steps:

[0008] S1. Obtain the atomic fault library file, save the atomic faults in JSON format, and store the atomic faults of the same category in the same TXT file;

[0009] S2. Use the JSON slicing algorithm to slice the TXT files of all atomic faults and use the embedding model to convert the nodes into vectors;

[0010] S3, constructing three search engines based on the atomic fault library, and using the three search engines to search respectively according to the user's questions to obtain the matched information nodes; the three search engines are: a similarity search engine based on node embedding, a keyword search engine, and a similarity search engine based on document summary embedding;

[0011] S4, de-duplicate and rearrange the information nodes obtained by the three retrievers to obtain a set of text blocks, and splice them into complete atomic fault information;

[0012] S5. Input the text block and user questions into the big model, construct prompt words, and guide the answer according to the user's question type.

[0013] Furthermore, in step S1, the atomic fault library file is in csv format, and atomic faults are extracted line by line from the csv file, and the fault fields of each atomic fault are extracted, including: fault name, fault parameters, and fault explanation; after preprocessing, the atomic faults are processed into JSON format, and the atomic faults in JSON format are clustered according to the naming rules of the fault names to form a TXT file of atomic faults.

[0014] Furthermore, in step S2, the JSON slicing algorithm is used to slice the TXT files of all atomic faults, as follows:

[0015] S201, construct a JSON node splitter to extract a JSON list from each TXT file of an atomic failure;

[0016] S202, converting the JSON list into plain text one by one, where one segment of plain text corresponds to one atomic fault node, and each atomic fault contains only one atomic fault information;

[0017] S203. Convert the text of each node into a vector using the bce-embedding-base_v1 model to obtain a node vector, which is stored in JSON format.

[0018] Furthermore, in step S3, a similarity retriever based on node embedding is used: the user question is converted into a vector through the bce-embedding-base_v1 model, and its similarity with each node embedding is calculated to match several atomic fault nodes.

[0019] A keyword retriever is used: the node keywords are extracted one by one through the keyword extraction algorithm, the keywords contained in the user's question are confirmed, and finally several nodes are matched.

[0020] A similarity retriever based on document summary embedding is used: the TXT documents obtained in step 1 are queried with the big model separately to obtain the big model's summary of each TXT document, including the summary of atomic fault information and the prediction of the user's possible questions about this document. The document summary is converted into a vector through the bce-embedding-base_v1 model, and finally the embedding similarity between the user's question and the document summary is calculated to match all nodes of similar documents.

[0021] Furthermore, in step S4, the information nodes matched by the three searchers are de-duplicated and re-arranged. The specific method is as follows:

[0022] S401, taking the information nodes that are repeatedly matched by the three searchers as selected nodes directly and adding them to the text block;

[0023] S402, among the information nodes that are not repeatedly matched, retain the nodes matched by the keyword searcher and add the text block;

[0024] S403, re-ranking the remaining information nodes matched by the document summary embedding-based similarity retriever and the node embedding-based similarity retriever through the bge-reranker rearrangement model, selecting several nodes with high correlation, and adding them to the text block in order;

[0025] S404: When the preset length of the text block is exceeded, the remaining information nodes are discarded.

[0026] In the second aspect, the present invention provides a chaos engineering atomic fault library question and answer system based on retrieval enhancement generation, which is used to implement any of the above-mentioned chaos engineering atomic fault library question and answer methods, including: a fault information processing module, a user problem processing module, a hybrid retrieval module, a node integration module, and a prompt question and answer module.

[0027] The fault information processing module processes the atomic fault library information into structured JSON nodes and archives similar atomic faults into summary nodes;

[0028] User question processing module, extracts keywords from user questions and converts them into embeddings;

[0029] The hybrid retrieval module builds a node embedding-based similarity retriever, a keyword retriever, and a document summary embedding-based similarity retriever, and retrieves relevant information nodes based on the three retrievers;

[0030] The node integration module removes duplicates and rearranges the information nodes matched by the search engine, controls the length of the integrated text blocks, and forms a highly relevant text set;

[0031] Prompt Q&A module: extract keywords related to chaos engineering and system stability from user questions; design prompt words to optimize the big model’s summary of document content and optimize the big model’s output specifications for specific issues.

[0032] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned chaos engineering atomic fault library question and answer methods based on retrieval-enhanced generation is implemented.

[0033] The technical solution of the present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in any of the above-mentioned chaos engineering atomic fault library question and answer methods based on retrieval enhancement generation are implemented.

[0034] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0035] 1. The atomic fault library question-and-answer method of the present invention largely retains the integrity of single atomic fault information through the JSON format and the JSON slicing algorithm.

[0036] 2. The present invention constructs three types of search engines for hybrid search. When a user question contains a certain fault name accurately, the keyword search engine can accurately retrieve the relevant atomic faults. At the same time, when the user question is vague and broad, the similarity search engine based on document summary embedding can match a class of similar atomic faults.

[0037] 3. The atomic fault library question and answer system of the present invention can effectively improve the integrity, reliability and availability of the existing chaos engineering atomic fault library question and answer system based on retrieval enhancement generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the flow chart of the atomic fault library question-answering method of the present invention;

[0039] Figure 2 This is a framework diagram of each module of the atomic fault library question and answer system of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in the field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0041] Embodiment 1:

[0042] This embodiment provides a method for answering questions based on a chaos engineering atomic fault library generated by retrieval enhancement. Figure 1 As shown, the following steps are included:

[0043] First, obtain the atomic fault library file, pre-process the atomic faults into JSON format, and store similar atomic faults of the same category in the same TXT file;

[0044] Secondly, use the JSON slicing algorithm to slice all atomic fault TXT files and use the embedding model to convert the nodes into vectors;

[0045] Thirdly, three retrievers are constructed based on the atomic fault library, namely, a similarity retriever based on node embedding, a keyword retriever, and a similarity retriever based on document summary embedding, to complete the user question retrieval process;

[0046] Then, the segments recalled by the three retrievers are deduplicated and rearranged to obtain a set of text blocks;

[0047] Finally, the text block is combined with the user's question and input into the large model for question and answering.

[0048] Specifically, the first step is to preprocess the atomic fault library information, which mainly includes the following steps:

[0049] Extract atomic faults line by line from the csv file and encapsulate them in JSON format, including: fault name, fault parameters and their explanations, in English.

[0050] Here are some examples:

[0051] Similar atomic faults are filed in the same TXT file. For example, the fault named docker process kill and the fault named docker process oom are obviously caused by docker, so they will be placed in the docker.TXT file.

[0052] Specifically, in the second step, all the above TXT documents are sliced ​​using JSON slicing and converted into vectors. The main steps are as follows:

[0053] Construct a JSON node splitter, which can extract JSON lists from TXT documents and convert JSON into plain text one by one. One piece of plain text corresponds to an atomic fault node.

[0054] Here are some examples:

[0055] Furthermore, the text of each node is converted into a vector using the bce-embedding-base_v1 model. The converted vector is stored in the local disk in JSON format, in the form of {node number: text embedding}.

[0056] Specifically, in the third step, three retrievers are constructed based on the atomic fault library, which is mainly divided into the following steps:

[0057] First, a similarity retriever based on node embedding is constructed.

[0058] The node vector obtained in the second step is recorded as nodes, and the user question is converted into a vector, recorded as query. The cosine similarity of the vector is calculated as follows: .

[0059] Recall the nodes with the highest similarity.

[0060] Then, building a keyword retriever is mainly divided into the following steps:

[0061] 1. Extract node keywords: segment words, remove stop words and determine keywords based on word frequency.

[0062] For each fault information node, the most important thing is the fault name, which is generally composed of 3-5 words. Therefore, in this embodiment, the number of node keywords is set to 6.

[0063] 2. Extract keywords from user questions: segment words, remove stop words, and use prompt words to enable the large model to extract keywords from user questions. The prompt word template is:

[0064] "You are a keyword extraction assistant. Users will ask questions related to chaos engineering and system stability. Please find out the specific fault name or general direction that users are concerned about. Avoid stop words and finally generate {number} keywords, and output them in a comma-separated format: 'KEYWORDS: <keywords>'

[0065] The user's question is: {query_str}"

[0066] 3. Find the nodes that contain the most keywords in the user’s question as the matched nodes.

[0067] Finally, building a similarity retriever based on document summary embedding is mainly divided into the following steps:

[0068] 1. Use the big model to summarize each TXT document and predict the questions that users may ask about this information.

[0069] 2. Convert the questions summarized by the big model into a vector, recorded as summary; convert the user questions into a vector, recorded as query.

[0070] 3. Calculate the cosine similarity between summary and query, take several summaries with the highest similarity, and recall all nodes corresponding to these summaries.

[0071] Specifically, in the fourth step, the segments recalled by the three retrievers are deduplicated and rearranged to obtain a set of text blocks, which is mainly divided into the following steps:

[0072] The nodes that are matched repeatedly are directly used as the final selected nodes, and the first node matched by the keyword searcher is retained if the node is not matched repeatedly.

[0073] The remaining nodes are reordered through the bge-reranker model, and the nodes are added to the text block in order. When the text block length is exceeded, the remaining node information is discarded.

[0074] Specifically, in the fifth step, the text block and the user question are input into the big model. In this embodiment, the big model is Qwen2.5-7B. The big model is used to guide the answer according to the user question type, which is mainly divided into the following steps:

[0075] Prompt word construction guides answering based on the user's question type. The prompt word template is:

[0076]

[0077] Embodiment 2:

[0078] This embodiment provides a chaos engineering atomic fault library question-answering system based on retrieval enhancement generation, such as Figure 2 As shown, including:

[0079] The fault information processing module is configured to: structure the atomic fault library CSV file into atomic fault JSON nodes, and archive the atomic fault JSON nodes by category.

[0080] The user question processing module is configured to: extract keywords from user questions; and convert user questions into embeddings.

[0081] The hybrid retrieval module is configured to retrieve relevant nodes through a similarity retriever based on node embedding and document embedding and a keyword retriever.

[0082] The node integration module is configured to: remove duplicates and rearrange the nodes matched by the retriever, add highly relevant nodes to the text block, control the length of the text block, and form a highly relevant text set.

[0083] The prompt question-answering module is configured to: extract keywords from user questions; summarize document content and predict questions; and provide standardized output for user-specific questions.

[0084] Embodiment three:

[0085] This embodiment provides a computer-readable storage medium, which includes a stored program. When the program is running, the device where the computer-readable storage medium is located is controlled to execute the chaos engineering atomic fault library question and answer method based on retrieval enhanced generation as described in Embodiment 1.

[0086] In an embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the program is executed by a processor, the steps in any one of the chaos engineering atomic fault library question and answer methods based on retrieval enhancement generation in the above embodiments are implemented.

[0087] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.< / keywords>

Claims

1. A chaos engineering atomic fault library question-answering method based on retrieval enhancement generation, characterized in that: The steps include: S1. Obtain the atomic fault library file, save the atomic faults in JSON format, and store the atomic faults of the same category in the same TXT file; S2. Use the JSON slicing algorithm to slice the TXT files of all atomic faults and use the embedding model to convert the nodes into vectors; S3, constructing three search engines based on the atomic fault library, and using the three search engines to search respectively according to the user's questions to obtain the matched information nodes; the three search engines are: a similarity search engine based on node embedding, a keyword search engine, and a similarity search engine based on document summary embedding; S4, de-duplicate and rearrange the information nodes obtained by the three retrievers to obtain a set of text blocks, and splice them into complete atomic fault information; S5. Input the text block and user questions into the big model, construct prompt words, and guide the answer according to the user's question type.

2. According to claim 1, the method for answering questions in a chaos engineering atomic fault library based on retrieval enhancement generation is characterized in that: In step S1, the atomic fault library file is in csv format, and atomic faults are extracted line by line from the csv file, and the fault fields of each atomic fault are extracted, including: fault name, fault parameters, and fault explanation; after preprocessing, the atomic faults are processed into JSON format, and the atomic faults in JSON format are clustered according to the naming rules of the fault names to form a TXT file of the atomic faults.

3. According to claim 2, the method for answering questions in a chaos engineering atomic fault library based on retrieval enhancement generation is characterized in that: In step S2, the JSON slicing algorithm is used to slice the TXT files of all atomic faults as follows: S201, construct a JSON node splitter to extract a JSON list from each TXT file of an atomic failure; S202, converting the JSON list into plain text one by one, where one segment of plain text corresponds to one atomic fault node, and each atomic fault contains only one atomic fault information; S203. Convert the text of each node into a vector using the bce-embedding-base_v1 model to obtain a node vector, which is stored in JSON format.

4. According to claim 3, the method for answering questions in a chaos engineering atomic fault library based on retrieval enhancement generation is characterized in that: In step S3, the node embedding-based similarity retriever records the node vector obtained in step S2.3 as nodes, converts the user question into a question vector as query, and calculates the similarity cosine_sim between the question vector and each node embedding, and the formula is: ； Several nodes with the highest similarity are used as matched information nodes.

5. According to claim 3, the method for answering questions in a chaos engineering atomic fault library based on retrieval enhancement generation is characterized in that: In step S3, the keyword searcher extracts the keywords of the atomic fault nodes and the keywords in the user's question one by one through a keyword extraction algorithm, and performs matching. The specific method is as follows: S301, extracting node keywords: for each atomic fault node, perform word segmentation, remove stop words, and determine keywords based on word frequency; S302, extracting keywords from user questions: segmenting user questions, removing stop words, and using prompt words to enable the large model to extract keywords from user questions; S303 , searching for a number of atomic fault nodes containing the most user question keywords as matched information nodes.

6. The method for answering a question of a chaos engineering atomic fault library based on retrieval enhancement generation according to claim 3 is characterized in that: In step S3, the similarity retriever based on document summary embedding has a retrieval method as follows: S311, query the big model for each TXT file of the atomic fault, and use the big model to summarize each TXT document. The resulting document summary includes: a summary of the atomic fault information and a prediction of possible questions that the user may raise about the document; S312, convert the questions summarized by the large model into summary vectors, and convert the user questions into query vectors; S313. Calculate the cosine similarity between the summary vector and the query vector, select several summaries with the highest similarity, and use the node corresponding to the summary with the highest similarity as the matched information node.

7. The method for answering a question of a chaos engineering atomic fault library based on retrieval enhancement generation according to claim 1 is characterized in that: In step S4, the information nodes matched by the three searchers are deduplicated and rearranged. The specific method is as follows: S401, taking the information nodes that are repeatedly matched by the three searchers as selected nodes directly and adding them to the text block; S402, among the information nodes that are not repeatedly matched, retain the nodes matched by the keyword searcher and add the text block; S403, re-ranking the remaining information nodes matched by the document summary embedding-based similarity retriever and the node embedding-based similarity retriever through the bge-reranker rearrangement model, selecting several nodes with high correlation, and adding them to the text block in order; S404: When the preset length of the text block is exceeded, the remaining information nodes are discarded.

8. A chaos engineering atomic fault library question-answering system based on retrieval enhancement generation, used to implement the chaos engineering atomic fault library question-answering method based on retrieval enhancement generation according to any one of claims 1 to 7, characterized in that: include: Fault information processing module, user problem processing module, hybrid retrieval module, node integration module, prompt question and answer module The fault information processing module processes the atomic fault library information into structured JSON nodes and archives similar atomic faults into summary nodes; The user question processing module extracts keywords from user questions and converts user questions into embeddings; The hybrid retrieval module constructs a node embedding-based similarity retriever, a keyword retriever, and a document summary embedding-based similarity retriever, and retrieves relevant information nodes based on the three retrievers; The node integration module removes duplicates and rearranges the information nodes matched by the retriever, controls the length of the integrated text blocks, and forms a highly relevant text set; The prompt question-and-answer module extracts keywords related to chaos engineering and system stability from user questions; designs prompt words to optimize the big model's summary of document content and optimizes the big model's output specifications for specific questions.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the chaos engineering atomic fault library question and answer method based on retrieval enhancement generation as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the steps in the chaos engineering atomic fault library question and answer method based on retrieval enhanced generation as described in any one of claims 1 to 7 are implemented.

Images