Avionics software defect prevention method based on RAG and optimization hint LLM

By constructing reasoning state trees and combining them with external knowledge sources, and utilizing large language models (LLM) and retrieval enhancement techniques, complex problems in avionics software defect prevention have been solved, improving the efficiency and accuracy of software quality management for avionics systems.

CN119376777BActive Publication Date: 2025-10-24CHINESE AERONAUTICAL RADIO ELECTRONICS RES INST
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411321663.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-23
Publication Date
2025-10-24
Estimated Expiration
2044-09-23

AI Technical Summary

Technical Problem

Existing technologies lack effective defect prevention solutions in avionics systems, especially in the development of complex avionics software. LLMs lack the reasoning ability in their specialized fields and are unable to provide accurate solutions.

Method used

We employ a method based on RAG and optimized prompting LLM to construct a reasoning state tree. Combining the concept of thought chain and retrieval enhancement techniques, we utilize a large language model (LLM) with external knowledge sources and extend the reasoning path to generate solutions through a vector database and Monte Carlo tree search algorithm.

Benefits of technology

It improves LLM's reasoning ability in complex professional fields, provides more accurate and comprehensive defect prevention solutions, and enhances the software quality management level of avionics systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119376777B_ABST
    Figure CN119376777B_ABST
Patent Text Reader

Abstract

The application discloses a kind of based on RAG and optimization prompt LLM avionics software defect prevention method, first execute source data collection processing, subsequently utilize LLM to carry out text cutting and vectorization, constructs vector database to indicate the position of text in vector space;In search and information integration stage, according to the question content in vector database retrieval relevant text block, and these blocks are decoded back to original text;Reasoning state tree extension involves starting from initial state node, by calculating node value score, select and expand path, utilize LLM to generate new reasoning state node, until reach leaf node;In value evaluation stage, use LLM to score reasoning state text, decide whether as solution or continue to expand, until obtain complete solution or reach maximum reasoning step number.The application will effectively combine text information and accurate multi-step reasoning, provide accurate solution by calling large language model.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the field of software quality assurance and project management, and relates to avionics systems, large language models LLM, prompt engineering and retrieval augmented generation technology RAG, and specifically provides an avionics software defect prevention method based on RAG and optimized prompt LLM. BACKGROUND

[0002] In the current aviation field, the application of embedded airborne avionics software has become an important factor in promoting aircraft innovation and performance improvement. The functions and performance level of the aircraft are directly subject to its complex and precise avionics software. These avionics software must meet strict safety standards and certification requirements, because the failure of avionics software can lead to serious consequences. Therefore, strict safety specifications must be followed during the design, development and testing of avionics software to ensure that the aircraft operates reliably in all situations.

[0003] Considering the key role of avionics software in avionics systems, higher requirements are placed on software quality management. According to the Capability Maturity Model CMMI, software quality management includes means such as software quality assurance, peer review, quantitative quality management and defect prevention. Currently, software quality management of avionics system software mainly covers methods such as software quality assurance and peer review, but there is still a lack of mature solutions in defect prevention.

[0004] By utilizing the large amount of relevant data generated in the past avionics software development process, defect and failure record information that has occurred in the development process is extracted, providing knowledge and experience guidance for subsequent development process defect prevention to improve software quality management. This is a possible solution to defect prevention. By making full use of past data, problems that may occur during the development process can be better prevented, thereby improving development efficiency and the overall quality level of avionics systems.

[0005] In recent years, large language models LLM trained on extensive datasets have demonstrated enhanced capabilities in tasks such as semantic recognition and text reasoning, particularly in general question answering. However, the ability of LLM to acquire and accurately process specific knowledge, such as reasoning and answering questions about internal enterprise data, is still insufficient. In this case, it is necessary to integrate LLM with external knowledge sources. Retrieval augmented generation technology RAG introduces a retrieval algorithm based on the similarity of embedding spaces between queries and documents. With the help of these external knowledge, this capability not only broadens the application field of LLM, but also improves the ability to reason and provide solutions to problems in specialized fields. When using private data or in less common fields, LLM based on factual external knowledge can reduce the generation of inaccurate facts and hallucinations.

[0006] Defect problems in avionics systems are often very complex, and the problems that occur may involve various stages such as research and development, coding, integration, testing, etc., or various functional modules of avionics systems, which requires LLM to have higher complex reasoning ability when giving solutions to defect problems. The patent document with the domestic publication number CN117933392A proposes a Monte Carlo tree search method and device based on scientific question answering tasks, which constructs a reasoning tree to improve the reasoning performance of large language models in scientific question answering tasks. In addition, when LLM introduces external knowledge using retrieval enhancement technology, it needs to integrate knowledge data related to various aspects of the problem. Combine the thought of thought chain with RAG, retrieve the relevant content information in each reasoning step to assist reasoning and decision-making, expand the reasoning path, and thus improve the ability of LLM to solve complex professional field problems. SUMMARY

[0007] The purpose of the present application is to provide an avionics software defect prevention method based on RAG and optimized prompt LLM, which adopts the thought of thought chain, constructs a reasoning state tree, and introduces retrieval enhancement technology to improve the ability of LLM to solve complex professional field problems.

[0008] The purpose of the present application is achieved by the following technical solutions:

[0009] An avionics software defect prevention method based on RAG and optimized prompt LLM, comprising the following steps:

[0010] S1: Collect literature and books related to digital avionics technology, collect various version history records and defect sample data in the development process of avionics systems;

[0011] S2: Convert the literature and books related to digital avionics technology into standardized field text data; use LLM to extract change sample data from various version history records, and extract problem content, reason and solution from defect text data, and then generate defect sample data in a unified format;

[0012] S3: Use LLM to segment the field text data and defect sample data, the segmented text blocks are input to LLM for embedding operation, and are converted into vector representations that can capture the meaning and related information of the text, and then a vector database is constructed, the position of each text in the vector database is determined by its corresponding vector;

[0013] S4: According to the content of the defect problem, search for several text block vectors with high relevance in the vector database, decode the searched text block vectors, and convert them into text data;

[0014] S5: Take the defect problem text as the initial reasoning state node of the reasoning state tree, expand along the path with high value score to the leaf node direction until a leaf node is reached; if the leaf node has been visited, combine the reasoning state node text generated by the current leaf node or retrieve the text embedding with high relevance in the vector database through text embedding, after converting the text data, use LLM to generate a new reasoning state node as the child node of the leaf node to expand the branches of the tree; if the leaf node has not been visited, go to S6 and update the access times of the nodes on the current path;

[0015] S6: Combine the reasoning state text of the current node with the input problem and call LLM for scoring; if the LLM evaluation result indicates that the text is sufficient to solve the problem, the text is selected as the final solution; if the text is not sufficient to solve the problem, the text is scored for value and the value score and access times of all visited nodes on the current path are updated; when the number of reasoning steps reaches the preset maximum limit, the leaf node in the path with the highest value score is selected, the reasoning state text and the input problem are combined, and LLM is called again to generate the final solution; if the maximum step limit has not been reached, return to S5 to continue building and evaluating the reasoning path.

[0016] Further, in S1, the relevant field of the enterprise is crawled by using automatic technology to obtain various version history records and defect sample data generated in the actual work of the development process of the avionics system.

[0017] Further, in S3, the collected literature and books related to digital avionics technology field are subjected to optical character recognition, the image format is converted into editable text, non-text elements in the document are removed, non-text characters generated during the OCR process are deleted, OCR errors and spelling errors are repaired, and then the text format is standardized;

[0018] The collected various version history records are analyzed to extract key information such as change content and change reason analysis to form change sample data;

[0019] The problem report data, review data and defect report data are processed to extract problem content, reason and solution, and defect sample data is generated in a unified format;

[0020] The change sample data and defect sample data contain detailed description of defects, scope of influence, solution measures and prevention strategies.

[0021] Further, the specific steps of S5 are as follows:

[0022] S51: Obtain the defect problem text and take the defect problem text as the initial reasoning state node;

[0023] S52: Calculate the value score of each node in the current inference state tree using the upper confidence bound formula, select the node with the highest value score as the next step path selection, and build the path from the root node to the leaf node;

[0024] S53: Check if the leaf node has been visited, if it has been visited, take the inference state text corresponding to the node as input, combine the inference state node text generated so far or according to the embedding of the text, retrieve a number of text embeddings with high relevance in the vector database, merge these data after converting to text data, and call the LLM to generate a number of inference state nodes as child nodes of the leaf node, thereby expanding the inference state tree; if the node has not been visited, go to S6 to score the value of the node to determine whether to include it as part of the inference path;

[0025] S54: After value evaluation, select the first child node generated as the current node and go to S6 again to continue expanding the inference state tree;

[0026] S6 is as follows:

[0027] S61: Combine the inference state text corresponding to the node and the defect problem text, and call the LLM to evaluate the completion of the node in solving the defect problem;

[0028] S62: If the inference state text is not sufficient to solve the problem, perform value evaluation and update the nodes visited on the path and their visit counts;

[0029] S63: If the maximum number of inference steps has been reached, select the leaf node from the path with the highest value score, combine the inference state text corresponding to the node and the defect problem, and call the LLM to generate a final solution; if the maximum number of inference steps has not been reached, return to step S52 and repeat the process until a satisfactory solution is reached or the maximum number of inference steps is reached.

[0030] The beneficial effects of the present application are:

[0031] The method for preventing avionics software defects based on RAG and optimized prompt LLM combines the text inference capability of the large language model and the text data in the professional field. The Monte Carlo tree search algorithm is used to improve the relevance of text information retrieval and the accuracy of multi-step reasoning of the large language model. The avionics system belongs to a super large-scale complex system, and the defect problems that occur during the development process involve a large amount of professional field content and complex logical reasoning. This method combines effective text information and accurate multi-step reasoning to provide accurate solutions by calling the large language model. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 This is a flow chart of an avionics software defect prevention method based on RAG and optimization hint LLM according to the present invention.

[0033] Figure 2 Flowchart for the Monte Carlo Tree Search algorithm.

[0034] Figure 3 Build a graph for the inference state tree. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0036] like Figure 1 As shown in this embodiment, an avionics software defect prevention method based on RAG and optimization hints LLM comprises three main components: knowledge base construction, reasoning state tree construction, and value assessment. Knowledge base construction includes source data collection and processing (S1), data preprocessing (S2), and vector database construction (S3). Reasoning state tree construction includes search and information integration (S4) and reasoning state tree expansion (S5). Value assessment includes value assessment (S6). This embodiment first performs source data collection, which involves processing documents using optical character recognition (OCR) technology to remove non-text elements, fix errors, and standardize text formats. It also collects and extracts key text records and defect sample data from the avionics system R&D process. Subsequently, LLM is used to segment and vectorize the text, building a vector database to represent the text's location in vector space. During the search and information integration phase, relevant text blocks are retrieved from the vector database based on the problem content and decoded back into the original text for use by the LLM to generate content. Reasoning state tree expansion involves starting from an initial state node, calculating node value scores, selecting and expanding paths, and generating new reasoning state nodes using the LLM until a leaf node is reached. In the value assessment phase, the LLM is used to score the reasoning status text and decide whether to accept it as a solution or continue to expand until a complete solution is obtained or the maximum number of reasoning steps is reached. The details of each step are as follows:

[0037] Source data collection and processing S1: Collect literature and books related to the field of digital avionics technology, and ensure that the collected information is extensive and representative by systematically sorting out the knowledge system in this field; collect relevant text records in the avionics system R&D process, including various version history records, defect sample data, etc.

[0038] Among them, the relevant fields of enterprise internal text materials can be crawled by using automation technology, which includes but is not limited to email communication, internal reports, technical specifications and design documents, etc. to obtain various types of text information generated by the development process of the avionics system in actual work. For example, from the version control system such as SVN, various version history records, change content records and other records of the avionics software development process stage are crawled, which can reflect the specific details of software iteration; At the same time, from the project management and problem tracking tool such as JIRA, the problem report PR (Problem Report), change request CR (Change Request) data, review data of peer review meeting records and defect report data are crawled, which provides direct evidence for defect discovery and processing.

[0039] Data preprocessing S2: Convert the literature and books related to the field of digital avionics technology into standardized field text data; use LLM to extract change example data from various version history records, and extract problem content, reasons and solutions from defect text data. Key information such as solutions is uniformly formatted to generate defect example data. The specific method is as follows:

[0040] Optical character recognition (OCR) is performed on the collected literature and books related to the field of digital avionics technology, converting image formats to editable text, removing non-text elements such as pictures and tables in the document, and deleting non-text characters generated during the OCR process. Repair OCR errors and spelling errors to ensure the accuracy and readability of the text. Next, the text format is standardized, and the font, font size and paragraph format are unified to provide a consistent text basis for subsequent analysis.

[0041] For various version history records in the version control system, key information such as change content and change reason analysis is extracted through analysis to form change example data. At the same time, the problem report data, review data and defect report data in the project management and problem tracking tool are processed to extract key content such as problem content, reasons and solutions, and generate defect example data in a unified format. This extraction process can be realized by calling a large language model (LLM), which uses its text processing capabilities to automatically identify and extract key information in the text, thereby generating change example data and defect example data. These change example data and defect example data usually contain detailed descriptions of defects, impact scope, solutions and prevention strategies, providing rich examples for building a defect prevention model. In the entire data preprocessing step, the extraction of key content and the generation of example data need to be accurately executed to ensure the quality and consistency of the data.

[0042] Vector Database Construction S3: Utilize large language models (LLM) to segment domain text data and defect example data, the segmented text blocks are input to LLM for embedding operation, converted into vector representation that can capture the meaning and related information of the text. These vectors are then used to construct a vector database, where the position of each text is determined by its corresponding vector, representing the distribution of the text in the vector space. The specific method is as follows:

[0043] S31: Use data loader to load domain text data and defect example data, use large language model (LLM) to call cyclically for semantic analysis, set appropriate text cutting points based on content-based semantic recognition, ensure the semantic integrity of text blocks.

[0044] S32: Embedding processing is performed on the cut text blocks, and LLM is used to convert the text into meaningful vector representation. This process involves mapping the language information in the text to a point in a high-dimensional space to capture the semantic content and context information of the text.

[0045] S33: Based on these embedded vectors, a vector database is constructed, which will store the position of the text data in the vector space, providing an index for subsequent queries and analysis. During construction, the structure design and index strategy of the database need to be considered to optimize query efficiency and data processing speed.

[0046] Search and information integration S4: According to the content of the defect problem, search for several text block vectors with high relevance in the vector database, decode the searched text block vectors, convert them into text data to extract information, and use these information in subsequent LLM content generation. The specific steps are as follows:

[0047] S41: Based on the content of the input defect problem, perform search operation in the vector database, identify several text block vectors with high semantic relevance by calculating the similarity between the defect problem and the stored text block vectors in the database. This search process is completed by calculating the similarity between the input problem vector and the stored vectors in the database.

[0048] S42: The retrieved text block vectors are then converted back to text data through decoding. This conversion process involves mapping the vector representation information back to a readable text format, ensuring the integrity and readability of the information, and the extracted information will be used in the subsequent content generation process, where the large language model (LLM) can be called again to generate targeted content, using the retrieved information to improve the relevance and accuracy of the generated text.

[0049] The reasoning state tree is expanded S5, with the defect problem text as the initial reasoning state node of the reasoning state tree, and expanded along the path with high value score to the leaf node direction until a leaf node is reached. If the leaf node has been visited, the reasoning state text of the node is used, combined with the currently generated reasoning state node text or by retrieving the text embedding with high relevance in the vector database, converted into text data, and a new reasoning state node is generated using the LLM as a child node of the leaf node to expand the branch of the tree. If the leaf node has not been visited, the value of the node is scored in the value evaluation S6, and the number of visits to the nodes on the current path is updated.

[0050] The specific steps are as follows:

[0051] S51: Obtain the defect problem text and use the defect problem text as the initial reasoning state node. The defect problem text is set as the root node of the reasoning state tree, representing the starting point of the reasoning process.

[0052] S52: Calculate the value score of each node in the current reasoning state tree, and build a path from the root node to the leaf node direction. The node with high UCB value is selected as the next path by using the Upper Confidence Bound (UCB) formula, until a leaf node is reached.

[0053] S53: Check if the leaf node has been visited. If it has been visited, use the reasoning state text corresponding to the node as input, combine the currently generated reasoning state node text or retrieve several text embeddings with high relevance in the vector database according to the embedding of the text, convert the data into text data, merge these data and call the LLM to generate several reasoning state nodes as child nodes of the leaf node, thereby expanding the reasoning state tree. If the node has not been visited, enter the value evaluation step to score the value of the node to determine whether it is part of the reasoning path.

[0054] S54: After value evaluation, the first child node generated is selected as the current node, and the value evaluation step is entered again to continue expanding the reasoning state tree. Throughout the process, the Chain of Thought (CoT) strategy is adopted to combine the input content and a series of intermediate reasoning steps. The reasoning state tree is a variant of the Chain of Thought, with the root node of the tree being the known input content and the intermediate nodes being the intermediate reasoning steps expanded from the input content. According to the current reasoning state combined with external knowledge or the content of other reasoning states, the LLM is called to generate new reasoning state nodes as child nodes of these nodes, thereby expanding the reasoning decision to improve the comprehensiveness and accuracy of the solution to the input problem.

[0055] When the maximum number of reasoning steps is reached, the leaf node with the highest value score is selected, and the reasoning state text of the node is combined with the input question to call the LLM to give the final solution. If the maximum number of reasoning steps is not reached, return to the step of selecting the node with the highest value score and continue to expand the reasoning path. Throughout the process, the thought chain strategy is adopted, the input content and intermediate reasoning steps are combined, and the LLM is used to generate new reasoning state nodes. At the same time, the Monte Carlo search algorithm is used to make efficient decisions and optimize the selection and expansion of reasoning steps, ensuring that the reasoning state tree can gradually build a comprehensive reasoning path for the input question.

[0056] Value evaluation S6: Combine the reasoning state text of the current node with the input question, and call the LLM for scoring to evaluate the degree to which the text solves the input question. If the LLM evaluation result indicates that the text is sufficient to solve the problem, the text is selected as the final solution. If the text is not sufficient to solve the problem, the text is scored for value, and the value scores and visit counts of all visited nodes on the path are updated. When the number of reasoning steps reaches the maximum limit, the leaf node in the path with the highest value score is selected, combined with its reasoning state text and input question, and the LLM is called again to generate the final solution. If the maximum step limit is not reached, return to the reasoning state tree expansion S5 to continue building and evaluating the reasoning path.

[0057] The specific steps are as follows:

[0058] S61: Combine the reasoning state text corresponding to the node and the defect problem text, and call the LLM to evaluate the degree to which the node solves the defect problem. This scoring process uses the self-criticism ability of the LLM to score whether the current reasoning state text accurately and comprehensively answers the defect problem through another pre-set prompt. If the LLM evaluation considers that the reasoning state text is sufficient to solve the defect problem, this reasoning state text is selected as the final solution.

[0059] S62: If the reasoning state text is not sufficient to solve the problem, value evaluation is performed. The nodes visited on the path and their visit counts are updated. This updating process involves backpropagating the value and visit count of all nodes on the reasoning path to reflect the contribution of the current node's reasoning state text to solving the defect problem.

[0060] S63: If the maximum number of reasoning steps has been reached at this time, the leaf node is selected from the path with the highest value score. Combine the reasoning state text corresponding to the node and the defect problem to call the LLM to generate the final solution.

[0061] S64: If the preset maximum reasoning step number is not reached at this time, return to step S52, and loop this process until a satisfactory solution is reached or the maximum reasoning step number limit is reached.

[0062] In this process, the self-criticism function of LLM is used to score the accuracy and completeness of the current reasoning state text through preset prompts, thereby updating the value score of the node through backpropagation. In addition, the Upper Confidence Bound formula (UCB) is used to select the node with the highest value score on the current reasoning path to achieve a balance between exploration and exploitation. The purpose of this step is to gradually optimize the reasoning state tree through accurate value evaluation and effective path selection to generate a solution that best meets the requirements of the input problem.

[0063] As shown in Figure 2 Monte Carlo tree search algorithm can be divided into four steps: selection, expansion, simulation and backpropagation. First, an input problem is received, which is immediately set as the root node of the reasoning state tree. The flowchart guides how to explore and generate solutions by building and expanding the reasoning state tree through a series of judgment and operation steps.

[0064] In the selection step, the algorithm first checks whether the current node is a leaf node, i.e., whether it has no unexplored child nodes. If the node has been explored, it will not be explored again, but will proceed directly to the next step. If the node is a leaf node and has not been explored, the algorithm will use the Upper Confidence Bound (UCB) strategy to select the child node with the maximum UCB value of the node as the current node for the next step. UCB is a strategy that balances exploration and exploitation by calculating the potential value and exploration frequency of the node to guide the search process.

[0065] In the expansion step, after selecting the node, the text corresponding to the node is combined with the texts of other nodes or external knowledge to call the LLM (Large Language Model) to generate several new node texts. These texts represent new states that can be obtained from the current state through different reasoning paths. After generating new node texts, the algorithm takes the first newly generated node as the new current node.

[0066] In the simulation step, the value of the current node needs to be evaluated. Value evaluation is done by calling the score given by LLM, which reflects the degree to which the current node text solves the input problem. If the score shows that the current node text is sufficient to solve the problem, it will be selected as the final solution.

[0067] In the backpropagation step, if the simulated nodes are not sufficient to solve the problem, the algorithm will update the value score and visit count of the nodes on the current path, and then propagate these information backward to optimize the selection of subsequent nodes. After that, the algorithm re-enters the selection step, and this process is repeated until the selected nodes are sufficient to solve the target problem or the maximum number of reasoning steps is reached. This is an iterative and recursive search process, which gradually approaches the solution of the problem by continuously expanding and evaluating the nodes in the reasoning state tree. This process effectively combines the strategies of deep search and breadth search, using the generation ability of LLM and the selection strategy of UCB, in order to find the optimal solution in the complex reasoning space.

[0068] As shown in Figure 3 The construction of the reasoning state tree is shown. The initial state starts with the input problem, i.e. Q, which is used to build the reasoning state tree as the root node. i The text content corresponding to each reasoning state node, D i The introduced external knowledge, N i The current exploration number of this node. n i The number of children of this node. First check if the node is a leaf node, in the starting state, the node corresponding to Q is a leaf node and has not been explored, then call LLM to determine whether to introduce external knowledge for the next step of reasoning. As shown in the figure, for the relevant text D1 and D2 retrieved in the vector database, LLM can get the next step reasoning state text S1, S2, or directly generate the next step reasoning state text S3 without external knowledge. In this cycle, LLM enumerates the possible next step reasoning states according to the input problem and adds them to the tree, setting the current node as the first generated new node, i.e. S1. Perform value evaluation operation on S1 to get the score V1. Enter the backpropagation operation, i.e. update the value V i and N i of all nodes on the exploration path of the current cycle, V i is the sum of the value scores of all child nodes of this node, N i is the exploration number of this node.

[0069] In subsequent cycles, the exploration path needs to be constructed from the root node, as shown in the red line marked path. In each selection of the next node, the UCB value of all child nodes of the current node needs to be calculated. For state node S i , calculate its UCB value, the UCB formula is:

[0070]

[0071] where C is a constant. The child node with the largest UCB value is selected as the current node, and the process is repeated until the current node is a leaf node, at which point subsequent operations are performed. During the expansion of a node, the LLM is called to determine whether to incorporate external knowledge or to generate new reasoning states based on other reasoning state nodes text.

[0072] The above-described embodiments have described the technical solutions and beneficial effects of the present application in detail. It should be understood that the above-described is only a specific embodiment of the present application and is not used to limit the present application. Any modification, supplement and equivalent replacement made within the principle range of the present application should be included in the protection range of the present application.

Claims

1. A RAG and optimization hint LLM based avionics software defect prevention method, characterized in that Comprising the following steps: S1: Collecting literature and books related to the field of digital avionics technology, collecting various version history records, defect text data in the development process of avionics system; S2: Convert the literature and books related to the field of digital avionics technology into standardized field text data; use LLM to extract the change content and change reason from various version history records to form change example data, and unify the format of the problem content, reason and solution extracted from the defect text data to generate defect example data; S3: Use LLM to segment the standardized field text data and defect example data, the segmented text blocks are input to LLM for embedding operation, and are converted into vector representation capable of capturing text meaning and related information to construct a vector database, and the position of each text in the vector database is determined by its corresponding vector; S4: According to the content of the defect problem, search for several text block vectors with high relevance in the vector database, decode the searched text block vectors, and convert them into text data; S5: Take the defect problem text as the initial reasoning state node of the reasoning state tree, expand along the path with high value score to the leaf node direction until a leaf node is reached; if the leaf node has been visited, combine the reasoning state node text generated by the current leaf node or retrieve the text embedding with high relevance in the vector database through text embedding, convert the text data into text data, and use LLM to generate a new reasoning state node as a child node of the leaf node, thereby expanding the branches of the tree and continuing to perform S5; if the leaf node has not been visited, go to S6 and update the access times of the nodes on the current path; S6: Combine the reasoning state text of the current node with the input problem, call the self-criticism function of LLM for scoring, and if the LLM evaluation result shows that the text is sufficient to solve the problem, the text is selected as the final solution; If the text is not sufficient to solve the problem, score the text and update the value score and access times of all access nodes on the path in this round; when the reasoning step reaches the preset maximum limit, select the leaf node in the path with the highest value score, combine its reasoning state text and input problem, and call LLM again to generate the final solution; if the maximum step limit has not been reached, return to S5 to continue building and evaluating the reasoning path.

2. The RAG and optimization hint LLM based avionics software defect prevention method of claim 1, wherein In S1, the automatic technology is used to crawl the text materials related to the field in the enterprise to obtain various version history records and defect example data generated in the development process of avionics system in actual work.

3. The RAG and optimization hint LLM based avionics software defect prevention method of claim 1, wherein In S3, the collected literature and books related to the field of digital avionics technology are subjected to optical character recognition, the image format is converted into editable text, non-text elements in the document are removed, OCR errors and spelling errors are repaired, and the text format is standardized; For the collected various version history records, the change content and change reason are extracted and analyzed to form change example data; The problem report data, review data and defect report data are processed to extract problem content, causes and solutions, and defect sample data is generated in a unified format; The change sample data and the defect sample data contain detailed description of defects, impact range, solution measures and prevention strategies.

4. The RAG and optimization hint LLM based avionics software defect prevention method of claim 1, wherein S5 is specifically as follows: S51: Obtain the defect problem text and take the defect problem text as an initial reasoning state node; S52: Calculate the value score of each node in the current reasoning state tree using the upper confidence bound formula, select the node with the value score as the next path selection, and build the path from the root node to the leaf node; S53: Check whether the leaf node has been visited, if it has been visited, take the reasoning state text corresponding to the node as input, combine the reasoning state node text generated currently or according to the text embedding, retrieve a number of text embeddings with high relevance in the vector database, convert the data into text data, merge the data and call the LLM to generate a number of reasoning state nodes as child nodes of the leaf node, thereby expanding the reasoning state tree; If the node has not been visited, go to S6 to evaluate the value of the node to determine whether it is part of the reasoning path; S54: After value evaluation, select the first child node generated as the current node, and go to S5 again to continue expanding the reasoning state tree; S6 is specifically as follows: S61: Combine the reasoning state text corresponding to the node and the defect problem text, and call the LLM to evaluate the completeness of the node in solving the defect problem; S62: If the reasoning state text is not enough to solve the problem, evaluate the value, and update the nodes visited in this round of path and their visit times; S63: If the maximum number of reasoning steps has been reached at this time, select the leaf node from the path with the highest value score, combine the reasoning state text corresponding to the node and the defect problem, and call the LLM to generate the final solution; if the maximum number of reasoning steps has not been reached at this time, return to step S52 and loop this process until a satisfactory solution is reached or the maximum number of reasoning steps is reached.

Citation Information

Patent Citations

  • Monte Carlo tree searching method and device based on science question answering task

    CN117933392A

  • Intelligent query reasoning method and device based on knowledge graph and large language model collaboration

    CN117743531A

  • Construction industry safety question and answer method and equipment based on large language model technology

    CN118332092A