Avionics software defect prompting method based on LLM and knowledge graph

Abstract

The invention discloses an avionics software defect prompting method based on LLM and a knowledge graph. The avionics software defect prompting method comprises the steps of firstly vectorizing related documents and books to form an aviation field corpus, then formulating entity classification and relation rules, performing named entity recognition and relation classification through LLM, extracting an entity-relation-entity triple, and forming an avionics system knowledge graph; processing avionics software defect data, constructing a defect sample knowledge base, further generating entities based on structured data, and establishing a relationship between the entities to form a defect knowledge graph; establishing a relationship between entities in the avionics system knowledge graph and the defect knowledge graph, and combining the relationship into an avionics software defect knowledge graph; and a reasoning path is constructed in the avionics software defect knowledge graph, and a conclusion is obtained after the reasoning path is screened out and evaluated. The knowledge graph is utilized to enhance understanding of the LLM on historical project data of avionics software, and compared with a common LLM, the text reasoning ability on an avionics system of a civil aircraft is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of civil aircraft software engineering, and relates to avionics systems, software quality, large language models (LLMs), knowledge graphs, and prompting engineering, and specifically to an avionics software defect prompting method based on LLMs and knowledge graphs. Background Art

[0002] In today's aviation industry, embedded onboard software applications are the key drivers of aircraft innovation and performance improvement. The functionality and performance level of modern aircraft are directly subject to their complex and sophisticated software avionics systems. Civil avionics systems need to meet extremely stringent safety standards and certification requirements. Due to the criticality of avionics software in the entire system, any software failure may lead to serious consequences. Therefore, strict safety specifications must be followed during the design, development and testing of the software to ensure that the aircraft can operate reliably in various situations.

[0003] Due to the criticality of avionics system software, avionics software requires more stringent software quality management than other software. According to the CMMI Capability Maturity Model, software quality management involves at least four means, namely: CMMI Level 2 software quality assurance, CMMI Level 3 peer review, CMMI Level 4 quantitative quality management, and CMMI Level 5 defect prevention. The current software quality management of avionics software covers software quality assurance, peer review and other methods, but there are few mature solutions for defect prevention.

[0004] Large language models (LLMs), such as GPT-4 or similar models, refer to language models that contain hundreds of billions or more parameters and are trained on large amounts of text. They have made significant progress in the field of natural language processing (NLP), mainly in the form of being able to generate text, answer questions, and conduct conversations. The text recognition, data processing, and reasoning interaction capabilities of LLMs provide a possible solution for defect prevention measures in software quality management of avionics systems.

[0005] However, there are still some problems with LLM, mainly manifested in:

[0006] 1. The hallucination problem: When generating factual or knowledge texts, LLMs may generate content that contradicts existing sources or lacks supporting evidence.

[0007] 2. Lack of interpretability: As a black box model, LLM implicitly represents knowledge in its parameters, but this knowledge is difficult to interpret and verify.

[0008] 3. Lack of domain-specific knowledge: Due to the lack of domain-specific knowledge or new training data, LLMs trained on general corpora cannot generalize well to specific domains. Considering the problem of catastrophic forgetting, injecting additional expertise may come at the cost of losing previously learned information.

[0009] 4. Lack of timeliness: Since LLM is trained based on past knowledge and requires a lot of training time, it is unable to continuously obtain the latest relevant knowledge and cannot provide accurate solutions to avionics software defects based on outdated knowledge.

[0010] 5. Lack of deep logical reasoning ability: When provided with question-answering examples, LLM showed good performance on some basic logical reasoning tasks. However, LLM performed poorly in tasks that require understanding and using supporting evidence to draw conclusions, and was unable to provide accurate answers to questions that require long logical chains and multi-hop knowledge reasoning.

[0011] Based on these characteristics, knowledge graph technology relies on the construction of professional knowledge and the structuring of data, so it has a high degree of accuracy and can provide more accurate and professional knowledge support. In addition, the knowledge graph can also be dynamically updated and expanded to ensure the timeliness of knowledge. By interactively exploring the relevant entities and relationships on the knowledge graph and reasoning based on the retrieved knowledge, LLM can have better knowledge traceability and knowledge correctability. For example, the patent document with domestic publication number CN117891930A proposes a knowledge graph-enhanced book question-and-answer method to improve the ability of LLM to answer professional field questions. Considering that avionics software defect prompts require knowledge in a specific field and must be scalable and provide defect solutions accurately, the combination of LLM and knowledge graph can make up for the shortcomings of LLM to a certain extent and improve its ability to solve such problems. Summary of the invention

[0012] The purpose of the invention is to provide an avionics software defect prompting method based on LLM and knowledge graph, which makes up for the deficiency of existing LLM in solving professional field problems by introducing knowledge graph.

[0013] The object of the present invention is achieved through the following technical solutions:

[0014] An avionics software defect prompting method based on LLM and knowledge graph includes the following steps:

[0015] S1. Convert the literature books related to avionics systems into structured vector representations and store them in a vector database to form an aviation field corpus;

[0016] S2. Use LLM to process the aviation domain corpus and extract relations and entities for knowledge graph construction;

[0017] S3, acquiring avionics software defect data and forming defect sample data after structured processing, converting the defect sample data into vector form and storing it in a vector database;

[0018] S4. Extract entities and relationships based on the defect sample structured data and build a defect knowledge graph;

[0019] S5. Based on the entities of the avionics system knowledge graph, the corresponding entities in the defect knowledge graph are identified through LLM, and relationships are established to merge into the avionics software defect knowledge graph;

[0020] S6, taking the information in the avionics software development process as input text, combining it with the avionics software defect knowledge graph to build a reasoning path, and cyclically calling LLM to screen out a suitable reasoning path;

[0021] S7. Call LLM to evaluate whether the reasoning path can solve the input problem and generate the final solution.

[0022] Further, the specific steps of S1 are as follows:

[0023] S11, firstly, performing optical character recognition on documents and books related to the avionics system to extract text data;

[0024] S12, cyclically calling LLM to identify the cutting points of the text data to perform document cutting;

[0025] S13, converting the cut documents into vector form and storing them in a vector database.

[0026] Further, the specific steps of S2 are as follows:

[0027] S21. Formulate entity classification rules and classification relationship rules between entities applicable to the field of avionics systems;

[0028] S22, calling LLM to perform named entity recognition and relationship classification on the text content of the aviation field corpus to extract entity-relationship-entity triples;

[0029] S23, normalize the entities extracted from the text to ensure that the same real-world object has only one unique identifier in the knowledge graph; for relationships, merge similar relationships and retain the main relationships;

[0030] S24. The structured knowledge data after entity normalization and relationship merging is stored in the graph database to form the domain knowledge graph of the avionics system.

[0031] Further, the specific steps of S3 are as follows:

[0032] S31. Obtain various defect data of avionics software;

[0033] S32, classifying the defect text data, and performing corresponding structural processing on the text data according to the classification result to form defect sample data;

[0034] S33, cyclically calling LLM to identify appropriate cutting points to cut the defect sample data;

[0035] S34, converting the defect sample data after cutting into a vector form and storing it in a vector database.

[0036] Further, the specific steps of S4 are as follows:

[0037] S41. Based on the structured data of the defect sample, generate entities according to data classification;

[0038] S42. Set relationship names between entities of different types, establish relationships for generated entities, and import these relationships into a graph database.

[0039] Further, the specific steps of S5 are as follows:

[0040] S51, setting a prompt template, the prompt template is used to guide the LLM to recognize entities in the input text, and then calling the LLM to recognize entities in the avionics system knowledge graph appearing in the text, and output the recognition result;

[0041] S52. Establish the relationship between the entities in the defect sample knowledge graph and the entities in the avionics system knowledge graph involved in the text content, merge the two knowledge graphs, and form an avionics software defect knowledge graph.

[0042] Further, the specific steps of S52 are as follows:

[0043] First, the entities in the defect knowledge graph are matched with the entities in the avionics system knowledge graph according to the names and attributes of the entities;

[0044] Subsequently, based on the matching results, the relationships between entities are established, and duplicates are removed and ambiguities are eliminated;

[0045] Finally, the defect sample knowledge graph is merged with the entities and their relationships in the avionics system knowledge graph to form the avionics software defect knowledge graph.

[0046] Further, the specific steps of S6 are as follows:

[0047] S61, taking the information in the avionics software development process as input text, calling LLM to automatically extract entities in the avionics system knowledge graph appearing in the input text, and forming a set of reasoning paths;

[0048] S62, using a beam search algorithm, searching for all linked relationships for the tail entity in each reasoning path set, and obtaining an entity-relationship set as a candidate path;

[0049] S63, converting the candidate paths into text, calling LLM to evaluate each candidate path in turn, scoring them according to their contribution to the problem, and selecting a new set of reasoning paths based on the ranking;

[0050] S64, according to the tail relations of the selected reasoning path set, all entities linked by these relations are taken as part of the candidate paths, and the reasoning path set is further expanded;

[0051] S65. Convert the expanded candidate paths into text, call LLM to evaluate each candidate path in turn, score them according to their contribution to the problem, and select a new set of reasoning paths based on the ranking.

[0052] Further, the specific steps of S7 are as follows:

[0053] S71, loop the process of S6, and call LLM after each loop to evaluate whether the current reasoning path is sufficient to generate the answer. If the LLM evaluation believes that the current path can generate the answer, then generate the answer according to the reasoning path and end the reasoning process; if the LLM evaluation believes that the current path is still insufficient to generate the answer, continue to search for new candidate paths in a loop until the set maximum search depth is reached;

[0054] S72. After reaching the maximum search depth, if no conclusion can be drawn, LLM calls texts in the knowledge base that are semantically similar to the problem text and uses these texts to generate solutions.

[0055] The beneficial effects of the present invention are:

[0056] The present invention completes the task of constructing a knowledge graph through a large language model, uses the knowledge graph to enhance LLM's understanding of the learned knowledge, and enhances the data- and knowledge-driven two-way reasoning capabilities by combining LLM and knowledge graph. At the same time, the domain language model trained with aviation domain knowledge significantly improves the text reasoning capabilities of avionics systems compared to the popular LLM, ensuring the relevance and accuracy of the given defect fault prompts and solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flowchart of an avionics software defect prompting method based on LLM and knowledge graph.

[0058] Figure 2 To prompt the LLM to build a reasoning path flowchart based on the knowledge graph during model construction. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0060] like Figure 1 As shown, the avionics software defect prompting method based on LLM and knowledge graph shown in this embodiment includes three aspects of processing, namely, the generation of avionics software defect knowledge base, the construction of avionics software defect knowledge graph, and the construction of prompt model. In the generation of avionics software defect knowledge base, it includes generating avionics domain professional knowledge base S1 and generating defect sample knowledge base S3. In the construction of avionics software defect knowledge graph, it includes constructing avionics system knowledge graph S2, constructing defect sample knowledge graph S4 and knowledge graph fusion S5. In the construction of prompt model, it includes reasoning path construction S6 and chain call LLMS7. First, the relevant document books are converted into plain text, and the documents are cut. Then, the cut documents are vectorized and stored in the vector database to form an aviation field corpus. Then, referring to the existing ontology consensus and combining expert knowledge, entity classification and relationship rules are formulated, named entity recognition and relationship classification are performed through LLM, entity-relationship-entity triples are extracted, and normalization is performed to form an avionics system knowledge graph and store it in the graph database. In addition, by acquiring avionics software defect data and performing structured processing, a defect sample knowledge base is constructed, and entities are further generated based on structured data, and relationships between entities are established to form a defect knowledge graph. In the knowledge graph fusion stage, LLM is called through the prompt template to identify avionics system entities in defect samples, establish relationships, and merge to form an avionics software defect knowledge graph. In the prompt model construction, the prompt is initialized and the entities in the R&D process are automatically extracted. The beam search algorithm and LLM evaluation are used to form and optimize the reasoning path set. Finally, in the result generation optimization step, the reasoning path evaluation is executed cyclically until the maximum search depth is reached or the answer is generated. If no conclusion can be drawn, the knowledge base text with a similarity close to the question text is called to generate the answer. The entire process uses natural language processing, knowledge graph construction, vector database storage, and beam search algorithm technologies to achieve LLM based on the avionics system knowledge graph. Deeper reasoning and accurate solutions to defect problems are given. The specific description of each step is as follows:

[0061] Aviation field corpus construction S1: Convert the literature books related to avionics systems into structured vector representations and store them in a vector database to form an aviation field corpus. The specific steps are as follows:

[0062] OCR and data processing S11. First, optical character recognition (OCR) is performed on the relevant documents and books of the avionics system. This process includes image preprocessing, which improves the image quality by removing noise, enhancing contrast and other operations; then feature extraction is performed, using image processing technology to extract character features in the document image; then the OCR algorithm is used to recognize the extracted character features and convert the characters in the image into text; then text post-processing is performed, and error correction, character segmentation optimization and other operations are performed to improve recognition accuracy and text quality; finally, data cleaning is performed, including deleting non-text content such as pictures and tables in the document, and converting the document into plain text format to prepare for subsequent processing and analysis.

[0063] Document loading and cutting S12. First, use the data loader to divide the extracted text data into individual documents and load them into the system. This process involves data format parsing. According to the format of the document, the corresponding parser is used to load the document content into the system, and data preprocessing is performed. The preprocessing includes text normalization, segmentation, sentence segmentation and other operations. Subsequently, the LLM is called cyclically to identify the cutting points, and the loaded documents are input into the LLM to identify the points suitable for cutting. These cutting points can be based on paragraphs, sentences or other semantic units to ensure that the cut documents have a certain semantic coherence. Finally, the document is cut and the document is divided into appropriate parts or paragraphs according to the cutting points identified by the LLM. This processing method provides a clear data structure for subsequent data processing and analysis.

[0064] Document vectorization and storage S13, convert the cut documents into vector form and store them in a vector database. This process usually uses text vectorization technology to convert the text information in the document into a numerical vector representation so that the computer can process and analyze it. A vector database is a database system specifically used to store and retrieve vector data. It can efficiently process large-scale vector data and provide fast retrieval functions. Storing document vectors in a vector database can facilitate operations such as similarity search and cluster analysis.

[0065] Avionics system knowledge graph construction S2, use LLM to process the aviation field corpus, extract relations and entities for knowledge graph construction. The specific steps are as follows:

[0066] Establish entity and relationship classification rules S21. Formulate entity classification rules and classification relationship rules between entities applicable to the field of avionics systems. This step can be implemented by drawing on the ontology consensus of existing knowledge bases in related fields and combining expert knowledge and field data. Existing knowledge bases in related fields may include existing ontology models, industry standards, professional terminology sets, etc. By studying existing knowledge bases in related fields, explore the ontology consensus that has been established therein. These ontology consensuses include domain concepts, entities, and the relationships between them. Based on these ontology consensuses, formulate entity classification rules and classification relationship rules between entities. These rules may include the definition, attributes, hierarchy, association relationships, etc. of entities, as well as classification relationships, association relationships, etc. between entities. Then, based on these rules, adjustments and expansions can be made based on expert knowledge and field data to meet the specific needs of avionics systems.

[0067] Named entity recognition and relationship classification S22. Call LLM to perform named entity recognition and relationship classification on the text content of the aviation field corpus to extract entity-relationship-entity triplets. Based on its learning of large-scale text data, LLM can effectively identify and extract entities in the text and the relationships between entities. First, we use LLM to perform named entity recognition (NER) to identify specific entities mentioned in the text, such as the primary flight display PFD. Subsequently, we use LLM for relationship classification, analyze the relationships between entities in the text, and extract entity-relationship-entity triplets. These triplets describe the relationships between entities in the text and provide basic data for the subsequent construction of the knowledge graph.

[0068] Entity normalization and relationship merging S23. Normalize the entities extracted from the text to ensure that the same real-world object has only one unique identifier in the knowledge graph; for relationships, merge similar relationships and retain the main relationships. This process includes comparing and deduplicating the entities extracted from the text, evaluating whether the entities appearing in different texts refer to the same real-world objects, and eliminating possible duplicates and ambiguities to ensure that each entity has a clear definition and semantics. By linking each entity to a unique identifier in the knowledge graph, the consistency and accuracy of the knowledge graph is ensured. The relationships are then processed, retaining the main relationships to simplify the structure of the knowledge graph and improve its readability and manageability. This means analyzing and classifying the relationships extracted from the text, merging similar relationships into one main relationship, and linking it to the corresponding entity.

[0069] Knowledge graph storage and construction S24. After entity normalization and relationship merging, structured knowledge data is stored in a graph database to form a domain knowledge graph of avionics systems. A graph database is a database system specifically used to store and process graph data structures. It can efficiently store and query large-scale graph data and support complex graph algorithms and query operations. In a graph database, each entity and relationship is represented as a node and edge in the graph, and the knowledge graph is a graph structure composed of these nodes and edges. By storing it in a graph database, it is convenient to query, analyze and visualize the knowledge graph and discover hidden patterns and associations.

[0070] Defect sample knowledge base construction S3, obtain avionics software defect data and structure it to form defect sample data, cut the defect sample data and convert it into vector form and store it in the vector database. The specific steps are as follows:

[0071] Avionics software defect data acquisition S31, obtain various defect data of avionics software, including but not limited to crawling PR (Problem Report) and CR (Change Request) data in JIRA tools, review data of peer review meeting records, defect report data, enterprise internal defect sample data, etc. These data sources cover various defect situations that may occur during software development and maintenance, including defect phenomena and solutions, and provide a sufficient data foundation for building a defect sample knowledge base.

[0072] Text data structuring S32, classify the defect text data into categories such as defect phenomenon, cause of the problem, module involved in the problem, solution to the problem, responsible person, time, and R&D process where the problem occurred. Such classification can make the data more readable and operable. Then, according to the classification results, the text data is structured accordingly, such as a table or database record, to form defect sample data for subsequent analysis and retrieval.

[0073] Defect sample data loading and cutting S33, the acquired defect sample data is loaded into the system through the data loader. This process involves converting the text data into a data format that can be processed by the computer, and loading and storing it. Subsequently, the LLM is called cyclically, and its semantic context recognition ability is used to identify appropriate cutting points to cut the defect sample data. These cutting points are determined based on semantic units or text paragraphs to ensure that the cut documents have a certain degree of coherence and comprehensibility.

[0074] Cut document vectorization and storage S34, convert the cut defect sample data into vector form and store it in a vector database. This process usually includes text vectorization processing, converting the text information in the document into a numerical vector representation for computer processing and analysis. Subsequently, the document vector is stored in a vector database, which is a database system specifically used for storing and retrieving vector data. It can efficiently process large-scale vector data and provide fast retrieval functions. Through this step, the vectorized representation and storage of the defect sample data are realized to facilitate subsequent analysis and application.

[0075] Defect knowledge graph construction S4: Based on the defect sample structured data, extract entities and relationships to construct the defect knowledge graph. The specific steps are as follows:

[0076] Generate entity S41, based on the structured data of the defect sample, generate entities according to data classification. This process involves preprocessing the data to ensure the accuracy and completeness of the data. Subsequently, natural language processing technology is used to identify key entities from the structured data. For example, the defect phenomenon can be used as an entity, the cause of the problem can be used as another entity, and so on, and these entities are classified into predefined categories according to their characteristics (such as defect phenomenon, cause of the problem, module involved in the problem, solution to the problem, etc.). Next, attributes are assigned to each entity, such as defect severity, priority, status, etc., to enrich the information content of the entity. Through this step, the defect sample data is converted into entity form.

[0077] Establish relationships between entities and import them into the graph database S42, set the relationship names between different types of entities, establish relationships between the generated entities, and import these relationships into the graph database. This means naming the relationships between different types of entities, for example, the relationship between the defect phenomenon and the cause of the problem can be named "cause", and the relationship between the defect phenomenon and the solution to the problem can be named "solve". After establishing the relationship, select a suitable graph database to store and manage the knowledge graph, such as Neo4j, etc., considering its ability to handle complex relationships and support for high-concurrency queries. Then, import the entities and their relationships into the graph database in the form of a graph, create nodes to represent entities, edges to represent relationships, and perform index optimization to improve query efficiency. In addition, in order to maintain the accuracy and availability of the knowledge graph, it is necessary to update and maintain the graph regularly. This includes adding new defect samples, correcting incorrect entity relationships, and optimizing the performance of the graph database.

[0078] Knowledge graph fusion S5: Based on the entities of the avionics system knowledge graph, the corresponding entities in the defect knowledge graph are identified through LLM, and relationships are established to merge into the avionics software defect knowledge graph. The specific steps are as follows:

[0079] Set prompt template and identify entity S51, set prompt template, the prompt template is used to guide the large language model (LLM) to identify entities in the input text, for example, different templates can be set to identify entities related to avionics system, such as flight control system, navigation system, communication system, etc. Then call LLM to identify entities in the avionics system knowledge graph appearing in the text, and output the recognition result. This step utilizes the ability of LLM in natural language understanding to ensure that relevant entity information can be accurately extracted from the text.

[0080] Knowledge graph fusion S52: Establish the relationship between the entities in the defect sample knowledge graph and the entities in the avionics system knowledge graph involved in the text content, delete duplicates and eliminate ambiguities, merge the two knowledge graphs, and form an avionics software defect knowledge graph.

[0081] First, the entities in the defect knowledge graph are matched with the entities in the avionics system knowledge graph based on the entity names, attributes and other features. Then, based on the matching results, the relationship between the entities is established, and duplicates are deleted and ambiguities are eliminated. In this process, the identified entities need to be carefully compared and analyzed to ensure that the relationship between the entities is accurate. Deleting duplicates and eliminating ambiguities is to ensure the neatness and consistency of the knowledge graph and avoid confusion caused by repeated or unclear information.

[0082] Finally, the defect sample knowledge graph is merged with the entities and their relationships in the avionics system knowledge graph to form an avionics software defect knowledge graph. This merging process involves complex data processing and integration technologies to ensure that the two knowledge graphs can be effectively combined to form a unified avionics software defect knowledge graph. The merged avionics software knowledge graph not only contains the specific information of the defect sample, but also incorporates the professional knowledge of the avionics system, thus providing richer and deeper data support for subsequent defect analysis, fault diagnosis and knowledge sharing.

[0083] Prompt model construction S6, take the information in the avionics software development process as input text, combine the avionics software defect knowledge graph to build a reasoning path, and call LLM cyclically to filter out the appropriate reasoning path. The specific steps are as follows:

[0084] Prompt initialization and reasoning path generation S61, prompt initialization is performed, that is, information in the avionics software development process, such as requirement text, code, design problem report, etc. is used as input text, and LLM (large language model) is called to automatically extract entities in the avionics system knowledge graph that appear in the input text to form a set of reasoning paths. The purpose of this step is to use the LLM model to intelligently identify entities related to the avionics system from the input text and form a reasoning path, that is, the possible associations between the entities mentioned in the text.

[0085] Candidate path generation S62, using a beam search algorithm, searches for all linked relationships for the tail entity in each reasoning path set, and obtains an entity-relationship set as a candidate path. Specifically, starting from the tail entity, search forward along the path to find the entities and relationships connected to it, and continuously expand the path until the stopping condition is met, and obtain a series of entity-relationship sets as candidate paths, where each set represents a possible reasoning path. The beam search algorithm is a heuristic search algorithm used to find the optimal path or solution in a graph or network. The use of the beam search algorithm helps to efficiently find potential and meaningful paths in a large relationship space.

[0086] Candidate path evaluation and selection S63, convert the candidate path into text, call LLM in turn to evaluate each candidate path, score it according to its contribution to the problem, and select a new set of reasoning paths based on the ranking. This step involves converting the candidate paths into text form so that the LLM model can evaluate it. Then, call the LLM model in turn to evaluate each candidate path, analyze its contribution to the problem, and give a corresponding score. Based on the scoring results, select the top-ranked paths as the new set of reasoning paths. Through this process, paths that contribute to problem solving can be screened out from the candidate paths, providing guidance for subsequent path search and reasoning.

[0087] Expanding candidate paths S64, based on the tail relations of the selected reasoning path set, taking all entities linked by these relations as part of the candidate paths, and further expanding the reasoning path set. This step involves adding all linked entities as new candidate paths based on the selected reasoning path set, starting from the tail relations of each path. Such expansion can increase the diversity of paths, thereby more comprehensively exploring possible paths for problem solving.

[0088] Update the reasoning path S65, convert the expanded candidate path into text, call LLM to evaluate each candidate path in turn, score them according to their contribution to the problem, and select a new set of reasoning paths based on the ranking. This step is similar to S63, but the expanded candidate paths are evaluated and selected. The new candidate paths are evaluated by calling the LLM model again, and the top-ranked paths are selected based on the scoring results to obtain the final set of reasoning paths. This final set represents the reasoning paths that are most likely to help solve the problem for a specific defect sample after being evaluated by LLM. This dual evaluation and selection process can improve the quality and credibility of the path.

[0089] Generate result optimization S7, call LLM to evaluate whether the reasoning path can solve the input problem, and generate the final solution. The specific steps are as follows:

[0090] In the loop reasoning process S71 and the loop S6 process, LLM is called after each loop to evaluate whether the current reasoning path is sufficient to generate the answer. If the LLM evaluation believes that the current path can generate the answer, the answer is generated according to the reasoning path, and the reasoning process ends. If the LLM evaluation believes that the current path is still insufficient to generate the answer, the new candidate path is continued to be searched in a loop until the set maximum search depth is reached. Through such a loop reasoning process, different reasoning paths are constantly tried until enough information is found to generate the answer or the maximum search depth is reached.

[0091] Solution generation based on similarity S72, after reaching the maximum search depth, if no conclusion can be drawn, that is, no satisfactory reasoning path is found to generate the answer, then the system will adopt an alternative strategy. Specifically, the system will call the text in the knowledge base that is semantically similar to the question text according to the LLM, and use these texts to generate solutions. This step involves using the LLM model to evaluate the similarity between the question text and the knowledge base text, and generating answers based on the degree of similarity. By comparing the semantics and context of the question text and the knowledge base text, the knowledge base text that is most similar to the question is selected, and the answer is generated based on it. Although no definitive conclusion can be drawn, answer generation based on similarity can provide a possible solution and provide a reference for problem solving. Through the above method, the generation result optimization step can ensure that when the reasoning path is not sufficient to directly generate an answer, the system can still provide an answer based on the knowledge base text, which reflects the flexibility and robustness of the system when facing complex problems, and can adopt appropriate strategies to generate solutions in different situations.

[0092] like Figure 2 As shown in the figure, the specific steps of the process of calling LLM to build a reasoning path based on the knowledge graph are as follows:

[0093] S1. Initialization based on knowledge graph reasoning. For a given question, the initial entities of the local knowledge graph are identified by calling LLM. First, LLM is prompted to automatically extract the relevant entities in question and obtain the first N entity sets E = {e1, e 2, …e n}.

[0094] S2, Reasoning path exploration. This part is divided into relationship exploration and entity exploration. The entity set is used as the set of initial reasoning paths, that is, each current reasoning path has only one entity e i .

[0095] S21. Relationship exploration. For the kth round of exploration, the set of all relationships associated with the tail entities of each reasoning path is: At this point, the candidate reasoning path P is a set of several entity-relationships. The candidate reasoning path P is converted into a text representation, and the LLM is called to score the contribution of the text to the input question, sort and retain the top N candidate reasoning paths as the current reasoning path.

[0096] S22, entity exploration. Similar to relationship exploration, all associated entities are obtained, connected to the tail relationship of each reasoning path, converted into text representation and called LLM for contribution score sorting, and the top N candidate reasoning paths are retained as the current reasoning paths.

[0097] S23, reasoning path evaluation. After completing this iteration, call LLM to evaluate whether the current set of reasoning paths is sufficient to answer the input question. If a positive result is obtained, this is used as a prompt to call LLM to generate a solution to the problem. If the number of iterations reaches the set maximum reasoning search depth, the problem is converted into a vector representation, a similarity comparison is performed in the vector database, and the retrieved text is returned as input, and LLM is called to generate a solution to the problem. If the set maximum reasoning search depth is not reached, continue the S2 process.

[0098] It is understandable that those skilled in the art can make equivalent substitutions or changes based on the technical solution and inventive concept of the present invention, and all these changes or substitutions should fall within the protection scope of the claims attached to the present invention.

Claims

1. A method for avionics software defect prompting based on LLM and knowledge graph, characterized in that The following steps are involved: S1. Convert the literature books related to avionics systems into structured vector representations and store them in a vector database to form an aviation field corpus; S2. Use LLM to process the aviation domain corpus and extract relations and entities for knowledge graph construction; S3, acquiring avionics software defect data and forming defect sample data after structured processing, converting the defect sample data into vector form and storing it in a vector database; S4. Extract entities and relationships based on the defect sample structured data and build a defect knowledge graph; S5. Based on the entities of the avionics system knowledge graph, the corresponding entities in the defect knowledge graph are identified through LLM, and relationships are established to merge into the avionics software defect knowledge graph; S6, taking the information in the avionics software development process as input text, combining it with the avionics software defect knowledge graph to build a reasoning path, and cyclically calling LLM to screen out a suitable reasoning path; S7. Call LLM to evaluate whether the reasoning path can solve the input problem and generate the final solution.

2. The avionics software defect prompting method based on LLM and knowledge graph according to claim 1 is characterized in that The specific steps of S1 are as follows: S11, firstly, performing optical character recognition on documents and books related to the avionics system to extract text data; S12, cyclically calling LLM to identify the cutting points of the text data to perform document cutting; S13, converting the cut documents into vector form and storing them in a vector database.

3. The avionics software defect prompting method based on LLM and knowledge graph according to claim 1, characterized in that S2 The specific steps are as follows: S21. Formulate entity classification rules and classification relationship rules between entities applicable to the field of avionics systems; S22, calling LLM to perform named entity recognition and relationship classification on the text content of the aviation field corpus to extract entity-relationship-entity triples; S23, normalize the entities extracted from the text to ensure that the same real-world object has only one unique identifier in the knowledge graph; for relationships, merge similar relationships and retain the main relationships; S24. The structured knowledge data after entity normalization and relationship merging is stored in the graph database to form the domain knowledge graph of the avionics system.

4. The avionics software defect prompting method based on LLM and knowledge graph according to claim 1 is characterized in that The specific steps of S3 are as follows: S31. Obtain various defect data of avionics software; S32, classifying the defect text data, and performing corresponding structural processing on the text data according to the classification result to form defect sample data; S33, cyclically calling LLM to identify appropriate cutting points to cut the defect sample data; S34, converting the defect sample data after cutting into a vector form and storing it in a vector database.

5. The avionics software defect prompting method based on LLM and knowledge graph according to claim 1 is characterized in that The specific steps of S4 are as follows: S41. Based on the structured data of the defect sample, generate entities according to data classification; S42. Set relationship names between entities of different types, establish relationships for generated entities, and import these relationships into a graph database.

6. The avionics software defect prompting method based on LLM and knowledge graph according to claim 1 is characterized in that The specific steps of S5 are as follows: S51, setting a prompt template, the prompt template is used to guide the LLM to recognize entities in the input text, and then calling the LLM to recognize entities in the avionics system knowledge graph appearing in the text, and output the recognition result; S52. Establish the relationship between the entities in the defect sample knowledge graph and the entities in the avionics system knowledge graph involved in the text content, merge the two knowledge graphs, and form an avionics software defect knowledge graph.

7. The avionics software defect prompting method based on LLM and knowledge graph according to claim 6 is characterized in that The specific steps of S52 are as follows: First, the entities in the defect knowledge graph are matched with the entities in the avionics system knowledge graph according to the names and attributes of the entities; Subsequently, based on the matching results, the relationships between entities are established, and duplicates are removed and ambiguities are eliminated; Finally, the defect sample knowledge graph is merged with the entities and their relationships in the avionics system knowledge graph to form the avionics software defect knowledge graph.

8. The avionics software defect prompting method based on LLM and knowledge graph according to claim 1 is characterized in that The specific steps of S6 are as follows: S61, taking the information in the avionics software development process as input text, calling LLM to automatically extract entities in the avionics system knowledge graph appearing in the input text, and forming a set of reasoning paths; S62, using a beam search algorithm, searching for all linked relationships for the tail entity in each reasoning path set, and obtaining an entity-relationship set as a candidate path; S63, converting the candidate paths into text, calling LLM to evaluate each candidate path in turn, scoring them according to their contribution to the problem, and selecting a new set of reasoning paths based on the ranking; S64, according to the tail relations of the selected reasoning path set, all entities linked by these relations are taken as part of the candidate paths, and the reasoning path set is further expanded; S65. Convert the expanded candidate paths into text, call LLM to evaluate each candidate path in turn, score them according to their contribution to the problem, and select a new set of reasoning paths based on the ranking.

9. The avionics software defect prompting method based on LLM and knowledge graph according to claim 1 is characterized in that S7 specific steps are as follows: S71, loop the process of S6, and call LLM after each loop to evaluate whether the current reasoning path is sufficient to generate the answer. If the LLM evaluation believes that the current path can generate the answer, then generate the answer according to the reasoning path and end the reasoning process; if the LLM evaluation believes that the current path is still insufficient to generate the answer, continue to search for new candidate paths in a loop until the set maximum search depth is reached; S72. After reaching the maximum search depth, if no conclusion can be drawn, LLM calls texts in the knowledge base that are semantically similar to the problem text and uses these texts to generate solutions.

Images