Method and device for determining quality of airworthiness text based on airworthiness certification platform

Abstract

This application provides a method and apparatus for determining the quality of airworthiness documents based on an airworthiness certification platform, relating to the fields of artificial intelligence and data processing. The application includes: acquiring the airworthiness documents to be inspected for an aircraft; determining the query type of the airworthiness documents to be inspected based on a preset certification task inspection system, and determining the target retrieval path corresponding to the query type; inspecting the airworthiness documents to be inspected according to the target retrieval path, and generating inspection result data; responding to user feedback operations, receiving inspection feedback data on the inspection result data; if the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, then optimizing the certification task inspection system at least once, and generating inspection result data with an inspection accuracy greater than or equal to the preset threshold based on the optimized certification task inspection system. Therefore, this solves the problems existing in the prior art and greatly improves the efficiency and accuracy of airworthiness certification tasks.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and data processing, and more specifically, to a method and apparatus for determining the quality of airworthiness texts based on an airworthiness certification platform. Background Technology

[0002] Currently, airworthiness refers to the ability of a civil aircraft to meet minimum safety standards and fly smoothly within a predetermined operating environment and usage restrictions. The essence of airworthiness certification is to ensure, through scientific and standardized verification methods, that an aircraft meets the stringent safety standards set by the International Civil Aviation Organization (ICAO) and national airworthiness authorities throughout its entire lifecycle, from design and manufacturing to operation. Therefore, the airworthiness certification process generates various types of aircraft airworthiness verification reports and airworthiness certification plan documents to verify whether an aircraft meets airworthiness requirements. These documents are characterized by a high degree of interdisciplinary collaboration, complex compliance with aviation regulations and industry standards, and the need for full lifecycle management and cross-stage collaboration. Therefore, how to quickly and effectively verify the airworthiness of aircraft has become a key focus.

[0003] In existing technologies, the traditional airworthiness certification process for document-based airworthiness certification tasks mostly relies on human experience for evaluation. Common methods for intelligent evaluation include rule-based expert systems, traditional machine learning evaluation systems, and natural language processing (NLP) technologies.

[0004] However, in existing technologies, traditional airworthiness certification relies excessively on the work experience and subjective judgment of certification personnel. Furthermore, compliance with airworthiness provisions and industry standards requires manual calibration and verification by certification personnel, resulting in lengthy certification cycles and low efficiency. Intelligent assessment methods generally suffer from problems such as limited assessment dimensions, insufficient dynamic adaptability, and lagging knowledge updates, leading to lower certification accuracy. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for determining the quality of airworthiness texts based on an airworthiness certification platform, which solves the above-mentioned problems existing in the prior art and greatly improves the efficiency and accuracy of airworthiness certification tasks.

[0006] Firstly, a method for determining the quality of airworthiness texts based on an airworthiness certification platform is provided, which may include: Obtain the airworthiness documents of the aircraft pending inspection; Based on a pre-defined approval task inspection system, the query type of the airworthiness document to be inspected is determined, and the target retrieval path corresponding to the query type is determined; the airworthiness document to be inspected is processed according to the target retrieval path to generate inspection result data; wherein, the pre-defined approval task inspection system is generated based on the basic data that has been deeply refined, and the approval task inspection system includes multiple pre-defined dimensions of inspection standards; the target retrieval path is a hybrid retrieval enhanced path, or a single retrieval path; In response to user feedback, the system receives inspection feedback data on the inspection result data. If the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, the system performs at least one optimization process on the review task inspection system and generates inspection result data with an inspection accuracy greater than or equal to the preset threshold based on the optimized review task inspection system.

[0007] Secondly, an airworthiness text quality determination device based on an airworthiness certification platform is provided, the device may include: The acquisition module is used to acquire the airworthiness documents to be inspected for the aircraft. The inspection module is used to determine the query type of the airworthiness document to be inspected based on a preset approval task inspection system, and to determine the target retrieval path corresponding to the query type; to inspect the airworthiness document to be inspected according to the target retrieval path, and to generate inspection result data; wherein, the preset approval task inspection system is generated based on the basic data that has been deeply refined, and the approval task inspection system includes multiple preset dimensions of inspection standards; the target retrieval path is a hybrid retrieval enhanced path, or a single retrieval path. The receiving module is used to receive inspection feedback data for the inspection result data in response to user feedback operations; The generation module is used to perform at least one optimization process on the review task inspection system if the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, and generate inspection result data with the inspection accuracy greater than or equal to the preset threshold based on the optimized review task inspection system.

[0008] Thirdly, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0009] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of any of the methods described in the first aspect above.

[0010] This application provides a method and apparatus for determining the quality of airworthiness documents based on an airworthiness certification platform. It constructs an inspection method for airworthiness certification tasks based on the platform, specifically an inspection method for airworthiness documents to be inspected. Through a pre-established certification task inspection system and technologies such as graph-based retrieval enhancement generation and retrieval enhancement generation (RAG) within this system, multi-dimensional inspections of the airworthiness documents to be inspected are performed. This method can quickly adapt to the constantly updated airworthiness regulations system, integrate multi-source data, and form a dynamic evaluation network covering the entire aircraft lifecycle. It breaks through the multi-level and multi-granularity airworthiness certification task evaluation, achieving intelligent evaluation of airworthiness certification tasks and continuously optimizing the evaluation process. It effectively solves the core bottlenecks of traditional evaluation methods, such as reliance on human experience, slow response, and difficulty in traceability. This provides key support for achieving high-precision, interpretable, and self-iterative intelligent certification, greatly improving the efficiency and accuracy of airworthiness certification tasks. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a method for determining the quality of airworthiness texts based on an airworthiness certification platform, provided in an embodiment of this application; Figure 2 A flowchart illustrating a method for determining the quality of airworthiness texts based on an airworthiness certification platform, provided in an embodiment of this application; Figure 3 An overall architecture diagram of an airworthiness text quality determination method based on an airworthiness certification platform provided in this application embodiment; Figure 4 A flowchart of an evaluation layer provided in an embodiment of this application; Figure 5 A technical block diagram of an optimization layer provided in an embodiment of this application; Figure 6 An operational architecture diagram of an airworthiness certification platform provided in this application embodiment; Figure 7 A schematic diagram of an airworthiness text quality determination device based on an airworthiness certification platform provided in this application embodiment; Figure 8 A schematic diagram of the structure of another airworthiness text quality determination device based on an airworthiness certification platform provided in this application embodiment; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art. The words "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are only used to distinguish different components. The words "comprising" or "including," etc., mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but do not exclude other elements or objects. The words "connected," "coupled," or "connected," etc., are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0014] The airworthiness text quality determination method based on an airworthiness certification platform provided in this application can be applied to electronic devices or other terminals. To ensure the accuracy of the certification, other terminals can be user equipment (UE) such as mobile phones, smartphones, laptops, digital broadcast receivers, personal digital assistants (PDAs), and tablet computers (PADs), handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem, mobile stations (MS), mobile terminals, etc. Other terminals and the server can be directly or indirectly connected through wired or wireless communication methods, which is not limited herein.

[0015] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0016] Figure 1 This is a flowchart illustrating a method for determining the quality of airworthiness texts based on an airworthiness certification platform, as provided in an embodiment of this application. Figure 1 As shown, the method may include: Step S210: Obtain the airworthiness documents to be inspected for the aircraft.

[0017] For example, the airworthiness documents to be inspected for the aircraft can be obtained. These documents include various data and parameters of the aircraft to be inspected, and are not limited thereto.

[0018] Step S220: Based on the preset approval task inspection system, determine the query type of the airworthiness document to be inspected, and determine the target retrieval path corresponding to the query type; perform inspection processing on the airworthiness document to be inspected according to the target retrieval path, and generate inspection result data; wherein, the preset approval task inspection system is generated based on the basic data that has been deeply refined, and the approval task inspection system includes multiple preset dimensions of inspection standards; the target retrieval path is a hybrid retrieval enhanced path, or a single retrieval path.

[0019] For example, the pre-defined certification task inspection system is generated based on the already refined basic data, and the certification task inspection system includes inspection standards across multiple pre-defined dimensions. The basic data includes airworthiness regulations, industry standards, and airworthiness case studies, etc.

[0020] In this step, based on the pre-defined review and approval task inspection system, the query type of the airworthiness document to be inspected is determined, and the target retrieval path corresponding to the query type is also determined. The target retrieval path can be a hybrid retrieval enhancement path or a single retrieval path. The airworthiness document to be inspected undergoes multi-dimensional inspection processing according to the target retrieval path, generating inspection result data. For example, if the target retrieval path is a hybrid retrieval enhancement path, then each part of the airworthiness document to be inspected is input into a graph-based retrieval enhancement generation database and a retrieval enhancement generation database for hybrid retrieval to generate inspection result data; or, if the target retrieval path is a single retrieval path, then each part of the airworthiness document to be inspected is input into a graph-based retrieval enhancement generation database or a retrieval enhancement generation database for hybrid retrieval to generate inspection result data.

[0021] Step S230: In response to the user's feedback operation, receive inspection feedback data for the inspection result data; if the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, then perform at least one optimization process on the approval task inspection system, and generate inspection result data with an inspection accuracy greater than or equal to the preset threshold based on the optimized approval task inspection system.

[0022] For example, based on a preset tracing mechanism, a reasoning process for generating inspection result data is generated, and this process is visualized. The reasoning process includes multiple reasoning steps, each corresponding to a jumpable original evidence text. Then, based on user feedback on each step and feedback on the inspection result data, inspection feedback data is received. It is determined whether the inspection accuracy represented by the feedback data is less than a preset threshold. If the inspection accuracy is less than the preset threshold, the review task inspection system is optimized based on the feedback data, and new inspection result data is generated based on the optimized system. If the inspection accuracy represented by the new result data is still less than the preset threshold, the review task inspection system is optimized again based on the feedback data, until inspection result data with an inspection accuracy greater than or equal to the preset threshold is generated.

[0023] In the embodiments of this application, the airworthiness documents to be inspected for an aircraft are obtained. Based on a preset certification task inspection system, the query type of the airworthiness documents to be inspected is determined, and the target retrieval path corresponding to the query type is determined. The airworthiness documents to be inspected are inspected according to the target retrieval path, and inspection result data is generated. The preset certification task inspection system is generated based on the basic data that has been deeply refined, and the certification task inspection system includes multiple preset dimensions of inspection standards. The target retrieval path is a hybrid retrieval enhanced path or a single retrieval path. In response to user feedback, inspection feedback data on the inspection result data is received. If the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, the certification task inspection system is optimized at least once, and inspection result data with an inspection accuracy greater than or equal to the preset threshold is generated based on the optimized certification task inspection system. This solution possesses a complete system, constructing a comprehensive theoretical framework encompassing data acquisition, knowledge refinement, model enhancement, rigorous verification, multi-dimensional evaluation, and continuous evolution, starting from the data layer, evaluation layer, and decision-making layer. Through a pre-established certification task inspection system and technologies such as graph-based retrieval enhancement generation and retrieval enhancement generation (RAG) within this system, it performs multi-dimensional inspections of airworthiness documents to be inspected. It can quickly adapt to the constantly updated airworthiness regulations, integrate multi-source data, and form a dynamic evaluation network covering the entire aircraft lifecycle. This breaks through the limitations of multi-level and multi-granularity airworthiness certification task evaluation, achieving intelligent evaluation of airworthiness certification tasks and continuously optimizing the evaluation process. It effectively solves the core bottlenecks of traditional evaluation methods, such as reliance on human experience, slow response times, and difficulty in traceability. This provides crucial support for achieving high-precision, interpretable, and self-iterative intelligent certification, greatly improving the efficiency and accuracy of airworthiness certification tasks.

[0024] Figure 2This is a flowchart illustrating a method for determining the quality of airworthiness texts based on an airworthiness certification platform, as provided in an embodiment of this application. Figure 2 As shown, the method may include: Step S301: Obtain basic data; whereby, basic data includes airworthiness regulations, industry standards, and airworthiness cases.

[0025] For example, Figure 3 An overall architecture diagram of an airworthiness text quality determination method based on an airworthiness certification platform, provided in this application embodiment, is shown below. Figure 3 As shown in the diagram, the overall architecture comprises a data layer, an evaluation layer, and an optimization layer. The data layer is used for data enrichment and expansion; the evaluation layer is used for airworthiness certification program evaluation and knowledge management; and the optimization layer is used for model optimization and intelligent services. Basic data includes airworthiness regulations, industry standards, and airworthiness cases. The data layer acquires and constructs high-quality basic data to provide reliable data input for subsequent task evaluation and model optimization. Specifically, it involves data collection and data augmentation based on airworthiness regulations, technical standards, and airworthiness cases, combined with manual review, to achieve efficient data collection, cleaning, and labeling, improving the efficiency and quality of data collection and augmentation. This is the foundational layer of the overall architecture; subsequent task evaluation and model optimization depend on the integrity and quality of the data at this stage.

[0026] Specifically, the core objective of the data collection phase is to build a comprehensive and high-quality airworthiness certification knowledge base. The effectiveness of this knowledge base directly determines the accuracy and reliability of the inspection results; therefore, the data collection phase is a crucial foundational step in the entire mission inspection. The data collected in this phase primarily comes from three sources: First, airworthiness regulations, including advisory circulars (ACs) and administrative procedures (APs) that provide operational details and interpretations of these regulations. Second, industry standards, such as software considerations in airborne systems and equipment certification (RTCA DO-178C) and airborne electronic hardware design assurance guidelines (DO-254). Third, airworthiness case studies, such as publicly available technical reports and incident case data.

[0027] Optionally, in terms of data collection, this application constructs a crawler system based on a distributed system architecture, employing Pdfplumber, Pytesseract, and Python-docx to handle the core processing of PDF documents, scanned documents, and DOCX documents, respectively. A reliable queue service is used for task scheduling and communication, and core modules such as crawling and page parsing are deployed in multiple locations to optimize resource utilization and improve overall processing speed. This crawler system integrates dynamic rendering, recursive crawling, and IP rotation mechanisms to crawl web page data in a legal and compliant manner. A tiered collection strategy is adopted based on the characteristics of different websites. For crawling airworthiness regulations, since the official websites are assumed to be static web pages with fixed content that directly return HTML or PDF links, a combination of a lightweight HTTP request library and an HTML parser (such as BeautifulSoup) is used to quickly traverse the page and identify and download all PDF links. At the same time, the scope of airworthiness regulations crawling is limited, prioritizing data related to aircraft airworthiness regulations, official documents issued by the Airworthiness Certification Department, and documents whose names contain keywords such as "airworthiness" and "certification." For airworthiness case studies published by pre-defined authoritative bodies, traditional crawlers cannot obtain complete content because these bodies rely on websites that dynamically load content using JavaScript. This application introduces headless browser technology, which launches a real browser (of any type) without a graphical interface in the background to simulate real user behavior (clicking, scrolling, typing, etc.) to ensure that all content loaded by JavaScript is fully rendered before data extraction. This application uses the Selenium WebDriver library to control the browser through an Application Programming Interface (API) and handle page waiting mechanisms. Simultaneously, the crawler system has a built-in automatic retry mechanism. When data flow is interrupted due to network fluctuations or request failures, it can automatically re-initiate requests, greatly improving the success rate of data collection. Through this layered architecture, this platform can seamlessly crawl from static websites and scale to handle the most complex dynamic content, ensuring the comprehensiveness and continuity of data. Furthermore, considering the timeliness of airworthiness regulations and case studies, the crawler system also includes a website change monitoring and early warning mechanism. ChangeTower monitors changes to the content of specific URLs in real time. Once a new regulation is issued or an existing regulation is revised, the system will automatically trigger a data update to ensure that the assessment is always based on the latest and most authoritative data.

[0028] Optionally, documents such as industry standards, such as Software Considerations in Airborne Systems and Equipment Certification (RTCA DO-178C), Airborne Electronic Hardware Design Assurance Guidelines (DO-254), Civil Aircraft and Systems Development Guidelines (SAE ARP4754B), and Guidelines and Methods for the Safety Assessment Process of Civil Aviation Systems and Equipment (SAE ARP4761A), are internal data that have been purchased and translated in advance in this invention, as they need to be purchased from the official website.

[0029] Step S302: Based on the preset filtering conditions, perform filtering and format conversion on the basic data to obtain basic data in the preset format.

[0030] For example, this step mainly involves identifying the structure of the basic data and, based on preset filtering conditions, filtering and format conversion of the basic data to obtain basic data in a preset format. Specifically, if the basic data is determined to be heterogeneous documents, such as unstructured or semi-structured documents, then knowledge refinement is performed on the collected raw basic data. The knowledge refinement process mainly involves deeply refining the collected basic data, transforming unstructured or semi-structured documents into structured knowledge that can be understood and utilized by machines. This process not only includes meticulous filtering but also encompasses format conversion, multi-dimensional annotation, and an intelligent feedback loop of human-machine collaboration, aiming to build a high-purity, highly relevant, and continuously optimized knowledge base.

[0031] Optionally, for the collected heterogeneous documents, a preliminary screening of metadata and keywords is first performed based on a machine learning-based intelligent filtering mechanism to ensure the accuracy and applicability of the collected knowledge. After the initial rough screening of metadata and keywords, a second, refined screening of the document content is performed using an open-source, pre-defined RoBERTa base classification model (i.e., the RoBERTa-base classifier neural network pre-trained model). During this second screening, the documents can be accurately classified into predefined airworthiness topic categories such as "airworthiness regulations" and "airworthiness cases," thereby effectively filtering out irrelevant general documents that do not belong to the predefined airworthiness topic categories, thus improving the accuracy of the data collected during the data collection phase.

[0032] The filtered documents are then fed into a multi-functional document processing pipeline for format standardization, converting the heterogeneous documents from the second round of filtering into heterogeneous documents in a preset format. First, all the heterogeneous documents from the second round of filtering, including PDFs and HTML, are converted to Markdown format using the PyMuPDF4LLM tool, as this format effectively preserves the original document structure (such as headings, tables, lists, etc.). Subsequently, to adapt to downstream applications, especially in preparation for knowledge graphs and RAG databases, the Markdown documents are further converted to a preset format, such as CSV. CSV text content is closer to question-answer pairs, which is more conducive to subsequent applications of Retrieval Augmentation (RAG).

[0033] Step S303: In response to the manual annotation operation, the basic data in the preset format is annotated to generate annotated data; and based on the annotated data and the basic data in the preset format, a graph-based retrieval enhancement generation and a retrieval enhancement generation database are generated.

[0034] For example, in the semantic understanding and annotation stage of this step, a mechanism of human-agent collaboration is integrated. This mechanism aims to combine human judgment with machine automation to ensure the accuracy of annotation and continuously optimize model performance. The annotation method varies for different types of data.

[0035] Optional airworthiness regulation knowledge graph annotation: For legal texts such as airworthiness regulations, the aim is to extract triples of entities and relationships to prepare for the construction of a knowledge graph. First, a pre-trained zero-shot named entity recognition (NER) model, GLiNER, is used to perform preliminary analysis of the documents, automatically identifying key entities (such as "CCAR-21" and "aircraft engine") and their relationships (such as "regulations" and "applicable to"). For example, a complete triple instance can be extracted from the "Regulations for Type Certification of Aircraft Engines" (CCAR-33): (Subject: CCAR-33, Relation: Regulations, Object: Airworthiness standards for aircraft engine type certificates). The triples are stored in JSON Lines (JSONL) format for subsequent creation of the airworthiness regulation knowledge graph. Subsequently, these automatically extracted results are pushed to a manual verification queue. To maximize the efficiency of this process, an active learning strategy is adopted. The core idea of ​​the active learning strategy is to enable the machine to automatically identify data points that require manual review, prioritize submitting these data points for manual annotation, and generate labeled data based on the user's manual annotation operations. There are two main criteria for determining whether data needs manual annotation: ① Low model confidence. Low confidence is determined through uncertainty sampling. This is achieved through sampling, meaning the model has low confidence in the prediction results. Specifically, a pre-defined GLiNER model is used to generate the probability distribution of the prediction results. When the probability distribution of the prediction results is close to uniform, it indicates that the model is highly uncertain about the prediction. This step uses an entropy-based metric to quantify this uncertainty, with the following formula:

[0036] Here, H(x) is the probability distribution, and P(y|x) is the model's prediction probability that the input x belongs to category y. The higher the value of H(x), the more unstable the model's prediction of the result. This data needs to be manually reviewed and labeled.

[0037] ② Content Anomalies. Content anomalies refer to documents whose content differs significantly from the existing training set distribution, constituting "outlier" data. Specifically, the Isolation Forest model from the Python Outlier Detection (PyOD) library is used to automatically detect and identify these anomalous documents, which are then sent to a human review queue. Humans use an interactive interface to confirm, correct, or supplement the entity types and relationships, ultimately forming high-quality "entity-relationship-entity" triples. This manually verified data is then fed back into the model as new training samples, forming a continuously iterative closed loop that constantly improves the model's extraction accuracy.

[0038] Optional RAG database annotation for industry standards and airworthiness cases: For publicly available technical reports, industry standards, and event case data, metadata is automatically extracted and annotated using Natural Language Processing (NLP) technology. A pre-defined GLiNER model is used to automatically identify and extract key information from the documents. For example, from a technical report, the document type, issuing organization, document number, and key topics are automatically identified. For industry standards such as RTCA DO-178C, DO-254, ARP4754, and ARP4761, structured metadata annotation is required before RAG vectorization. The pre-defined open-source model QWQ-32B is used to automatically extract key fields from the documents according to a predefined metadata pattern. The annotation style for key fields can include { "DocumentID": "RTCA DO-178C", "DocumentType": "Industry Standard", "SubjectArea": ​​"Airborne Software", "Purpose": "Provide software airworthiness certification guidelines"}. In the automated annotation process, situations requiring manual review mainly include: when the model's confidence in the extracted metadata is lower than a preset threshold, when the document content involves complex technical details or multiple contexts, and when new document formats or entity types that the model has never seen before appear. In these cases, manual review and supplementation of the metadata automatically extracted by the machine is required. For example, adding more in-depth tags to the cases or clarifying ambiguous events. This ensures that each data block in the Retrieval Augmentation Generative Database (RAG database) is accompanied by rich and accurate metadata, greatly improving the relevance and accuracy of retrieval.

[0039] Optionally, based on the labeled data and heterogeneous documents in a preset format obtained above, a graph-based retrieval enhancement generation (GraphRAG) and a retrieval enhancement generation database (RAG database) are generated. Specifically, since the collected airworthiness data can be divided into two categories: airworthiness regulations with highly structured features and closely related chapters and clauses, and industry standards and airworthiness cases with relatively low structure and a greater focus on technical details and event descriptions, this application chooses to combine GraphRAG with the traditional RAG architecture to specifically handle different types of airworthiness data. For airworthiness regulations, GraphRAG is used to clearly depict the complex relationships between different regulatory clauses, aircraft components, and operating procedures through nodes and edges in the graph, generating GraphRAG to support multi-hop reasoning and in-depth contextual understanding. For industry standards and airworthiness cases, the traditional RAG architecture is used, utilizing vector embedding technology for efficient semantic similarity search to generate the RAG database. This method can provide fast and relevant retrieval results when processing massive, scattered, and ambiguous text data. This hybrid architecture allows for the construction of a comprehensive airworthiness knowledge system that provides both deep insights and rapid response.

[0040] Specifically, the steps for constructing a knowledge graph are as follows: ① Definition of entity and relation triples The knowledge graph construction uses the airworthiness regulation triples automatically extracted by the GLiNER model and manually verified and labeled in the "knowledge refinement" process as the basic input data, and adopts the Microsoft open-source project GraphRAG as the core framework to realize subsequent graph calls. First, the entities and relationships extracted from the airworthiness regulations will be described as follows.

[0041] The structured representation of a knowledge graph relies on the definition of core entities and their relationships within the domain. In the airworthiness regulations knowledge graph, through the analysis of existing regulations, the following core entities and relationship types are defined and represented in the form of triples. The core entities and relationships are shown in Table 1: Table 1

[0042] ②Detailed rules for knowledge extraction and graph construction The open-source GraphRAG framework was chosen as the core framework for graph generation and subsequent graph retrieval. First, "entity-relationship-entity" triples were imported into Neo4j in JSON Lines (JSONL) format to generate the graph database. After identifying entities and relations in the text, GraphRAG uses an embedding model to represent the text and entities as vectors. In this application, the BGE-M3 model can be used, without limitation. These vectors not only include the semantic information of the text but also provide efficient representation for subsequent retrieval and querying. After the knowledge graph is constructed, the framework uses a graph machine learning algorithm (Leiden algorithm) to perform community detection on the constructed knowledge graph, dividing it into multiple highly cohesive "communities," each representing a specific topic or logical module. The Leiden algorithm, through its unique refinement operation, ensures that all communities have good connectivity, which is crucial for maintaining rigorous logical relationships between rules and regulations. The Leiden algorithm works like a "smart grouping" process: it analyzes the connection density between these nodes and automatically clusters closely related nodes together. For example, community A might contain all regulations, equipment components, and technical parameters related to "rotorcraft," forming a community about "rotorcraft type certification." Community B, on the other hand, might contain all content related to transport aircraft. In this way, the originally complex knowledge graph is structured into meaningful, focusable subgraphs.

[0043] Following community detection, GraphRAG uses a Large Language Model (LLM) to generate bottom-up summaries for each community. This process employs an efficient MapReduce model, aiming to pre-generate a "global" context capable of answering high-level, summarizing questions, as follows: Map Phase: LLM independently and in parallel analyzes all regulations, entities, and relationships within each community, generating a highly summarized text summary. For example, for the "Rotorwing Aircraft" community A mentioned above, LLM analyzes all regulations, weight limits, seat belt requirements, and other information related to rotorwing aircraft, and generates a concise summary: "This community summarizes the requirements for type certification of transport-class rotorwing aircraft, including detailed regulations on maximum weight, passenger seat belts, and propeller pitch limits."

[0044] Reduce Phase: This phase is the second step of the Map-Reduce model. Its core purpose is to integrate local summaries from multiple communities into a "global summary" that represents the overall knowledge. It takes all relevant community summaries as input and feeds them into the LLM (Local Level Management Model). The LLM integrates and summarizes these partial summaries from different communities to form a single, coherent "global summary" that represents the entire knowledge base's topic. This global summary is then stored and indexed for use as high-level retrieval content in subsequent user queries. In selecting the large-scale model for community summary generation, this application chooses the same large-scale model, Deepseek-R1-32B, as the final answer generation model, without limiting the model type.

[0045] Optionally, the RAG database construction mainly consists of three parts: a text indexer, a text retrieval unit, and an answer generator. The text indexer employs a hybrid indexing strategy, integrating vector indexing, semantic indexing, and a rule engine. This hybrid strategy transforms collected data into high-dimensional vectors, which the retrieval unit can then use for vector retrieval. The Faiss database is chosen as the vector database, as it converts unstructured text content into high-dimensional vectors and stores them there. The text retrieval unit is responsible for retrieving relevant contextual information from a standard database based on user-inputted documents. The text retrieval unit can choose the BGE-M3 embedding model, with no restrictions on the model type. The answer generator, i.e., a Large Language Model (LLM), generates the final filtering results based on the retrieved contextual information and the user-inputted documents. Currently, the Deepseek-R1 32B model is used, with no restrictions on the model type.

[0046] It should be noted that the traditional RAG architecture primarily deals with technical reports such as industry standards and airworthiness cases collected during the data collection phase. Since such content lacks standardized writing guidelines compared to airworthiness regulations, the RAG architecture is used to convert these reports into a vector-based rule database before uploading them to the larger model as external references. This application, however, adds operations such as refining and filtering the basic data, and constructing the RAG database based on a text indexer, text retrieval machine, and answer generator, ensuring the accuracy of the constructed RAG database.

[0047] Therefore, this application constructs a data collection method that encompasses data collection, knowledge refinement, knowledge graph, and RAG construction. By employing collaboration between human and intelligent systems, it achieves efficient acquisition, cleaning, annotation, and enhancement of airworthiness certification data. Furthermore, by constructing a knowledge graph and RAG architecture, the large language model can enhance its reasoning ability to complete complex tasks. The external knowledge base (i.e., the RAG database) also makes the text generation of the large language model more reliable, thereby ensuring the scientific nature and accuracy of the certification.

[0048] Step S304: Obtain the airworthiness documents to be inspected for the aircraft.

[0049] For example, this step is described in step S210, and will not be repeated here.

[0050] Step S305: Based on the document decomposition strategy of regular expression matching in the review task inspection system, extract the chapter content of the airworthiness document to be inspected and generate text blocks corresponding to each chapter content.

[0051] For example, Figure 4 A flowchart of an evaluation layer provided for an embodiment of this application, such as Figure 4 As shown, the main content of the evaluation layer is to evaluate the uploaded airworthiness documents to be inspected using a hybrid retrieval enhancement technology that combines a large language model with knowledge graphs and the traditional RAG architecture. The documents to be evaluated are the airworthiness documents to be inspected. After the airworthiness documents to be inspected are uploaded, they undergo document structure parsing based on chapter intent and multi-query transformation.

[0052] Specifically, since the airworthiness documents to be inspected are typically long documents with a clear hierarchical structure and a large word count, if the documents are not processed during database retrieval, GraphRAG and traditional RAG architectures may treat long documents as a whole, resulting in overly coarse inspection results. Therefore, during the evaluation phase, a document decomposition strategy based on regular expression matching is adopted for the uploaded documents to be inspected, defining the document's chapters as the smallest atomic unit for evaluation. First, the PyMuPDF4LLM tool is used to convert the uploaded documents into Markdown format text that preserves the logical structure and chapter hierarchy. Second, regular expression matching is used to extract the content of each chapter in the document into independent text blocks. These text blocks are forcibly bound to their chapter metadata, including the chapter title, key entities in the chapter description (e.g., "structural fatigue test," "flight control software"), and the complete content under the chapter. This chapter metadata is the basis for subsequent query enhancement and refined retrieval.

[0053] Step S306: Based on the semantic query rewriting strategy in the preset large language model, convert multiple text blocks into query sets respectively.

[0054] In one example, step S306 includes: for each text block, performing abstract intent and key entity extraction processing on the text block based on the semantic query rewriting strategy in the preset large language model to generate a query definition; wherein, the text block includes chapter content and metadata corresponding to the chapter content; based on concept matching, determining at least one subquery corresponding to the query definition; and fusing the extracted key entities with at least one subquery to generate a query set.

[0055] For example, in order to transform the extracted chapter content into queries that facilitate data retrieval at the data layer, this architecture adopts a semantic query rewriting strategy in LLM, leveraging the powerful generalization and reasoning capabilities of inference models such as Deepseek-R1-32B to transform the complete text content of the chapter into a set of queries that cover its compliance intent.

[0056] Specifically, the core LLM (Deepseek-R1-32B) first receives the complete chapter content and metadata, and performs abstract intent and key entity extraction to generate query interpretations. The LLM analyzes the complete chapter content and metadata such as chapter titles, identifies the core technologies and security themes of the chapter, and generates one or more semantically rich, multi-faceted query interpretations based on the chapter length. These interpretations must be broad and unconstrained, aiming to expand the semantic space of the retrieval and dynamically discover applicable data for the data layer through concept matching. For example, suppose the chapter to be evaluated is: [Fly-by-wire system software verification activities], and the complete text mentions "unit testing, integration testing, code coverage reaching 95%" and "independent software quality assurance (SQA) activities." The LLM will generate one or more subqueries based on the key concepts of this text content, targeting airworthiness regulations, industry standards, and airworthiness cases. The specific generation of the subqueries is shown below: Airworthiness regulations: What are the mandatory requirements for onboard software verification and safety objectives for transport category aircraft? Industry Standards: What are the normative guidelines regarding independence and coverage in unit testing and integration testing of aviation software? Airworthiness Cases: Identify ASN / NTSB airworthiness cases involving insufficient software testing or code coverage deficiencies on transport aircraft.

[0057] After generating subqueries, to further improve retrieval accuracy, LLM enhances the query by binding key entities extracted from the chapter content before sending it to the text search engine. These key entities include core technology names / verbs, etc. In other words, the subquery content is merged with the core technology names / verbs extracted from the chapter content to form the final search string. For example, the airworthiness regulations subquery might be "What are the mandatory requirements for onboard software verification and safety objectives on transport aircraft?", with key entities extracted from the chapter content being "unit testing," "integration testing," "95% code coverage," and "independent SQA." The final enhanced query would then be "What are the mandatory requirements for 95% code coverage, unit testing, and independent SQA activities on transport aircraft?".

[0058] Therefore, semantic rewriting and query enhancement can ensure a high degree of focus in retrieval, forcing the retrieval machine to search the entire knowledge base for content related to each chapter of the document to be checked.

[0059] Step S307: Determine the query type of the query set.

[0060] For example, such as Figure 4 As shown, it also includes dynamic query routing, which determines the query type of the query set.

[0061] Step S308: Based on the query type, the query set is assigned to the target retrieval path for asynchronous hybrid retrieval processing to generate inspection result data; wherein, the target retrieval path is a hybrid retrieval enhancement path or a single retrieval path; the hybrid retrieval enhancement path is a preset graph-based retrieval enhancement generation and a preset retrieval enhancement generation database; the single retrieval path is a graph-based retrieval enhancement generation or a retrieval enhancement generation database.

[0062] For example, such as Figure 4 As shown, this stage is responsible for intelligently allocating the optimized query set to the target retrieval path based on the query type for efficient asynchronous hybrid retrieval processing, generating inspection result data. The target retrieval path can be either a hybrid retrieval enhancement path or a single retrieval path. A hybrid retrieval enhancement path consists of a pre-defined graph-based retrieval enhancement generation (GraphRAG) and a pre-defined retrieval enhancement generation database (a traditional RAG database) in the data layer; a single retrieval path consists of either graph-based retrieval enhancement generation (GraphRAG) or a retrieval enhancement generation database (a traditional RAG database). LLM will classify the subqueries enhanced by the above queries into the following two types: Type A: Subqueries generated based on airworthiness regulations. These queries involve regulatory clauses, hierarchical relationships, scope of application requirements, or require multi-hop reasoning. They will subsequently be routed to the GraphRAG architecture for data retrieval. Type B: Subqueries generated based on industry standards and airworthiness case studies. These queries focus on technical details, industry best practices, specific verification processes, or historical cases. They will subsequently be routed to the traditional RAG architecture for data retrieval.

[0063] For the same chapter to be evaluated, LLM can simultaneously initiate asynchronous parallel searches to the GraphRAG and RAG databases to ensure that comprehensive, multi-source evidence is obtained within the same inference cycle.

[0064] Optional GraphRAG retrieval paths: multi-hop inference and Cypher templated queries are as follows: For type A queries, the LLM query intent is further translated into parameterized Cypher query templates. These templates are used for efficient, structured graph traversal within the Neo4j graph database. GraphRAG's evidence extraction goes beyond matching individual regulatory clause nodes; it leverages the structural advantages of the graph to extract crucial contextual information. This includes performing multi-hop relationship tracing to identify inherent logical connections between regulatory clauses (e.g., tracing parent-child hierarchies of clauses or advisory notices they reference through relationships such as inclusion, application, and interpretation).

[0065] In addition, GraphRAG returns a global community summary of the community to which the node belongs. These summaries are pre-generated using a MapReduce model and represent high-level context for a specific topic, such as "Type Certification of Transport-Class Rotarywing Aircraft". Providing this contextual information along with specific regulatory clauses to the LLM ensures that the LLM can understand the position and significance of isolated clauses within the overall regulatory framework when making inferences, thereby preventing misjudgments due to lack of context.

[0066] Optional, traditional retrieval path for the RAG database: high-dimensional vector similarity search as follows: For type B queries, the traditional RAG architecture vectorizes the query using an embedding model (BGE-M3) and performs efficient semantic similarity search in the Faiss vector database. The search targets primarily focus on unstructured or semi-structured technical documents, including industry standards (such as RTCA DO-178C, SAE ARP4754B) and airworthiness case reports. To further improve the accuracy of the initial search, metadata filtering is integrated into the search process. Only document blocks that match the metadata of the section to be evaluated (such as "System Components: Landing Gear") are included in the similarity comparison. This strategy significantly reduces noise in the search space, ensuring that the initial Top-N results returned by the searcher are highly relevant.

[0067] Step S309: Based on the preset cross-encoding reordering model, score the context blocks of the inspection result data to generate scores; determine the inspection result data whose scores meet the preset scoring conditions as the filtered inspection result data.

[0068] For example, the inspection result data includes structured information returned by graph-based retrieval enhancement and vector blocks returned by the retrieval enhancement database. Figure 4As shown, after obtaining heterogeneous evidence sets (i.e., returned structured information and vector blocks) from GraphRAG and traditional RAG databases in the asynchronous hybrid retrieval stage, the key task of this step is to refine and filter the retrieval results, including re-ranking models, evidence diversity checks, and reverse imputation strategies, so as to ensure that the context received by LLM has the highest accuracy and relevance.

[0069] Specifically, the inspection results consist of two parts: structured information (nodes, relations, community summaries) returned by GraphRAG and vector blocks (text, metadata) returned by the RAG database. Since the initial scores (path relevance, cosine similarity) generated by graph traversal and vector search are not directly comparable, a pre-defined cross-encoder reranking model (BGE-Reranker-Large model) is chosen to handle the scoring task, thus avoiding the use of complex fusion formulas such as Reverse Rank Fusion (RRF). The BGE-Reranker-Large model, by simultaneously inputting the query and context blocks into the model for joint processing, can capture deeper semantic interactions and relationships between input sequences. This mechanism enables it to generate a more accurate and reliable final relevance score than the initial vector similarity score. In this application, the BGE-Reranker-Large model is configured with a scoring mechanism of 1 point as the maximum score. It scores the structured information and the context blocks of the structured information to generate the first score; and scores the vector blocks and the context blocks of the vector blocks to generate the second score. The reordering model acts as a critical quality gate, ensuring that only the semantically most accurate and relevant Top-K evidence is passed to the LLM, thereby significantly reducing the risk of the LLM generating "illusions" or misinterpreting key terms.

[0070] Based on the final relevance score output by the re-ranking model, the K GraphRAG evidence blocks with scores higher than 0.7 and the K traditional RAG evidence blocks are used as key reference data. This strategy requires the large model to simultaneously utilize top-ranked content from both mandatory airworthiness regulations and industry standards / cases (providing technical details and risk guidance) to ensure comprehensive decision-making. A lightweight evidence diversity check is also implemented when selecting the Top-K evidence blocks to avoid over-concentration of search results on similar paragraphs within the same document. The final set of evidence blocks can be reverse-packed before being fed into the LLM. This strategy places the highest-scoring, most relevant evidence blocks at the end of the input context, leveraging the LLM's attention-focusing effect on end-of-line information to ensure that the Deepseek-R1-32B model assigns the highest weight to the most reliable evidence when making its final judgment.

[0071] Therefore, the airworthiness certification task evaluation technology, which covers multiple levels and granularities such as safety, reliability, and controllability, integrates multiple types of knowledge bases by constructing a knowledge graph and RAG architecture. At the same time, it introduces a re-ranking model to comprehensively evaluate the airworthiness documents to be inspected in the airworthiness certification task. This reflects the scientific nature of task evaluation and the intelligence of knowledge management, establishes an intelligent airworthiness certification task theoretical system, and supports the efficient implementation of knowledge-based tasks.

[0072] Step S310: In response to the user's feedback operation, receive inspection feedback data for the inspection result data.

[0073] In one example, before step S310, the method further includes: before receiving inspection feedback data for the inspection result data in response to the user's feedback operation, the method further includes: generating a reasoning process for the inspection result data based on a preset tracing mechanism, and visually displaying the reasoning process; wherein the reasoning process includes multiple reasoning steps, and each reasoning step corresponds to an original evidence text that can be jumped to.

[0074] In one example, step S310 includes: receiving inspection feedback data for the inspection result data in response to user feedback operations on both the inference step and the inspection result data.

[0075] For example, such as Figure 4 As shown, this step is used to evaluate the integration of results, the traceability mechanism of evaluation results, and their visualization presentation, as detailed below: 1. Integration of evaluation results Since Deepseek-R1-32B is a large language model with its own CoT (Copy of Reasoning), this application employs a Guided CoT (Copy of Reasoning) prompting strategy to enable a traceable reasoning process. This strategy requires the LLM (Large Language Model) to follow a predefined series of analytical steps before reaching a final conclusion, effectively simulating the structured diagnostic process of human airworthiness experts. The CoT structure template mandates that the Deepseek-R1-32B model adhere to the following rigorous analytical steps: ① Identify the target: Determine the core systems, components, and design intent of the chapter to be evaluated.

[0076] ② Regulatory anchoring: Based on GraphRAG evidence, identify and articulate directly applicable CCAR provisions and verify their multi-hop relationship constraints.

[0077] ③ Technical Standard Verification: Based on RAG evidence, verify whether the design details meet recognized industry standards and methodologies.

[0078] ④ Risk elimination: Refer to historical airworthiness cases to assess whether the design has effectively avoided or mitigated known risk points.

[0079] ⑤ Uncertainty Quantification: Based on the completeness of the retrieved evidence, the relevance score, and the existence of evidence conflicts, the confidence level of this judgment is quantified.

[0080] ⑥ Final structured decision: Based on the above steps, a clear conclusion of compliance is given.

[0081] Guided by the above, LLM assesses the compliance of each chapter in the airworthiness document to be inspected with three main categories of documents: airworthiness regulations, industry standards, and airworthiness case studies (such as technical report templates). By integrating the assessment results of each chapter, LLM can obtain the complete assessment results of the entire airworthiness document to be inspected, i.e., the inspection result data.

[0082] 2. The traceability mechanism of the evaluation results and its visualization presentation The purpose of the tracing mechanism and visualization of the evaluation results is to intuitively demonstrate the judgment criteria of the model, especially the dual-architecture retrieval process. The visualization of the evaluation results mainly focuses on the visualization of the large language model's thought process and the tracing mechanism and visualization of the dual-architecture retrieval process. Specifically, such as... Figure 3 As shown, the visualization is as follows: ①CoT path visualization This application presents the LLM's guided CoT reasoning process clearly in the form of structured text. Each reasoning step (such as "regulatory anchoring" or "technical standard verification") can be clicked to expand and directly link to the original evidence text it cites.

[0083] ② Airworthiness regulations traceability and map visualization For hierarchical and interconnected airworthiness regulation evidence extracted from the knowledge graph, its multi-hop path must be demonstrated. This structured tracing path clearly shows the progression from final judgment → section to be evaluated → core regulation clause node cited → relationship type between nodes (e.g., "requirement," "applies to," "includes," or "interpretation") → intermediate nodes / edges in the path → associated advisory / notification nodes. Simultaneously, for inspection results data related to airworthiness regulations, the local substructure traversed within the knowledge graph can be clearly displayed, including core regulation nodes, activated edges (relationships), and associated supplementary document nodes. The output format records the complete node / edge sequence traversed from the query to the final evidence node, ensuring the transparency and reproducibility of the graph reasoning.

[0084] ③ Industry standards, case studies, and their visualization For unstructured text fragments retrieved from vector databases, the source tracing mechanism focuses on tracking semantic information and original sources to verify the relevance of the evidence. Tracking records include: the semantic intent used for retrieval (the subquery generated by the LLM), and the name of the original document (DO-178C document or NTSB case report), serving as strength evidence supporting the judgment for that fragment. Simultaneously, the original text of the document to be evaluated, along with the LLM-generated subquery and retrieved industry standard or case text fragments, are displayed side-by-side for quick identification of the document's origin.

[0085] Finally, based on the feedback from the evaluation experts on each reasoning step and the inspection result data, inspection feedback data on the inspection result data is received.

[0086] Step S311: If the inspection feedback data indicates that the inspection accuracy is less than the preset threshold, then the review task inspection system is optimized at least once, and inspection result data with an inspection accuracy greater than or equal to the preset threshold is generated based on the optimized review task inspection system.

[0087] In one example, step S311 includes: if the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, then based on the inspection feedback data, perform at least one optimization process on the graph-based retrieval enhancement generation and retrieval enhancement generation database in the review task inspection system; and based on the optimized review task inspection system, generate inspection result data with an inspection accuracy greater than or equal to the preset threshold.

[0088] For example, Figure 5 A technical block diagram of an optimization layer provided in an embodiment of this application, such as... Figure 5 As shown, the optimization layer aims to continuously and automatically correct the knowledge base of the data layer and the reasoning model of the evaluation layer by systematically receiving inspection feedback data from evaluation experts. This ensures that the implementing entity's system maintains the highest accuracy and stability in an environment of rapidly evolving regulations and technologies. The optimization layer mainly consists of the following parts: 1. Check the collection of feedback data. The inspection feedback data involves assessment experts judging the correctness of the inspection results and assessment process at the evaluation layer, transforming the experts' knowledge into structured feedback signals that can be used for training. Assessment experts provide feedback on the evaluation results through a traceability mechanism and its visualization, including marking the correctness of the final conformity ruling of the LLM, providing fine-grained scoring for each step of the CoT reasoning process generated by the LLM, assessing logical rigor, accuracy of evidence citation, and completeness of steps, and labeling the retrieved Top-K evidence blocks as "correctly relevant," "incorrectly relevant," "misleading," or "missing key evidence."

[0089] 2. Continuous refinement and adaptive maintenance of the knowledge base architecture The continuous refinement and adaptive maintenance of the knowledge base architecture drives the correction of the data layer knowledge base through the collected inspection feedback data, thereby ensuring the accuracy and structured quality of the knowledge source.

[0090] ①GraphRAG Structured Knowledge Correction: For erroneous feedback involving regulatory clauses and relationships, hierarchical knowledge graph correction is performed. Specifically, relationship correction: Incorrect relationship types between entities are corrected (e.g., changing "contains" to "applies to"), or spurious triples generated during model extraction are removed. Structure expansion: Experts can annotate missing or critical regulatory associations; the system generates new triples based on these annotations for incremental import into the Neo4j graph database, thereby improving the multi-hop reasoning paths of the knowledge graph. Counterfactual analysis: Using reasoning chains marked as erroneous by experts, the system can perform counterfactual analysis, simulating modifications to specific nodes or relationships in the graph and observing whether the LLM reasoning results become correct, thus locating and repairing hidden logical flaws in the graph.

[0091] ② Traditional RAG Data Lifecycle Maintenance: For industry standards and airworthiness case data, the optimization process focuses on content accuracy and metadata refinement. Specifically, data timeliness: Checking feedback data to see if specific standards or advisory circulars have been superseded or repealed by newer versions triggers automatic crawler updates, ensuring that the content in the RAG database is always based on the latest and most authoritative technical guidelines. Metadata refinement: For evidence fragments marked as "incorrectly relevant," "misleading," or "missing key evidence," experts can correct their original metadata tags (such as document type, subject area), thereby optimizing the metadata filtering accuracy in the initial retrieval stage and improving subsequent query efficiency.

[0092] 3. Depth calibration of LLM Based on the feedback from the evaluation layer, an alignment strategy based on reinforcement learning is used to continuously calibrate the core LLMDeepseek-R1-32B model of the evaluation layer, so that its reasoning logic can be closer to the thinking mode of human airworthiness experts.

[0093] First, Supervised Fine-Tuning (SFT) is used as the first stage of deep model calibration alignment. The input for this stage is the text of the chapter to be evaluated and the retrieved Top-K evidence blocks (from GraphRAG and traditional RAG). The fine-tuning data is a chain of thought (CoT) consisting of a sequence of structured reasoning steps corrected and approved by experts. This sequence strictly follows a predefined six-step analysis framework in the evaluation layer, ensuring logical rigor and accurate anchoring to airworthiness regulations. Its output is the structured decision finally confirmed by the experts (such as a compliance conclusion in JSON Lines format). The goal of SFT is to teach Deepseek-R1-32B to stably mimic expert thinking, follow the requirements of structured prompts, and output CoT reasoning chains with high format consistency and logical coherence.

[0094] Secondly, Reinforcement Learning Fine-Tuning (RFT) is used to continuously optimize and calibrate the inference process. RFT assigns rewards based on fine-grained scores from expert evaluators on the CoT inference chain. Experts directly tell the model which inference path is "better" or "worse" based on their scores, rather than simply whether the final answer is right or wrong. RFT optimizes the weights of Deepseek-R1-32B, forcing the model to avoid errors such as logical jumps and ignoring key regulatory evidence, making the inference process more robust. The reward function primarily penalizes the use of incorrect or fabricated evidence (illusions), thereby reinforcing the model's reliance on and accurate use of retrieved Top-K evidence during inference. Through the RFT mechanism, LLM can incrementally learn based on continuous expert feedback, ensuring that its inference strategy remains aligned with evolving airworthiness certification standards in the long term.

[0095] Therefore, by optimizing the entire lifecycle model for airworthiness certification assessment, and based on the verification of inspection results data by assessment experts, focusing on improving the performance of the large model, continuous optimization of the assessment can be achieved, thereby ensuring the quality, rationality, and adaptability of the assessment results. The assessment layer employs methods such as model fine-tuning, workflow iteration, data optimization, and prompt word optimization to iteratively optimize model performance based on inspection results data. The data layer uses a hybrid retrieval approach combining knowledge graphs and traditional RAG (Research and Analysis Group) to ensure data quality while maintaining real-time updates, ensuring that the model can accurately match the relevant knowledge base based on the query.

[0096] In this embodiment, basic data is acquired, including airworthiness regulations, industry standards, and airworthiness cases. Based on preset filtering conditions, the basic data is filtered and format-converted to obtain basic data in a preset format. In response to manual annotation, the basic data in the preset format is annotated to generate annotated data. Based on the annotated data and the basic data in the preset format, graph-based retrieval enhancement generation and a retrieval enhancement generation database are generated. Airworthiness documents to be inspected for the aircraft are acquired. Based on the document decomposition strategy of regular expression matching in the certification task inspection system, the chapter content of the airworthiness documents to be inspected is extracted, generating text blocks corresponding to each chapter. Based on the semantic query rewriting strategy in the preset large language model, multiple text blocks are converted into query sets. The query type of the query set is determined. Based on the query type, the query set is assigned to the target retrieval path for asynchronous hybrid retrieval processing, generating inspection result data. The target retrieval path is either a hybrid retrieval enhancement path or a single retrieval path. The hybrid retrieval enhancement path is a preset graph-based retrieval enhancement generation and a preset retrieval enhancement generation database; the single retrieval path is either graph-based retrieval enhancement generation or a retrieval enhancement generation database. Based on a pre-defined cross-encoding reordering model, the context blocks of the inspection result data are scored to generate scores. Inspection result data whose scores meet pre-defined scoring criteria are selected as the filtered inspection result data. Responding to user feedback, inspection feedback data is received regarding the inspection result data. If the inspection feedback data indicates that the inspection accuracy is less than a pre-defined threshold, the review task inspection system is optimized at least once, and based on the optimized review task inspection system, inspection result data with an inspection accuracy greater than or equal to the pre-defined threshold is generated. This solution utilizes a pre-established certification task inspection system and technologies such as graph-based retrieval enhancement generation and retrieval enhancement generation (RAG) within that system to conduct multi-dimensional inspections of airworthiness documents. It can quickly adapt to the constantly updated airworthiness regulations, integrate multi-source data, and form a dynamic evaluation network covering the entire aircraft lifecycle. This breakthrough overcomes the limitations of multi-level and multi-granularity airworthiness certification task evaluation, enabling intelligent evaluation of airworthiness certification tasks and continuous optimization of the evaluation process. It effectively solves the core bottlenecks of traditional evaluation methods, such as reliance on human experience, delayed response, and difficulty in traceability. This provides crucial support for achieving high-precision, interpretable, and self-iterative intelligent certification, significantly improving the efficiency and accuracy of airworthiness certification tasks.

[0097] Figure 6 An operational architecture diagram of an airworthiness certification platform provided in this application embodiment is shown below. Figure 6As shown, this platform uses the LangChain framework as its core integration and orchestration tool to achieve unified management and web interface display of the data layer, evaluation layer, and optimization layer. LangChain transforms the complex modular RAG process into an efficient and auditable execution pipeline. At the data layer, this includes knowledge base construction and building a business data file system. Knowledge base construction includes vector representation, data retrieval, answer generation, RAG vector database, RAG architecture generation; graph structure, vector representation, incremental updates, two-layer retrieval, knowledge graph vector database, and knowledge graph hierarchy generation. Building the business data file system includes acquiring real data, airworthiness regulations, industry standards, airworthiness cases, and document conversion to OCR and semantic segmentation.

[0098] In the evaluation layer, its components—including subquery generation, dynamic routing, hybrid retrieval of GraphRAG and traditional RAG, and the re-ranking model—are pluggably connected. The Deepseek-R1-32B model is also encapsulated, forcing it to follow a Guided CoT (Coding in Thought) hint strategy to ensure the structured and auditable nature of the inspection results data. Furthermore, a large LLM (Limited Learning Model) is used to score the inspection results data, generating inspection feedback data.

[0099] At the optimization layer, LangChain receives structured inspection feedback data from airworthiness experts and transforms the structured feedback into reward signal data. It then performs analysis and optimization, forming a complete closed loop from data and reasoning to continuous learning. This intuitively demonstrates the dual-architecture retrieval path (multi-hop relationships in the graph and semantic matching of RAG) and the complete inference chain of LLM, providing airworthiness experts with a highly transparent and auditable user interaction experience.

[0100] It should be noted that the algorithms, models, and numerical values ​​in the above embodiments are merely examples and are not intended to be limiting.

[0101] Corresponding to the above method, Figure 7 An airworthiness text quality determination device based on an airworthiness certification platform is provided in this application embodiment, such as... Figure 7 As shown, the device includes: Module 41 is used to acquire the airworthiness documents to be inspected for the aircraft; The inspection module 42 is used to determine the query type of the airworthiness document to be inspected based on the preset approval task inspection system, and to determine the target retrieval path corresponding to the query type; to inspect and process the airworthiness document to be inspected according to the target retrieval path, and to generate inspection result data; wherein, the preset approval task inspection system is generated based on the basic data that has been deeply refined, and the approval task inspection system includes multiple preset dimensions of inspection standards; the target retrieval path is a hybrid retrieval enhanced path, or a single retrieval path; The receiving module 43 is used to receive inspection feedback data for the inspection result data in response to the user's feedback operation; The generation module 44 is used to perform at least one optimization process on the review task inspection system if the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, and to generate inspection result data with an inspection accuracy greater than or equal to the preset threshold based on the optimized review task inspection system.

[0102] The functions of each functional unit of the airworthiness text quality determination device based on the airworthiness certification platform provided in the above embodiments of this application can be implemented through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the airworthiness text quality determination device based on the airworthiness certification platform provided in the embodiments of this application will not be repeated here.

[0103] Corresponding to the above method, Figure 8 A schematic diagram of another airworthiness text quality determination device based on an airworthiness certification platform provided in this application embodiment is shown below. Figure 8 As shown, the inspection module 42 includes: Extraction unit 421 is used to extract the chapter content of the airworthiness document to be inspected based on the document decomposition strategy of regular expression matching in the review task inspection system, and generate text blocks corresponding to each chapter content. Transformation unit 422 is used to transform multiple text blocks into query sets based on a semantic query rewriting strategy in a preset large language model. Unit 423 is used to determine the query type of the query set; The generation unit 424 is used to allocate the query set to the target retrieval path for asynchronous hybrid retrieval processing based on the query type, and generate inspection result data; wherein, the target retrieval path is a hybrid retrieval enhancement path or a single retrieval path; the hybrid retrieval enhancement path is a preset graph-based retrieval enhancement generation and a preset retrieval enhancement generation database; the single retrieval path is a graph-based retrieval enhancement generation or a retrieval enhancement generation database.

[0104] In one example, conversion unit 422 includes: For each text block, based on the semantic query rewriting strategy in the preset large language model, the text block is processed to extract abstract intent and key entities, and a query definition is generated; the text block includes chapter content and the metadata corresponding to the chapter content; Based on concept matching, determine at least one subquery corresponding to the query definition; The extracted key entities are merged with at least one subquery to generate a query set.

[0105] In one example, the device also includes: After inspecting the airworthiness document to be inspected according to the target retrieval path and generating inspection result data, the context blocks of the inspection result data are scored based on a preset cross-encoding reordering model to generate scores. The inspection result data that determines whether the score meets the preset scoring criteria is the filtered inspection result data.

[0106] In one example, the device also includes: Before receiving inspection feedback data in response to user feedback, the system generates a reasoning process for the inspection results data based on a preset tracing mechanism and visualizes the reasoning process. The reasoning process includes multiple reasoning steps, and each reasoning step corresponds to a jumpable original evidence text.

[0107] Receiver module 43 includes: In response to user feedback on each reasoning step and the inspection result data, it receives inspection feedback data on the inspection result data.

[0108] In one example, module 44 is generated, including: If the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, then based on the inspection feedback data, the graph-based retrieval enhancement generation and retrieval enhancement generation database in the review task inspection system are optimized at least once; and based on the optimized review task inspection system, inspection result data with an inspection accuracy greater than or equal to the preset threshold is generated. In one example, the device further includes: Acquire basic data; this includes airworthiness regulations, industry standards, and airworthiness case studies. Based on preset filtering criteria, the basic data is filtered and converted to obtain basic data in the preset format. In response to manual annotation operations, the system annotates basic data in a preset format to generate annotated data; and based on the annotated data and the basic data in the preset format, it generates a graph-based retrieval enhancement generation database and a retrieval enhancement generation database.

[0109] The functions of each functional unit of the airworthiness text quality determination device based on the airworthiness certification platform provided in the above embodiments of this application can be implemented through the above method steps. Therefore, the specific working process and beneficial effects of each unit in the airworthiness text quality determination device based on the airworthiness certification platform provided in the embodiments of this application will not be repeated here.

[0110] This application also provides an electronic device, such as... Figure 9 As shown, it includes a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540.

[0111] Memory 530 is used to store computer programs; The processor 510 performs the above steps when executing the program stored in the memory 530. The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not indicate that there is only one bus or one type of bus.

[0112] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0113] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0114] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0115] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments for solving the problem can be found in [reference needed]. Figure 1 The steps in the illustrated embodiments are used to implement the electronic device. Therefore, the specific working process and beneficial effects of the electronic device provided in this application will not be repeated here.

[0116] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the airworthiness text quality determination method based on any of the above embodiments.

[0117] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the airworthiness text quality determination method based on an airworthiness certification platform as described in any of the above embodiments.

[0118] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments in this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0122] Although preferred embodiments have been described in this application, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of this application.

[0123] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims in this application and their equivalents, then this application also intends to include these modifications and variations.

Claims

1. A method for determining the quality of airworthiness documents based on an airworthiness certification platform, characterized in that, The method includes: Obtain the airworthiness documents of the aircraft pending inspection; Based on a pre-defined approval task inspection system, the query type of the airworthiness document to be inspected is determined, and the target retrieval path corresponding to the query type is determined; the airworthiness document to be inspected is processed according to the target retrieval path to generate inspection result data; the pre-defined approval task inspection system is generated based on the basic data that has been deeply refined, and the approval task inspection system includes multiple pre-defined dimensions of inspection standards; the target retrieval path is a hybrid retrieval enhanced path, or a single retrieval path. In response to user feedback, the system receives inspection feedback data for the inspection result data; if the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, the system performs at least one optimization process on the review task inspection system, and generates inspection result data with an inspection accuracy greater than or equal to the preset threshold based on the optimized review task inspection system. Based on a pre-defined approval task inspection system, the query type of the airworthiness document to be inspected is determined, and the target retrieval path corresponding to the query type is determined; the airworthiness document to be inspected is processed according to the target retrieval path to generate inspection result data, including: Based on the document decomposition strategy of regular expression matching in the approval task inspection system, the chapter content of the airworthiness document to be inspected is extracted to generate text blocks corresponding to each chapter content. Based on the semantic query rewriting strategy in the pre-defined large language model, multiple text blocks are transformed into query sets respectively; Determine the query type of the query set; Based on the query type, the query set is assigned to the target retrieval path for asynchronous hybrid retrieval processing to generate inspection result data; wherein, the target retrieval path is a hybrid retrieval enhancement path or a single retrieval path; the hybrid retrieval enhancement path is a preset graph-based retrieval enhancement generation and a preset retrieval enhancement generation database; the single retrieval path is a graph-based retrieval enhancement generation or a retrieval enhancement generation database; Based on the semantic query rewriting strategy in the pre-defined large language model, multiple text blocks are transformed into query sets, including: For each text block, based on the semantic query rewriting strategy in the preset large language model, the text block is subjected to abstract intent and key entity extraction processing to generate a query definition; wherein, the text block includes chapter content and the metadata corresponding to the chapter content; Based on concept matching, at least one subquery corresponding to the query definition is determined; The extracted key entities are merged with at least one subquery to generate a query set; After processing the airworthiness document to be inspected according to the target retrieval path and generating inspection result data, the method further includes: Based on a preset cross-encoding reordering model, the context blocks of the inspection result data are scored to generate scores; The inspection result data that determines whether the score meets the preset scoring criteria is the filtered inspection result data. If the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, then the review task inspection system is optimized at least once, and based on the optimized review task inspection system, inspection result data with an inspection accuracy greater than or equal to the preset threshold is generated, including: If the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, then based on the inspection feedback data, the graph-based retrieval enhancement generation and retrieval enhancement generation database in the review task inspection system are optimized at least once; and based on the optimized review task inspection system, inspection result data with an inspection accuracy greater than or equal to the preset threshold is generated.

2. The method as described in claim 1, characterized in that, Before receiving inspection feedback data regarding the inspection result data in response to user feedback, the method further includes: Based on a preset tracing mechanism, a reasoning process for generating the inspection result data is generated, and the reasoning process is visualized; wherein, the reasoning process includes multiple reasoning steps, and each reasoning step corresponds to an original evidence text that can be jumped to.

3. The method as described in claim 2, characterized in that, The response to user feedback, receiving inspection feedback data regarding the inspection result data, includes: In response to user feedback on both the inference step and the inspection result data, inspection feedback data is received for the inspection result data.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Acquire basic data; wherein, the basic data includes airworthiness regulations, industry standards, and airworthiness cases; Based on preset filtering conditions, the basic data is filtered and format converted to obtain basic data in a preset format. In response to manual annotation operations, the system annotates basic data in a preset format to generate annotated data; and based on the annotated data and the basic data in the preset format, it generates a graph-based retrieval enhancement generation database and a retrieval enhancement generation database.

5. An airworthiness text quality determination device based on an airworthiness certification platform, characterized in that, The device includes: The acquisition module is used to acquire the airworthiness documents to be inspected for the aircraft. The inspection module is used to determine the query type of the airworthiness document to be inspected based on a preset approval task inspection system, and to determine the target retrieval path corresponding to the query type; to inspect the airworthiness document to be inspected according to the target retrieval path, and to generate inspection result data; wherein, the preset approval task inspection system is generated based on the basic data that has been deeply refined, and the approval task inspection system includes multiple preset dimensions of inspection standards; the target retrieval path is a hybrid retrieval enhanced path, or a single retrieval path. The receiving module is used to receive inspection feedback data for the inspection result data in response to user feedback operations; The generation module is used to optimize the review task inspection system at least once if the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, and generate inspection result data with the inspection accuracy greater than or equal to the preset threshold based on the optimized review task inspection system. The inspection module includes: The extraction unit is used to extract the chapter content of the airworthiness document to be inspected based on the document decomposition strategy of regular expression matching in the review task inspection system, and generate text blocks corresponding to each chapter content. The transformation unit is used to transform multiple text blocks into query sets based on the semantic query rewriting strategy in the preset large language model. A determining unit is used to determine the query type of the query set; A generation unit is used to allocate the query set to a target retrieval path for asynchronous hybrid retrieval processing based on the query type, and generate inspection result data; wherein, the target retrieval path is a hybrid retrieval enhancement path or a single retrieval path; the hybrid retrieval enhancement path is a preset graph-based retrieval enhancement generation and a preset retrieval enhancement generation database; the single retrieval path is a graph-based retrieval enhancement generation or a retrieval enhancement generation database; The conversion unit includes: For each text block, based on the semantic query rewriting strategy in the preset large language model, the text block is subjected to abstract intent and key entity extraction processing to generate a query definition; wherein, the text block includes chapter content and the metadata corresponding to the chapter content; Based on concept matching, at least one subquery corresponding to the query definition is determined; The extracted key entities are merged with at least one subquery to generate a query set; The device further includes a sorting module, the sorting module comprising: After inspecting the airworthiness document to be inspected according to the target retrieval path and generating inspection result data, the context blocks of the inspection result data are scored based on a preset cross-encoding reordering model to generate scores. The inspection result data that determines whether the score meets the preset scoring criteria is the filtered inspection result data. The generation module includes: If the inspection feedback data indicates that the inspection accuracy is less than a preset threshold, then based on the inspection feedback data, the graph-based retrieval enhancement generation and retrieval enhancement generation database in the review task inspection system are optimized at least once; and based on the optimized review task inspection system, inspection result data with an inspection accuracy greater than or equal to the preset threshold is generated.

Images