A large model reliable fault diagnosis method and system for an aviation maintenance scene

Abstract

The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The application discloses a large model reliable fault diagnosis method for an aviation maintenance scene, and relates to the technical field of aviation maintenance diagnosis. The method and system can realize stable generation, reliable verification and risk controllable output of aviation maintenance diagnosis content.

Description

Technical Field

[0001] This invention belongs to the field of aviation maintenance and intelligent diagnostic technology, and in particular relates to a reliable fault diagnosis method and system for large models in aviation maintenance scenarios. Background Technology

[0002] As air transport continues to expand and aircraft systems become increasingly complex, their operational safety and maintenance efficiency directly impact the successful completion of flight missions. In actual maintenance, technicians typically need to consult numerous technical manuals, historical fault records, and maintenance logs, relying on experience for comprehensive judgment. However, modern aircraft consist of numerous components and highly coupled systems. Given the complexity of multi-source information and the variability of operating conditions, any anomaly in any link can trigger a system-wide chain reaction. Therefore, quickly and accurately locating faults and developing reliable handling procedures has become a key technical requirement for ensuring airworthiness and improving maintenance efficiency.

[0003] In recent years, intelligent methods have been gradually introduced into the maintenance and diagnosis of complex equipment, and have made initial progress in some tasks. For example, Chinese patent CN115269860A proposes a method for constructing an aircraft maintenance knowledge graph based on an improved BERT_BiLSTM_CRF and relation extraction model. By structurally modeling entities and their relationships in maintenance logs, it provides auxiliary support for subsequent fault analysis. However, this type of method relies on a pre-set dictionary and a fixed model architecture, making it difficult to adapt to complex maintenance semantics across systems and multiple steps, and even more difficult to handle diagnostic tasks that require comprehensive reasoning based on multiple source documents. On the other hand, Chinese patent CN118312896A discloses a fault diagnosis expert system based on a large model. By extracting features from diagnostic rules and performing reasoning, it improves the automation capability of diagnosis to some extent. However, its reasoning process still relies on static rules, lacks a consistency verification mechanism with the content of technical manuals, and does not constrain the illusion problems that may arise from the large model, such as fictitious steps and incorrect part numbers, making it difficult to construct a traceable chain of evidence for the diagnostic process. Existing technologies are insufficient to meet the stringent requirements of aircraft maintenance scenarios in terms of reliability, consistency, and verifiability.

[0004] Therefore, how to construct a large-scale model diagnostic method and system that is suitable for aviation maintenance scenarios, has reliable knowledge adaptation capabilities, traceable evidence chain construction capabilities, and hallucination detection mechanisms, so as to improve the credibility, verifiability, and security of the generated results, has become an urgent technical problem to be solved in the field of intelligent aviation maintenance. Summary of the Invention

[0005] The purpose of this invention is to provide a reliable fault diagnosis method and system for large-scale models in aviation maintenance scenarios, so as to achieve stable generation, reliable verification and risk-controllable output of aviation maintenance diagnostic content.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A reliable fault diagnosis method for large-scale models in aviation maintenance scenarios, the method comprising the following steps: S1: Collect and structure multi-source aviation maintenance data, construct an instruction dataset, and use LoRA to perform domain-specific adaptation training on the pre-trained model to obtain a domain model for aviation troubleshooting tasks; S2: Standardize the input of the fault description and perform keyword search and vector search on a multi-source heterogeneous knowledge base; then normalize and weight the scores of the multi-way search results to generate a unified sorted list of candidate documents, and select a number of high-scoring documents to perform S3. S3: Perform dynamic context expansion on high-scoring documents to restore continuous semantic structure, obtain candidate evidence, and then perform semantic relevance verification on each segment of the candidate evidence to obtain a set of structured evidence. S4: Using the structured evidence set in S3 as the input to the domain model of the aviation troubleshooting task in S1, the domain model generates structured troubleshooting steps and automatically fills in reference information, and outputs the troubleshooting solution. S5: Decompose the troubleshooting solution into several sub-steps in a structured manner, generate verification questions for the sub-steps and sample candidate answers in multiple rounds; construct the semantic distribution of answers through semantic implication judgment and semantic clustering, and calculate the semantic entropy based on the semantic distribution to quantify the uncertainty of the sub-steps; S6: Combining fuzzy expression recognition with the uncertainty classification in S5 to label the risk level of each sub-step, and automatically triggering manual review of each sub-step, after manual confirmation, revision or replacement, a reliable aviation maintenance diagnosis solution is finally formed.

[0007] The method provided by this invention organically integrates efficient parameter fine-tuning, enhanced retrieval generation, citation verification, semantic entropy illusion detection, and manual review mechanism to construct an end-to-end closed-loop system of "knowledge injection - evidence retrieval - risk identification - manual intervention". This system aims to solve the problems of illusion, untraceability, and unquantifiable risks in existing large-scale language models in safety-critical scenarios, thereby achieving stable generation, reliable verification, and controllable risk output of aviation maintenance diagnostic content.

[0008] In S1, the pre-trained model can be a large-scale language model pre-trained on a general corpus. Its model parameters are determined before the implementation of this invention, and it possesses general natural language understanding and generation capabilities. This invention does not limit the specific source or model of the pre-trained model, but rather adapts it to aviation maintenance and troubleshooting tasks through efficient parameter fine-tuning. S1 uses LoRA (Low-Rank Adaptation).

[0009] Furthermore, in S1, specifically: S11: Collect aviation maintenance manuals, operation and maintenance logs and related technical documents, covering fault phenomena, chapter numbers and troubleshooting logic, as raw maintenance data (providing a source for subsequent data structuring). S12: The original maintenance data is unified, standardized, and structured to obtain the instruction dataset; S13: Train the pre-trained model using the instruction dataset, and fine-tune it using LoRA while keeping the main parameters of the pre-trained model frozen during the training process; S14: Use the fine-tuned parameters as the domain adaptation weights of the pre-trained model after training to construct a domain model for aviation troubleshooting tasks.

[0010] Furthermore, in S12, the methods for unifying, standardizing, and structuring the raw maintenance data include: S121: Standardize terminology and extract and align fault symptoms and handling instructions from operation and maintenance logs and maintenance manuals to build a standardized "fault phenomenon-handling instruction" sample. S122: Perform semantic consistency verification on the samples, and perform structured merging on sample pairs that are determined to be semantically similar to generate high-quality instruction corpus with consistent semantics; S123: Based on the high-quality instruction corpus obtained in step S122 through semantic consistency verification and redundancy merging, a retrieval enhancement generation mechanism is used to recall reference documents, and a text generation model with hierarchical structure generation capability is used to automatically generate multi-level troubleshooting steps that conform to the maintenance manual format; the troubleshooting steps are organized into structured JSON instruction samples, and based on this, training set, validation set and test set are formed as instruction dataset.

[0011] In S121, the processing includes: terminology standardization and structured extraction, semantic consistency verification and redundancy merging, and generation of troubleshooting schemes based on hierarchical structures, specifically steps S121-S123 above. In S122, consistency verification is used to eliminate redundant items and improve data quality. The verification may include: exact string matching, similarity determination based on semantic reasoning models, and semantic equivalence verification based on deep language models.

[0012] In S13, while keeping the main parameters of the basic model frozen, a parameter-efficient fine-tuning method based on low-rank matrix updates is used to perform lightweight adjustments on some sub-layers in the attention structure, enabling the model to learn the professional terminology system, troubleshooting logic, and structured output format of the aviation maintenance field. In S14, the constructed aviation maintenance domain model is used for generating structured troubleshooting chains and outputting diagnostic logic that includes manual references.

[0013] Furthermore, in step S2, specifically: S21: Perform language detection and language unification on user input, convert the input content into a semantic space consistent with the knowledge base, and form a standardized query; S22: For the standardized query, keyword matching retrieval and vector semantic retrieval are performed respectively in a multi-source heterogeneous knowledge base including maintenance manuals, operation and maintenance logs and instruction sample library to obtain a candidate document set; S23: Perform normalization processing on the candidate document set to obtain normalized scores; and merge and sort the normalized scores according to preset weight parameters to generate a unified sorted candidate document list. S24: Select the highest-scoring documents from the uniformly sorted candidate document list and execute S3.

[0014] In S23, collaborative ranking of different retrieval methods is achieved by generating a unified ranking score.

[0015] In S24, the documents with the highest scores are selected as the candidate document set for subsequent context expansion and semantic verification.

[0016] Furthermore, in step S3, specifically: S31: Perform context expansion on several high-scoring documents from step S2, extract adjacent paragraphs and related index areas to restore the continuous semantic structure of the documents and form a set of candidate evidence fragments containing contextual information. S32: Determine the relevance of the candidate evidence fragment set, and remove fragments that are irrelevant to the query content or have semantic deviations, generating a structured evidence set that highly matches the query for subsequent troubleshooting and reasoning.

[0017] In S3, only information highly relevant to the query is retained, thus forming a refined and reliable structured evidence set. In S32, a semantic verification module with deep semantic understanding capabilities is used to execute S32. This structured evidence set is then used for subsequent troubleshooting and reasoning.

[0018] Furthermore, in step S4, specifically: S41: Based on the structured evidence set obtained in step S3, generate and query the corresponding structured troubleshooting steps using the domain model of the aviation troubleshooting task in S1, and fill in the preset reference field in each troubleshooting step. The reference field is used to represent the document source information related to the step. S42: Based on the reference fields generated in step S41, perform reference matching verification against the structured evidence set; for troubleshooting steps that can match the evidence source, record their corresponding reference mapping relationship; for troubleshooting steps that cannot match, mark their reference fields as unverified; generate a structured output containing reference mapping and troubleshooting steps based on the verification results, as a troubleshooting solution.

[0019] In S41, the referenced fields include, but are not limited to, manual page numbers, chapter numbers, and text fragment positions. In S42, matching and validation are used to achieve the auditability and traceability of the troubleshooting logic, thereby outputting a troubleshooting solution that is auditable and traceable.

[0020] Furthermore, in step S5, specifically: S51: Decompose the troubleshooting solution generated in step S4 into a structured form, and refine the troubleshooting steps in natural language form into several independently verifiable sub-steps. S52: For each sub-step, use the domain model of the aviation troubleshooting task in step S1 to automatically generate a query statement to verify the content of the sub-step, and generate a corresponding baseline answer, which will be used as a candidate answer later. S53: Based on the query statement in step S52, multiple candidate answers are obtained through diversified generation mechanisms, and these are combined with the baseline answer to form a candidate answer set; S54: Perform semantic relevance judgment on the candidate answer set and divide it into several semantic clusters based on the similarity of the candidate answer content; S55: Calculate semantic entropy based on the distribution characteristics of semantic clusters to quantify the uncertainty of sub-steps.

[0021] In S51, the process is broken down into sub-steps to facilitate subsequent item-by-item verification. The baseline answer generated in S52 serves as reference information for subsequent consistency analysis. The multiple candidate answers obtained in S53 are used to reflect the semantic changes of the model under different generation conditions. The semantic clusters divided in S54 are used to construct a semantic structure that reflects the distribution characteristics of the answers. The quantitative evaluation in S55 is used to identify troubleshooting steps that may have semantic deviations or insufficient reliability, thereby providing interpretability and risk warning capabilities for the overall diagnostic results.

[0022] Furthermore, in step S6, specifically: S61: Perform fuzzy expression recognition on the candidate answers obtained in step S5 to detect uncertain or semantic avoidance expressions contained therein, and adjust the risk level of the corresponding candidate responses or semantic distributions according to the recognition results; S62: Based on the semantic entropy in step S5 and combined with the adjustment results of the risk level after fuzzy expression recognition in step S61, generate risk scores for each sub-step, compare the risk scores with preset thresholds, and classify and label different risk levels according to the comparison results. S63: When the risk level exceeds the preset threshold, the system triggers a manual review process, outputting the steps that need to be reviewed to the maintenance engineer; the maintenance engineer confirms, revises or replaces the steps to form a reliable troubleshooting result that has been manually verified, and uses it as the final structured aviation maintenance diagnostic solution.

[0023] In S61, risks are adjusted to reflect potential model uncertainties or reasoning biases, and in S62, hierarchical labeling is performed so that maintenance personnel can quickly identify steps or decision points that may pose risks.

[0024] This invention also provides a large-scale reliable fault diagnosis system for aviation maintenance scenarios, the system comprising: The input standardization module standardizes the input of fault descriptions; The multi-source retrieval module performs keyword retrieval and vector retrieval on a multi-source knowledge base; then, it normalizes and weights the multi-source retrieval results to generate a unified sorted list of candidate documents and selects a number of high-scoring documents. The context semantic extension module performs dynamic context extension on high-scoring documents to restore continuous semantic structure, obtain candidate evidence, and then performs semantic relevance verification on each segment of the candidate evidence to obtain a set of structured evidence. The domain model inference module collects and structures multi-source aviation maintenance data, constructs an instruction dataset, and performs domain-specific adaptation training on the pre-trained model using LoRA to obtain a domain model for aviation troubleshooting tasks. The structured evidence set is used as the input to the domain model for aviation troubleshooting tasks in S1. The domain model generates structured troubleshooting steps and automatically fills in reference information. Subsequently, the reference information is matched and verified. References that do not correspond to the structured evidence set are marked as "unverified," and a troubleshooting solution is output. The uncertainty assessment module breaks down the troubleshooting solution into several sub-steps, generates verification questions for each sub-step, and performs multiple rounds of candidate answer sampling. It constructs the semantic distribution of answers through semantic implication judgment and semantic clustering, and calculates semantic entropy based on the semantic distribution to quantify the uncertainty of the sub-steps. The risk assessment and manual review module combines fuzzy expression recognition and uncertainty classification to label the risk level of each sub-step, and automatically triggers manual review of each sub-step. After manual confirmation, revision or replacement, a reliable aviation maintenance diagnostic solution is finally formed.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention discloses a reliable fault diagnosis method and system for large-scale models in aviation maintenance scenarios. Through domain-specific model training, multi-source retrieval fusion, and structured evidence construction, the system can automatically extract key fault-related information from complex maintenance texts and operational data and generate an auditable troubleshooting step chain, significantly improving the traceability and reliability of diagnostic results. This invention combines semantic entropy assessment and fuzzy expression recognition to quantify step-level uncertainty, effectively identifying potential hallucinations or insufficiently evidenced diagnostic content, avoiding misleading outputs that could impact maintenance activities. Furthermore, the system's manual review loop further ensures that key steps are professionally confirmed, guaranteeing that the final troubleshooting solution achieves a controllable level in terms of logical consistency, completeness of evidence, and engineering usability. Overall, this invention enhances the intelligence and standardization of the aviation troubleshooting process, providing a reliable technical foundation for building highly reliable large-scale model aviation maintenance applications. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall process of an embodiment of the present invention; Figure 2 This is a flowchart illustrating the instruction data construction and domain model adaptation process in an embodiment of the present invention. Figure 3 This is a system overall architecture diagram according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the multi-source retrieval enhancement and evidence construction process according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the risk detection process based on semantic entropy according to an embodiment of the present invention; Figure 6 This is an example input-output diagram of an embodiment of the present invention. Detailed Implementation

[0027] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0028] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, the main steps of the method provided by this invention are as follows:

[0029] S1: Collect and structure multi-source aviation maintenance data, construct a command dataset, and perform domain-specific adaptation training on the pre-trained model using LoRA to obtain the domain model M for aviation troubleshooting tasks. a .

[0030] In this embodiment, as Figure 2 As shown, the instruction data construction and domain model adaptation process includes, in sequence, multi-source data collection, terminology standardization and structured extraction, three-level semantic consistency verification, intelligent merging of redundant samples, generation of hierarchical troubleshooting steps, construction of JSON instruction samples, and LoRA domain-specific fine-tuning. Based on this process, the specific steps are as follows: S11: Collect aviation maintenance manuals, operation and maintenance logs, and related technical documents, covering fault phenomena, chapter numbers, and troubleshooting logic, to provide a source for subsequent data structuring.

[0031] S12: The raw data is unified, standardized, and structured to form an instruction dataset that can be used for model fine-tuning. This includes three sub-steps: terminology standardization and structured extraction, semantic consistency verification and redundancy merging, and hierarchical troubleshooting scheme generation.

[0032] S121: Standardize terminology and extract and align fault symptoms and handling instructions from operation and maintenance logs and maintenance manuals to build a standardized "fault symptoms - handling instructions" sample.

[0033] S122: Through a three-level verification mechanism consisting of group comparison, precise matching, semantic implication judgment and deep verification of large models, the consistency of samples is checked and semantically similar samples are intelligently merged to generate high-quality instruction corpus.

[0034] (1) Perform string-level precise matching on text samples within the same group to quickly identify and remove completely duplicate sample records.

[0035] (2) The DeBERTa-v2-xlarge-MNLI model (or a similar pre-trained model with natural language reasoning ability) is used to perform sentence pair semantic entailment reasoning on the sample pairs to determine whether they belong to highly similar samples. Potential redundant or nearly duplicate samples are initially screened out.

[0036] (3) Deep semantic validation is performed on the sample pairs that have passed the second-level screening by calling the API of a large commercial model (such as a general large model with conversational reasoning capabilities). During the deep semantic validation process, this embodiment uses the sample pairs in the form of "text A - text B" as model input and explicitly requests the model to return structured validation results in the prompts, including:

[0037] Are the two texts semantically equivalent? If they are not equivalent, indicate the differences (such as numerical differences, differences in operating conditions, or differences in step logic). Provide the category label for the difference (such as parameter difference, timing conflict, logical inconsistency, etc.).

[0038] For sample pairs deemed semantically similar or semantically redundant in the aforementioned three-level semantic verification process, this embodiment employs a structured intelligent merging strategy: API calls are used to merge redundant steps, generating unified samples that retain key information and avoid semantic conflicts, and the original source is traced and marked. This mechanism effectively compresses redundant data and improves sample quality.

[0039] S123: Utilize the RAG mechanism and DeepSeek-R1 (or similar large models with hierarchical structure generation capabilities) to perform document retrieval for standardized queries, generate flat troubleshooting steps that conform to the hierarchical format of the maintenance manual, and finally construct structured JSON instruction samples and complete the dataset partitioning.

[0040] In this embodiment, the RAG module uses a 1024-dimensional vector embedding model to perform top-K (K=5) document retrieval on the standardized query q to construct a contextual retrieval window; the retrieved document fragments are used as external knowledge input to DeepSeek-R1, and the model generates a troubleshooting plan containing multi-level troubleshooting steps according to the hierarchical numbering of the maintenance manual (such as 1, 1.1, 1.2, etc.).

[0041] To facilitate model training, this embodiment organizes the generated troubleshooting solutions into a unified two-field JSON sample. The "instruction" field corresponds to the fault description, and the "response" field represents the corresponding flat, hierarchical list of troubleshooting steps. The generated JSON sample is further divided into training, validation, and test sets for domain-specific model fine-tuning.

[0042] S13: While keeping the basic model parameters frozen, update the low-rank matrix of the attention sublayer through LoRA, so that the model learns the terminology system, troubleshooting logic and structured output format of the aviation maintenance field.

[0043] To ensure the feasibility and repeatability of the LoRA domain adaptation training process, this embodiment further provides specific training hyperparameter configurations.

[0044] In this embodiment, a 7B pre-trained base model is used as the foundation model for domain-specific adaptation. Typical engineering parameters used during model training and the LoRA insertion module are shown in Tables 1 and 2, respectively, including the learning rate, gradient accumulation strategy, maximum sequence length, Rank value, Dropout parameters, and the specific sub-layer type inserted by the LoRA module. During training, all parameters except for the LoRA insertion sub-layer are frozen; only the low-rank matrix parameters are updated, enabling efficient injection of domain knowledge within a lightweight parameter space. Those skilled in the art can adjust the following parameters within a reasonable range based on computational power and data scale, all of which fall within the scope of this invention.

[0045] Table 1 Model Training Hyperparameters .

[0046] Table 2 LoRA Parameter Settings and Insertion Module Description .

[0047] The above parameter configuration enables the training process to achieve stable convergence under lightweight conditions, which helps improve the model's terminology understanding, troubleshooting logic modeling, and the generation quality of structured troubleshooting chains in the field of aviation maintenance, thereby further ensuring the adaptation of model M. a Reliability.

[0048] S14: Use the parameters obtained from LoRA fine-tuning as adaptation weights to form the aviation maintenance domain model M. a It can be used to generate structured troubleshooting chains and output diagnostic logic that includes references from manuals.

[0049] In completing the domain model M a After training and construction, this invention enters the online fault diagnosis stage for actual maintenance scenarios. For example... Figure 3 As shown, the overall system architecture of this embodiment consists of an input standardization module, a multi-source retrieval module, a context expansion and semantic verification module, a domain model reasoning module, an uncertainty assessment module, and a risk assessment and manual review module.

[0050] Supported by the domain model, the above modules form a collaborative workflow, realizing a closed-loop diagnostic process from fault input parsing, evidence extraction, troubleshooting generation to uncertainty assessment and manual auditing. Based on this system architecture, the specific workflow of the online diagnostic phase is described below: like Figure 4As shown, the multi-source retrieval enhancement and evidence construction process in this embodiment includes, in sequence, language detection, keyword retrieval and vector retrieval, score normalization and weighted fusion, Top-K document filtering, dynamic context expansion, semantic relevance verification, and generation of structured troubleshooting steps. Based on this process, after the system completes input standardization, it performs parallel retrieval on the multi-source knowledge base and generates an auditable troubleshooting chain based on semantic verification.

[0051] S2: Standardize the input of the fault description and perform keyword retrieval and vector retrieval on a multi-source knowledge base; then normalize and weight the scores of the multi-way retrieval results to generate a unified ranked list of candidate documents, and select high-scoring documents as input for subsequent evidence construction.

[0052] In this embodiment, the system first performs language recognition and language unification on the fault description input by the user, and completes keyword retrieval and vector retrieval on a multi-source knowledge base. Then, the retrieval scores are fused to generate a sorted document list, specifically including the following steps: S21: Perform language detection and language unification on user input, convert the input content into a language semantic space consistent with the knowledge base, and generate query q.

[0053] S22: For query q, two sub-steps, keyword retrieval and vector semantic retrieval, are executed in parallel on heterogeneous knowledge bases such as maintenance manuals, operation and maintenance logs and instruction sample libraries. A unified ranking list of candidate documents is generated by score normalization and weighted fusion.

[0054] In this embodiment, the heterogeneous knowledge base includes: a text-based knowledge base for storing documents such as maintenance manuals and troubleshooting manuals; a structured database for storing operation and maintenance logs; and an instruction sample library built during model training. These multi-source knowledge bases can respectively support symbolic entry retrieval, operation and maintenance data query, and semantic instruction matching, thereby ensuring the comprehensiveness and consistency of the retrieval results.

[0055] S221: BM25 keyword search, used to match chapter numbers, symbolic entries, and technical terms to obtain a set of candidate documents. ;

[0056] S222: Vector semantic retrieval based on the embedding model Em, used to obtain semantically similar documents under cross-linguistic and complex sentence structure conditions, and obtain a candidate document set. .

[0057] S223: Perform min-max normalization on the BM25 retrieval score and the vector semantic retrieval score respectively to obtain the normalized score: ; And according to weight parameters The two types of search scores are weighted and fused: ; in, This represents the raw score of a document d and query q for a specific retrieval method (BM25 or vector semantic retrieval); , These represent the minimum and maximum scores for this retrieval method within the current candidate document set; normalized results. Mapped to an interval This is used to eliminate the problem of inconsistent scoring scales among different retrieval methods.

[0058] in, and These represent the normalized results of the vector semantic retrieval score and the BM25 score, respectively. For the weight parameters, satisfying This is used to balance the contributions of the two retrieval methods to the fusion score. Final fusion score. Used to uniformly sort candidate documents.

[0059] S23: From the list of sorted documents obtained in step S22, select the top K documents with the highest fusion scores as the candidate document set for subsequent context expansion and semantic verification, so as to ensure retrieval quality and control the upper bound of computational load.

[0060] S3: Perform dynamic context expansion on the selected document to restore the continuous semantic structure, and call the semantic verification module to filter candidate evidence segment by segment, retaining only information that is highly relevant to the query, thereby forming a refined and reliable set of structured evidence.

[0061] In this embodiment, the system performs contextual expansion and semantic relevance verification on the retrieved candidate documents to filter out valid evidence closely related to the query. Specifically, this includes the following steps: S31: Perform dynamic context window expansion on the Top-K documents selected in step S24, extract adjacent paragraphs and index areas, restore the continuous semantic structure of the documents, and construct an expanded set of candidate evidence blocks.

[0062] S32: Call the LLM semantic verification module to perform semantic relevance verification on each segment of the candidate evidence blocks from step S31, retaining only those related to the query. Highly matched content is used to remove redundant, off-topic, or conflicting segments, resulting in a refined, structured evidence set. .

[0063] S4: Generate structured troubleshooting steps based on the evidence set and automatically fill in the reference information; the system then matches and verifies the referenced content, marks the content that does not correspond to the evidence as "unverified", and finally outputs a troubleshooting solution that is auditable and traceable.

[0064] In this embodiment, the system automatically generates structured troubleshooting steps based on the filtered evidence set and completes the filling of reference information. Subsequently, the referenced content is verified to ensure the auditability of the troubleshooting results. Specifically, the steps include the following: S41: Guiding the Domain-Specific Model M through Prompt Engineering a The structured troubleshooting steps are generated based on evidence set E, and preset reference slots are filled in each step, including reference fields such as manual page number, chapter number and text fragment position.

[0065] S42: Based on the reference information generated in step S41, the system performs reference matching verification item by item against the structured evidence set E. If a troubleshooting step fails to match any evidence, its reference slot is automatically marked as "unverified". Based on the verification results, a reference mapping table and structured troubleshooting output are generated to ensure the auditability and traceability of the results.

[0066] After generating the structured troubleshooting solution based on evidence verification, the system further needs to quantitatively analyze the reliability and risk level of the troubleshooting steps to ensure that the solution is traceable, auditable, and robust. Therefore, this embodiment introduces an uncertainty measurement mechanism based on semantic entropy on top of the structured troubleshooting output to assess the risk level of each step. Specifically, as follows... Figure 5 As shown, after the troubleshooting step is verified and a reference mapping table is generated, the system will continue to carry out the risk detection process, sequentially executing steps such as step decomposition, verification question generation, candidate answer sampling, and semantic analysis, thereby constructing step-level uncertainty indicators.

[0067] S5: Based on the troubleshooting plan, perform step decomposition, generate verification questions for each step and conduct multiple rounds of candidate answer sampling; construct the semantic distribution of answers through semantic implication judgment and semantic clustering, and calculate semantic entropy accordingly to quantify the uncertainty of the steps.

[0068] In this embodiment, the system performs structured step decomposition on the generated troubleshooting solution and constructs a step-level uncertainty metric through multi-round answer sampling and semantic analysis, specifically including the following steps: S51: Perform structured decomposition processing on the natural language output of the diagnostic draft to obtain a set of diagnostic steps: ; Each diagnostic step It contains a set of one or more independently testable substeps: ; Each sub-step The corresponding minimum verifiable operation or judgment condition in the maintenance manual serves as the basic unit for subsequent uncertainty assessment.

[0069] S52: For the set of steps Each sub-step in Call the domain-specific model to generate the corresponding sub-step. A set of confirmatory question-answer pairs: Each of the verification questions Used to examine sub-steps from different semantic perspectives The consistency of the operational logic or technical conditions included in each reference answer It serves as the baseline semantic representation for the corresponding verification question and is used for subsequent semantic consistency analysis of candidate answers and uncertainty quantification calculation.

[0070] S53: For each verification question output in step S52 Under various preset sampling configurations, multiple rounds of random sampling are performed to generate corresponding candidate answer sets: ; The random sampling includes at least: a first sampling configuration that uses a low randomness parameter to generate semantically stable candidate answers; and a second sampling configuration that uses a high randomness parameter to generate semantically diverse candidate answers; and different rounds of sampling use mutually independent random seeds to ensure the statistical independence of candidate answers.

[0071] The token-level generation probability for each candidate answer is: ; The candidate answer set is used for subsequent semantic clustering and distribution estimation.

[0072] S54: Perform batch semantic judgment on the candidate answer set using the semantic entailment model, and complete semantic clustering based on the disjoint-set data structure algorithm to obtain a set of semantic clusters: ; The probability of each semantic cluster is estimated using a frequency approximation method as follows: .

[0073] S55: Based on sub-steps Each confirmatory question Given the semantic cluster probability distributions, calculate the corresponding semantic entropy for each: ; and for the same sub-step The semantic entropy of multiple verification questions is aggregated to obtain the overall semantic uncertainty measure of this sub-step: ; Among them, aggregate semantic entropy is used to characterize the overall uncertainty level of sub-steps under multiple semantic verification perspectives, when A higher value indicates that the sub-step has significant discrepancies under different verification semantics and carries a high potential risk.

[0074] S6: Combining fuzzy expression recognition and semantic entropy risk scoring to mark potentially high-risk steps and automatically triggering manual review; engineers revise or confirm relevant content based on the audit results, ultimately forming a credible aviation maintenance diagnostic solution.

[0075] In this embodiment, the system completes risk identification based on the semantic features and uncertainty indicators of candidate answers, and ensures the reliability of the diagnostic results through a manual review mechanism. Specifically, the system includes the following steps: S61: Perform fuzzy language recognition on the candidate answer set. This embodiment uses a keyword matching mechanism to detect fuzzy expressions, i.e., a preset fuzzy dictionary (such as "possible," "cannot be determined," "uncertain," etc.) is used to scan each candidate answer. When a candidate answer contains the above keywords or their equivalent expressions, the answer is marked as a "fuzzy expression." Based on this, if a semantic cluster contains one or more candidate answers that are judged to be fuzzy expressions, the risk weight of that semantic cluster is increased to identify avoidance illusions, semantic swings, or model uncertainty.

[0076] S62: Normalize the semantic entropy of the steps to obtain a step-level risk score; compare the risk score with a preset threshold, and the system uses a traffic light color coding mechanism to visually label different levels of certainty. Low risk corresponds to a green alert, medium risk to a yellow alert, and high risk to a red alert, so that maintenance personnel can quickly identify potential risk steps and abnormal decision points.

[0077] The corresponding semantic entropy normalization process is as follows: ; in, The risk score for the sub-step is a unified unit, ranging from 0% to 100%; max(SE) represents the maximum reference value of semantic entropy, corresponding to the case where all sampled candidate answers are judged to be semantically inequivalent and the semantic distribution reaches the maximum degree of dispersion under the same verification question, and is used as the upper bound for semantic entropy normalization.

[0078] Based on all sub-steps The level of uncertainty for the same diagnostic procedure The risk scores of multiple sub-steps are aggregated to calculate the steps. Uncertainty level: ; Furthermore, based on all steps Given the level of uncertainty, calculate the global reliability index: ; Where N represents the total number of diagnostic steps. The global reliability index reflects the consistency and robustness of the overall reasoning chain: the lower the G value, the more coherent and reliable the reasoning process; the higher the G value, the more potential for illusion or semantic inconsistencies in the diagnostic process.

[0079] S63: The system automatically triggers the manual review process, outputting the steps to be reviewed to the maintenance engineer; the maintenance engineer confirms, corrects or replaces the steps based on the audit results, generates the final credible structured aviation maintenance diagnostic solution S*, and uses it as the recommended troubleshooting result for the execution of aviation maintenance tasks.

[0080] To verify the applicability of the method of the present invention in actual aviation maintenance scenarios, a complete process demonstration of the method is presented based on a typical line maintenance fault report. This embodiment demonstrates the overall application process from user input of fault description, to evidence retrieval and construction, to troubleshooting step generation, and then to semantic uncertainty assessment and manual review, to prove the feasibility and effectiveness of the technical solution described in the present invention.

[0081] like Figure 6 As shown, after a flight landed, maintenance personnel submitted the following fault description (information such as flight schedules, part numbers, and mission codes has been anonymized): "During the start-up of a certain flight on a certain day, the N1 vibration value of the right engine was high, reaching 4.0. After the engine started successfully, the vibration value returned to normal. How should this be handled?"

[0082] The system performs language detection and normalization processing on the natural language input according to step S21. If the input language is found to be inconsistent with the language used in the knowledge base, the built-in language conversion module is invoked for conversion. After normalization, a standardized query q is generated and input into the parallel retrieval module in step S22.

[0083] The system performs keyword and vector semantic searches on multiple heterogeneous knowledge bases based on a standardized query q. Keyword searches focus on matching typical terms such as vibration inspection, startup anomalies, and accelerometer inspection, while vector semantic searches are used to find deeper semantic fragments related to vibration, including historical cases, component health inspections, and anomaly trend analysis. The search results are then sorted after directional consistency, normalization, and weighted fusion. Subsequently, the system performs context expansion and semantic verification on the top-ranked documents, removing information irrelevant or conflicting to the query, ultimately forming a refined structured evidence set E. This evidence set includes common causes of high vibration during the startup phase, inspection recommendations, methods for using monitoring data, and related trend analysis steps.

[0084] Based on the structured evidence set E, the system generates a structured troubleshooting step chain according to step S41, and verifies the references in each step. If the system does not find the corresponding evidence in the evidence set, the step is marked as "unverified".

[0085] The system further performs semantic uncertainty assessment on the troubleshooting step chain. Specifically, this includes breaking down each step into verifiable sub-steps, generating verifiable questions, constructing a candidate answer set through multiple rounds of sampling and performing semantic clustering, calculating semantic entropy, and labeling the risk level accordingly. For example, some steps with low uncertainty are marked as low risk; some steps have semantic dispersion or lack supporting evidence and are marked as medium or high risk.

[0086] During the manual review phase, engineers will adjust or supplement certain steps based on their experience and the maintenance manual. For example, they may confirm that certain steps fall under normal monitoring procedures and correct them, or replace content involving specific components to avoid misleading information. After manual review, an auditable and traceable troubleshooting solution S* is generated. The final output solution includes all verified or confirmed steps and retains the referenced matching status.

[0087] In summary, this invention discloses a reliable fault diagnosis method and system for large-scale models in aviation maintenance scenarios. Through domain-specific model training, multi-source retrieval fusion, and structured evidence construction, the system can automatically extract key fault-related information from complex maintenance texts and operational data, generating an auditable troubleshooting step chain, significantly improving the traceability and reliability of diagnostic results. The system combines semantic entropy assessment and fuzzy expression recognition to quantify step-level uncertainties, effectively identifying potential hallucinations or insufficiently evidenced diagnostic content, avoiding misleading outputs that could impact maintenance activities. Furthermore, the system's manual review loop further ensures that key steps are professionally confirmed, guaranteeing that the final troubleshooting solution achieves a controllable level in terms of logical consistency, completeness of evidence, and engineering usability. Overall, this invention enhances the intelligence and standardization of the aviation troubleshooting process, providing a reliable technical foundation for building highly reliable large-scale model aviation maintenance applications.

[0088] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be within the scope of protection of the present invention.

Claims

1. A reliable fault diagnosis method for large-scale models in aviation maintenance scenarios, characterized in that, The method includes the following steps: S1: Collect and structure multi-source aviation maintenance data, construct an instruction dataset, and use LoRA to perform domain-specific adaptation training on the pre-trained model to obtain a domain model for aviation troubleshooting tasks; S2: Standardize the input of the fault description and perform keyword search and vector search on a multi-source heterogeneous knowledge base; then normalize and weight the scores of the multi-way search results to generate a unified sorted list of candidate documents, and select a number of high-scoring documents to perform S3. S3: Perform dynamic context expansion on high-scoring documents to restore continuous semantic structure, obtain candidate evidence, and then perform semantic relevance verification on each segment of the candidate evidence to obtain a set of structured evidence. S4: Using the structured evidence set in S3 as the input to the domain model of the aviation troubleshooting task in S1, the domain model generates structured troubleshooting steps and automatically fills in reference information, and outputs the troubleshooting solution. S5: Decompose the troubleshooting solution into several sub-steps in a structured manner, generate verification questions for the sub-steps and sample candidate answers in multiple rounds; construct the semantic distribution of answers through semantic implication judgment and semantic clustering, and calculate the semantic entropy based on the semantic distribution to quantify the uncertainty of the sub-steps; S6: Combining fuzzy expression recognition with the uncertainty classification in S5 to label the risk level of each sub-step, and automatically triggering manual review of each sub-step, after manual confirmation, revision or replacement, a reliable aviation maintenance diagnosis solution is finally formed.

2. The large-scale reliable fault diagnosis method for aviation maintenance scenarios according to claim 1, characterized in that, In step S1, specifically: S11: Collect aviation maintenance manuals, operation and maintenance logs and related technical documents, covering fault phenomena, chapter numbers and troubleshooting logic, as raw maintenance data; S12: The original maintenance data is unified, standardized, and structured to obtain the instruction dataset; S13: Train the pre-trained model using the instruction dataset, and fine-tune it using LoRA while keeping the main parameters of the pre-trained model frozen during the training process; S14: Use the fine-tuned parameters as the domain adaptation weights of the pre-trained model after training to construct a domain model for aviation troubleshooting tasks.

3. The large-scale reliable fault diagnosis method for aviation maintenance scenarios according to claim 2, characterized in that, In S12, the methods for unifying, standardizing, and structuring raw maintenance data include: S121: Standardize terminology and extract and align fault symptoms and handling instructions from operation and maintenance logs and maintenance manuals to build standardized "fault symptoms-handling instructions" samples. S122: Perform semantic consistency verification on the samples, and perform structured merging on sample pairs that are determined to be semantically similar to generate high-quality instruction corpus with consistent semantics; S123: Based on the high-quality instruction corpus obtained in step S122 through semantic consistency verification and redundancy merging, a retrieval enhancement generation mechanism is used to recall reference documents, and a text generation model with hierarchical structure generation capability is used to automatically generate multi-level troubleshooting steps that conform to the maintenance manual format; the troubleshooting steps are organized into structured JSON instruction samples, and based on this, training set, validation set and test set are formed as instruction dataset.

4. The large-scale reliable fault diagnosis method for aviation maintenance scenarios according to claim 1, characterized in that, In step S2, specifically: S21: Perform language detection and language unification on user input, convert the input content into a semantic space consistent with the knowledge base, and form a standardized query; S22: For the standardized query, keyword matching retrieval and vector semantic retrieval are performed respectively in a multi-source heterogeneous knowledge base including maintenance manuals, operation and maintenance logs and instruction sample library to obtain a candidate document set; S23: Perform normalization processing on the candidate document set to obtain normalized scores; and merge and sort the normalized scores according to preset weight parameters to generate a unified sorted candidate document list. S24: Select the highest-scoring documents from the uniformly sorted candidate document list and execute S3.

5. The large-scale reliable fault diagnosis method for aviation maintenance scenarios according to claim 1, characterized in that, In step S3, specifically: S31: Perform context expansion on several high-scoring documents from step S2, extract adjacent paragraphs and related index areas to restore the continuous semantic structure of the documents and form a set of candidate evidence fragments containing contextual information. S32: Determine the relevance of the candidate evidence fragment set, and remove fragments that are irrelevant to the query content or have semantic deviations, generating a structured evidence set that highly matches the query.

6. The large-scale reliable fault diagnosis method for aviation maintenance scenarios according to claim 1, characterized in that, In step S4, specifically: S41: Based on the structured evidence set obtained in step S3, generate and query the corresponding structured troubleshooting steps using the domain model of the aviation troubleshooting task in S1, and fill in the preset reference field in each troubleshooting step. The reference field is used to represent the document source information related to the step. S42: Based on the reference fields generated in step S41, perform reference matching verification against the structured evidence set; for troubleshooting steps that can match the evidence source, record their corresponding reference mapping relationship; for troubleshooting steps that cannot match, mark their reference fields as unverified; generate a structured output containing reference mapping and troubleshooting steps based on the verification results, as a troubleshooting solution.

7. The large-scale reliable fault diagnosis method for aviation maintenance scenarios according to claim 1, characterized in that, In step S5, specifically: S51: Decompose the troubleshooting solution generated in step S4 into a structured form, and refine the troubleshooting steps in natural language form into several independently verifiable sub-steps. S52: For each sub-step, use the domain model of the aviation troubleshooting task in step S1 to automatically generate a query statement to verify the content of the sub-step, and generate a corresponding baseline answer, which will be used as a candidate answer later. S53: Based on the query statement in step S52, multiple candidate answers are obtained through diversified generation mechanisms, and these are combined with the baseline answer to form a candidate answer set; S54: Perform semantic relevance judgment on the candidate answer set and divide it into several semantic clusters based on the similarity of the candidate answer content; S55: Calculate semantic entropy based on the distribution characteristics of semantic clusters to quantify the uncertainty of sub-steps.

8. The large-scale reliable fault diagnosis method for aviation maintenance scenarios according to claim 1, characterized in that, In step S6, specifically: S61: Perform fuzzy expression recognition on the candidate answers obtained in step S5 to detect uncertain or semantic avoidance expressions contained therein, and adjust the risk level of the corresponding candidate responses or semantic distributions according to the recognition results; S62: Based on the semantic entropy in step S5 and combined with the adjustment results of the risk level after fuzzy expression recognition in step S61, generate risk scores for each sub-step, compare the risk scores with preset thresholds, and classify and label different risk levels according to the comparison results. S63: When the risk level exceeds the preset threshold, the system triggers a manual review process, outputting the steps that need to be reviewed to the maintenance engineer; the maintenance engineer confirms, revises or replaces the steps to form a reliable troubleshooting result that has been manually verified, and uses it as the final structured aviation maintenance diagnostic solution.

9. A large-scale reliable fault diagnosis system for aviation maintenance scenarios, characterized in that, The system includes: The input standardization module standardizes the input of fault descriptions; The multi-source retrieval module performs keyword retrieval and vector retrieval on a multi-source knowledge base; then, it normalizes and weights the multi-source retrieval results to generate a unified sorted list of candidate documents and selects a number of high-scoring documents. The context semantic extension module performs dynamic context extension on high-scoring documents to restore continuous semantic structure, obtain candidate evidence, and then performs semantic relevance verification on each segment of the candidate evidence to obtain a set of structured evidence. The domain model inference module collects and structures multi-source aviation maintenance data, constructs an instruction dataset, and performs domain-specific adaptation training on the pre-trained model using LoRA to obtain a domain model for aviation troubleshooting tasks. The structured evidence set is used as the input to the domain model for aviation troubleshooting tasks in S1. The domain model generates structured troubleshooting steps and automatically fills in reference information. Subsequently, the reference information is matched and verified. References that do not correspond to the structured evidence set are marked as "unverified," and a troubleshooting solution is output. The uncertainty assessment module breaks down the troubleshooting solution into several sub-steps, generates verification questions for each sub-step, and performs multiple rounds of candidate answer sampling. It constructs the semantic distribution of answers through semantic implication judgment and semantic clustering, and calculates semantic entropy based on the semantic distribution to quantify the uncertainty of the sub-steps. The risk assessment and manual review module combines fuzzy expression recognition and uncertainty classification to label the risk level of each sub-step, and automatically triggers manual review of each sub-step. After manual confirmation, revision or replacement, a reliable aviation maintenance diagnostic solution is finally formed.

Images