Civil aircraft phm model modeling method, medium and equipment based on cross-modal coupling

Abstract

The application discloses a civil aircraft PHM model modeling method based on cross-modal coupling, a medium and equipment, and belongs to the technical field of aircrafts. The method comprises the following steps: S1, multi-modal data preprocessing: for flight time series data, text data and image data of three types of modes, feature extraction and space / time / event alignment are carried out, thereby laying a foundation for subsequent fusion modeling; S2, cross-modal feature fusion and holographic data sample construction: through cross-domain fusion and dynamic optimization, a holographic data sample library covering the whole life cycle is constructed; and S3, PHM physical-knowledge fusion comprehensive modeling: a double-layer collaborative framework combining physical simulation and knowledge reasoning is constructed, and controllable fusion of physical simulation and multi-modal knowledge is realized. The application can solve the problems of the prior art, such as difficulty in multi-source heterogeneous data alignment, insufficient fusion of physical mechanism and data driving, poor model interpretability and the like.

Description

Technical Field

[0001] This invention belongs to the field of aircraft technology, specifically relating to a civil aircraft PHM modeling method, medium, and equipment based on cross-modal coupling. Background Technology

[0002] Aircraft PHM (Prognostics and Health Management) is a comprehensive system based on multidisciplinary technologies. It aims to improve flight safety, reduce operating costs, and enhance maintenance efficiency by enabling early fault prediction, health status assessment, and optimized maintenance decisions through real-time monitoring and data analysis of critical aircraft components and systems. Aircraft PHM is a key technology in aviation operations and maintenance and has become a standard feature of modern commercial airliners (such as the Boeing 787 and Airbus A350).

[0003] Currently, civil aircraft PHM systems face the following technical challenges in multimodal data processing and modeling:

[0004] 1. Issues at the data processing level:

[0005] Existing civil aircraft operation and maintenance data suffers from problems such as inconsistent spatiotemporal references and insufficient knowledge embedding, making it difficult to construct high-precision digital mirror models;

[0006] Aligning multi-source heterogeneous data such as flight data (WQAR), maintenance work orders, and fault reports across time and event dimensions is difficult.

[0007] The lack of an effective cross-modal feature extraction and fusion mechanism makes it impossible to fully utilize multi-dimensional information.

[0008] 2. Problems at the modeling method level:

[0009] Simply relying on data-driven methods makes it difficult to embed the complex physical mechanisms and engineering constraints in the civil aviation field, resulting in insufficient model interpretability;

[0010] The modeling system dominated by physical equations is not adaptable to new failure modes and cannot effectively integrate unstructured operation and maintenance knowledge;

[0011] Existing methods lack sufficient prediction accuracy and robustness when data input is incomplete. Summary of the Invention

[0012] The present invention aims to at least partially solve one of the technical problems in the aforementioned related technologies.

[0013] Therefore, the purpose of this invention is to provide a civil aircraft PHM modeling method, medium, and device based on cross-modal coupling, which can solve the problems of difficulty in aligning multi-source heterogeneous data, insufficient integration of physical mechanism and data-driven approaches, and poor model interpretability in the prior art.

[0014] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0015] This invention provides a civil aircraft PHM modeling method based on cross-modal coupling. The method achieves efficient integration of multimodal data and physical-knowledge fusion modeling through a dual-drive data processing mechanism of cross-modal coupling and holographic knowledge evolution and a dual-layer collaborative framework of fusion processing and dynamic decision-making.

[0016] In addition, the civil aircraft PHM modeling method based on cross-modal coupling according to the present invention may also have the following additional technical features:

[0017] In some embodiments, the steps of the method include:

[0018] S1. Multimodal data preprocessing: Feature extraction and spatiotemporal / event alignment are performed for three types of modalities: flight time series data, text data, and image data, laying the foundation for subsequent fusion modeling;

[0019] S2. Cross-modal feature fusion and holographic data sample construction: Through cross-domain fusion and dynamic optimization, a holographic data sample library covering the entire life cycle is constructed;

[0020] S3, PHM Physics-Knowledge Fusion Integrated Modeling: Constructing a two-layer collaborative framework that combines physical simulation and knowledge reasoning to achieve controllable fusion of physical simulation and multimodal knowledge.

[0021] In some implementations, the feature extraction in step S1 includes:

[0022] For flight time-series data: a multivariate heterogeneous long-term time-series prediction model is used to extract feature vectors of flight dynamic patterns. Anomaly scores are calculated through multi-sensor redundancy verification. Mark candidate fault anchor points;

[0023] For text data: Utilizing a domain-knowledge-enhanced large language model, unstructured maintenance work orders and fault reports are parsed into structured event chain feature vectors. Anomaly scores were calculated using KL divergence. Identify abnormal events;

[0024] For image data: Based on ResNet-50 and channel attention mechanism, local feature vectors of fault images are extracted. ; Calculate anomaly scores using L2 norm Identify the areas of significant faults.

[0025] In some implementations, the spatiotemporal alignment in step S1 includes:

[0026] Based on the improved elastic time warping model, the temporal deviation problem of multimodal data is solved by jointly optimizing the consistency of morphological similarity and physical laws.

[0027] Event alignment includes:

[0028] A six-tuple key-value mechanism driven by a composite logic chain is designed to realize the association mapping between faults and maintenance records and flight missions, ensuring multimodal characteristics. , , The precise association between )

[0029] In some implementations, the cross-modal feature fusion in step S2 includes:

[0030] Employing a cross-domain attention mechanism to integrate flight dynamics pattern feature vectors Structured event chain feature vector Local feature vectors of fault images Mapped to a unified joint embedding space, joint features containing multi-dimensional information are formed; joint anomaly scores are calculated using a confidence-based computation method to accurately determine fault anchor points.

[0031] In some implementations, the holographic data sample construction in step S2 includes:

[0032] Aircraft history dimension: Mapping multimodal data to a unified time coordinate system, identifying key nodes, constructing a continuous evolution curve of aircraft health status, and reflecting performance degradation trends;

[0033] Fault history dimension: The node features and edge weights of the fault evolution chain are optimized by graph neural network, and the alignment error is corrected in real time by closed-loop online feedback mechanism to ensure the accuracy of the fault evolution chain.

[0034] In some of these implementations, the fusion processing-dynamic decision-making two-layer collaborative framework includes:

[0035] Fusion Processing Layer: Used to achieve boundary docking between cross-modal knowledge and physical equations; includes digital twin module and knowledge enhancement module;

[0036] The digital twin module, based on homogeneous mapping, embeds multidisciplinary coupled physical equations into a neural network to obtain a physical information neural network, and then inputs real-time sensor data, control variables and attribute parameters to realize physical simulation and output physical state embedding.

[0037] The knowledge enhancement module parses text data using a pre-trained large language model and outputs domain knowledge embeddings.

[0038] Dynamic decision-making layer: The weights of the digital twin module and the knowledge enhancement module are dynamically allocated through a task-adaptive attention mechanism to generate a traceable decision logic chain; combined with a dynamic confidence evaluation mechanism, the prediction results with confidence are output.

[0039] In some of these implementations, the digital twin module dynamically updates the physical equation parameters through an adaptive correction mechanism to achieve high-fidelity physical simulation.

[0040] The updated physical equation parameters are readjusted through a constraint and release mechanism that deviates from the space. Specifically, if the physical equation deviates after the parameter update, the deviation is limited by a dynamic gating mechanism, and the physical conservation laws are verified. If the verification passes, the current parameters are retained. If the limit is exceeded, parameter rollback is triggered.

[0041] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the content of the civil aircraft PHM model modeling method based on cross-modal coupling as described in any of the preceding embodiments.

[0042] This invention also provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the content of the civil aircraft PHM model modeling method based on cross-modal coupling as described in any of the preceding embodiments.

[0043] Compared with the prior art, the present invention has at least the following beneficial effects:

[0044] In this embodiment of the invention, the civil aircraft PHM modeling method based on cross-modal coupling, through a collaborative architecture of "mechanism as the core and knowledge as the vein," achieves high-fidelity simulation of the degradation process of key civil aircraft systems and interpretable reasoning of fault modes. This framework not only solves the boundary docking problem between cross-modal knowledge and physical equations, but also reserves a controllable deviation space for new fault modes through a dynamic relaxation mechanism. Its output joint features and logic chains provide a structured input base for multi-agent collaborative training, ensuring efficient optimization and adaptive evolution of the model under complex multi-objective constraints.

[0045] In this embodiment of the invention, the civil aircraft PHM modeling method based on cross-modal coupling provided has the effect of data processing: it achieves accurate alignment of multi-source heterogeneous data through cross-modal coupling mechanism, and solves the problem of inconsistent spatiotemporal references;

[0046] In this embodiment of the invention, the civil aircraft PHM modeling method based on cross-modal coupling can improve modeling accuracy: by integrating physical mechanisms and multimodal knowledge, it can improve prediction accuracy while ensuring interpretability;

[0047] In this embodiment of the invention, the civil aircraft PHM modeling method based on cross-modal coupling provided can enhance robustness: the dynamic activation mechanism enables the model to maintain stable performance even when the data is incomplete;

[0048] In this embodiment of the invention, the civil aircraft PHM modeling method based on cross-modal coupling can improve interpretability: it generates a traceable decision logic chain and provides a transparent reasoning process.

[0049] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0050] Figure 1 This is a flowchart of the dual-drive data processing mechanism of "cross-modal coupling-holographic knowledge evolution" for the civil aircraft PHM multimodal large model, as disclosed in one embodiment of the present invention.

[0051] Figure 2 This is a framework diagram of the PHM physical-knowledge fusion integrated modeling method disclosed in one embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific examples and application scenarios.

[0054] Please see Figure 1 As shown, in some embodiments of the present invention, a civil aircraft PHM modeling method based on cross-modal coupling is provided, specifically a civil aircraft predictive health management (PHM) multimodal large model modeling method that integrates flight data, maintenance text and fault images.

[0055] In some embodiments of the present invention, a civil aircraft PHM modeling method based on cross-modal coupling is provided, employing a dual-drive data processing mechanism of "cross-modal coupling-holographic knowledge evolution" and a two-layer collaborative modeling framework of "fusion processing-dynamic decision-making". The specific technical solution is as follows.

[0056] High-quality data is the first step in establishing a multimodal large-scale PHM model for civil aircraft. This invention proposes a dual-drive data processing mechanism of "cross-modal coupling-holographic knowledge generation," aiming to construct a multi-dimensional, multi-level, and multi-source PHM data foundation. By integrating multi-source heterogeneous PHM-related data such as flight data, maintenance orders, fault reports, and spare parts circulation records, it solves the problem of aligning data in time, space, and event dimensions. Ultimately, it forms a computable holographic data sample library covering multiple dimensions such as flight, faults, and maintenance, creating a holographic dataset that can both retain the original characteristics of multimodal data and support in-depth reasoning about health status, providing a high-quality data foundation for intelligent operation and maintenance of civil aircraft.

[0057] In some embodiments of the present invention, the modeling method first preprocesses the multimodal data, then performs cross-modal feature fusion to construct holographic data samples, and finally performs PHM (Physical-Knowledge Fusion) comprehensive modeling based on the samples. These three parts are explained in detail below.

[0058] I. Multimodal data preprocessing: including feature extraction and alignment of multi-source data, multimodal spatiotemporal alignment and semantic association.

[0059] In some embodiments of this invention, for continuous time-series data (such as WQAR flight data), text data (maintenance work orders, fault reports), and image data (fault inspection records) generated during civil aircraft operation and maintenance, this invention first constructs data cleaning and transformation rules, and preprocesses and extracts features from each modality of data. For flight time-series data, a multivariate heterogeneous long-term time-series prediction model is studied to capture long-range dependencies and adaptive segmentation of flight phases, and to extract feature vectors that can characterize flight dynamic patterns. For text data, a domain-knowledge-enhanced large language model is used to parse unstructured text into structured event chain feature vectors. This enables the computational representation of the maintenance process and fault description. For image data, a feature-enhanced visual processing model is designed to extract and quantify local fault features (such as cracks, ablation, and wear) in the image, forming a fault image feature vector. After feature extraction, the data is aligned in terms of time and event dimensions. In the time dimension, a dynamic time warping method is studied to synchronously calibrate multimodal data with different sampling rates, ensuring that time-series features from different data sources are aligned in the same time coordinate system. In the event dimension, a key-value mechanism driven by a composite logic chain is studied to associate and map the maintenance records corresponding to each fault with key information such as the flight mission before the fault, realizing the alignment of different modal features (…). , , Precise alignment of data provides a unified and traceable foundation for the subsequent construction of holographic data samples. Based on the completed data alignment, the cross-domain attention mechanism is studied. , , By fusing data within the same joint embedding space, a joint feature representation containing information from various dimensions is formed. Based on this fused representation, holographic data sample sets are generated in two dimensions: aircraft history and fault history. In the aircraft history dimension, a time-series data archive covering the entire lifecycle is established for each aircraft, forming an aircraft holographic history table that records each flight mission phase, maintenance events, and key fault nodes. In the fault dimension, for common or high-maintenance-cost faults of key civil aircraft equipment, multiple fault and maintenance records from different aircraft and different time periods are integrated, and relevant multimodal data is aggregated to construct a fault holographic data subset. This holographic data sample not only comprehensively presents information from all dimensions of the civil aircraft operation and maintenance support process, but also provides standardized, high-dimensional data support for the civil aircraft PHM multimodal large model, enabling the model to use domain knowledge to assist in interpretation, real-time monitoring, and dynamic updates during high-fidelity physical simulation.

[0060] The construction of a multimodal large-scale PHM (Prognostics and Maintenance Model) for civil aircraft relies on the efficient integration and deep knowledge representation of multimodal data such as flight data, maintenance work orders, and fault images. Current civil aircraft operation and maintenance data suffers from problems such as inconsistent spatiotemporal references and insufficient knowledge embedding, making it difficult to construct a high-precision digital mirror model. This invention proposes a dual-drive data processing mechanism of "cross-modal coupling-holographic knowledge evolution," such as... Figure 1 As shown, this method integrates multi-source heterogeneous data, aiming to address the adaptive capture and depth alignment issues of "fault anchor points" in heterogeneous data, ultimately forming holographic data samples covering multiple dimensions. The specific steps for multi-source data feature extraction and alignment include:

[0061] Step 1: Flight timing data processing;

[0062] After acquiring the raw flight data, outliers and sensor fault points are filtered out, segmented, and normalized. To address the long-range dependency characteristics of the flight data, a flight phase perception mechanism and an incremental information capture mechanism are introduced, and the Transformer-based feature extraction model is optimized. The flight phase perception mechanism dynamically divides the flight into phases based on flight parameters and their second derivatives, thus reflecting the changes in each phase of the flight process. The specific expression is as follows:

[0063]

[0064]

[0065] in, For stage-aware indicators, and These are the speed and acceleration of flight, respectively. It is an additional offset item during the flight phase. This represents the incremental changes in flight data. Each time new data is input, the model updates the feature representation based on information from the previous time step, ensuring that the latest flight information is incorporated into each data processing step. Through this incremental mechanism, the model can dynamically update the representation of flight data, making the feature vector at each time step more accurately reflect changes in flight status. The feature vector extracted by the Transformer model after incremental information processing... A temporal evolution pattern characterizing flight state.

[0066] To further enhance the ability to detect anomalies and pinpoint faults in the event of data loss or sensor failure, a multi-sensor redundancy verification mechanism is introduced. When multiple sensors exist for the same physical parameter, weights are assigned based on the historical noise levels of each sensor, and their measurements are weighted and fused to obtain a fused value. And calculate the anomaly score:

[0067]

[0068] when Greater than the preset threshold If it is, then it is marked as a candidate fault anchor point.

[0069] Step 2: Repair text data processing;

[0070] The maintenance text is denoised, and initial cleaning, including word segmentation and entity recognition, is performed based on an aviation terminology database. The BERT model is then fine-tuned for domain adaptation using a civil aviation maintenance corpus. This fine-tuned BERT model is used to identify maintenance actions, timestamps, component identifiers, and fault descriptions within the text. Semantic role labeling is used to parse sentence structure, extracting "subject-action-object" triples and associating them with contextual timestamps and component identifiers.

[0071] Arrange the triples in chronological order to generate a maintenance event sequence. By using graph attention networks to model causal relationships between events, a weighted directed graph is constructed. edge weight Indicates an event arrive The causal strength. The contextual semantic embedding of the event node is obtained through BERT. Attention weights are calculated based on the importance of events to fault diagnosis. Weighted aggregation generates structured feature vectors:

[0072]

[0073] The KL divergence was calculated by comparing the distribution differences of historical normal event chains. ,when Mark abnormal events.

[0074] Step 3: Fault image data processing;

[0075] For the detection of minute defects in aerial images, a feature difference quantization method based on ResNet-50 is proposed. First, the image is preprocessed with denoising and brightness normalization to remove blurry or irrelevant images. The preprocessed image is then input into the ResNet-50 network to extract the feature map of the last convolutional layer. and ,in For the number of channels, and The spatial dimensions are used. The saliency of the fault region is quantified by calculating feature map differences and introducing a channel attention mechanism.

[0076]

[0077] Reflecting the The difference contribution of each channel is calculated. Global average pooling is used to map the difference map to fault image feature vectors. and generate anomaly scores. .when At that time, it was determined that there was a significant fault area in the image.

[0078] In some embodiments of the present invention, the content of multimodal spatiotemporal alignment and semantic association includes:

[0079] In the time dimension, there is a temporal discrepancy between WQAR data and maintenance event records, making it difficult for traditional Dynamic Time Warping (DTW) algorithms to maintain physical consistency. This invention proposes an improved Flexible Time Warping (IDTW) model, whose objective function is:

[0080]

[0081] in, The KL divergence represents the local time series statistics. The weights are used to constrain physical consistency. This method achieves precise time alignment of multimodal data by jointly optimizing morphological similarity and physical consistency. By matching the morphological characteristics of the pressure curve with the statistical distribution characteristics of maintenance events, the two are aligned to the same time window.

[0082] At the event dimension, this invention uses a composite key-value mechanism as the semantic association dependency for multi-source data. A six-tuple key space is defined. ,in The tail number of the aircraft. For the event time window, For fault ATA coding, For component serial number, For the flight phase, The fault level is determined by a multi-dimensional index table constructed using six-tuple keys. This maps data from different modalities to a unified logical framework, enabling semantic association and rapid source tracing of multi-source data. To improve retrieval efficiency, a dynamic priority queue is designed to automatically adjust index weights based on historical query frequency, ensuring fast response times in high-concurrency scenarios.

[0083] II. Cross-modal feature fusion and holographic data sample construction

[0084] Cross-modal feature fusion: To achieve deep fusion of multimodal features, a cross-domain attention mechanism is used to construct a joint embedding space, which integrates multi-dimensional information into the same feature space to generate a joint feature representation that can comprehensively reflect the health status of the aircraft.

[0085] To fully leverage the advantages of each modality, a cross-modal assisted inference mechanism is introduced to fuse anomaly scores from each modality. Anomaly scores for flight data, maintenance text, and images are as follows: , and . The confidence coefficients for each modality are obtained through training on historical data. A joint decision formula is used:

[0086]

[0087] By achieving cross-modal information complementarity, and ensuring that the existence of fault anchor points is confirmed when the joint anomaly score exceeds the threshold, effective alignment of semantic and temporal information of different modalities can be achieved.

[0088] In some embodiments of the present invention, holographic data sample construction involves integrating multi-source data to construct a full lifecycle archive for aircraft health management and fault prediction, using time warping technology to map various types of information to a unified time coordinate system, identifying key event nodes, and constructing a continuous evolution curve of the aircraft's health status to reflect the performance degradation trend of the aircraft during long-term service.

[0089] Based on the initially constructed fault evolution chain, a graph neural network (GNN) is used to iteratively fuse the information of each node in the chain and optimize the edge weights. The node update formula is as follows:

[0090]

[0091] in, For nodes In the The feature vector of the layer, For nodes The neighborhood set. Simultaneously, a closed-loop online feedback mechanism is designed to update model parameters by comparing real-time collected data with offline historical data.

[0092]

[0093] in, For time Model parameters at time, For learning rate, Represents the loss function Regarding joint features With real labels The gradient. This mechanism helps to continuously correct alignment errors during system operation, ensuring the accuracy and robustness of the fault evolution chain.

[0094] III. Comprehensive Modeling Method for Civil Aircraft PHM Multimodal Large Model

[0095] After the multimodal holographic data sample library is constructed, PHM (Physical-Knowledge Fusion Modeling) will be performed. To address the difficulties in integrating physical models with engineering knowledge information and the poor interpretability of black-box models in PHM modeling of key civil aircraft equipment, this invention researches and proposes a fusion modeling method based on "mechanism as the core, knowledge as the framework, and data as the application," and proposes a two-layer collaborative framework of "fusion processing-dynamic decision-making." Through deep coupling of digital twin modules and knowledge enhancement modules, controllable fusion of physical laws and multimodal knowledge is achieved. This invention is a layered fusion architecture of Large Language Model (LLM) and Digital Twin (DT), effectively integrating physical mechanisms, real-world operational data, engineering technical information, and human experience knowledge. It overcomes the limitations of LLM in handling numerical calculations and DT in handling multi-source data, constructing an integrated multimodal large-scale civil aircraft PHM model that possesses both high-fidelity physical simulation capabilities and rich domain knowledge. This model makes full use of multimodal data from the PHM (Prognostics and Health Management) of civil aircraft. It can not only accurately simulate the dynamic changes in the health status of the equipment, but also automatically interpret the output results of the model by combining domain knowledge, providing comprehensive, intelligent and interpretable support for equipment status monitoring, fault diagnosis and remaining life prediction.

[0096] In some embodiments of the present invention, PHM physical-knowledge fusion integrated modeling includes two parts: a two-layer collaborative framework of "fusion processing-dynamic decision-making" and a two-layer collaborative working mechanism.

[0097] The core of the research on the PHM physical-knowledge fusion integrated modeling method lies in constructing a two-layer collaborative model framework of "fusion processing-dynamic decision-making," such as... Figure 2As shown, this framework aims to solve complex problems in PHM (Prognostics and Health Management) modeling of civil aircraft through deep integration of physical mechanisms and multimodal knowledge. The first stage of the framework is the fusion processing layer, composed of a Digital Twin (DT) module and a knowledge enhancement module. These two modules complement each other through cross-modal fusion technology to achieve physical simulation and domain knowledge complementarity. The DT module is based on Physical Information Neural Network (PINN) and establishes a simulation model of multidisciplinary coupled physical equations using a homogeneous mapping method. Inputs include real-time sensor data (such as temperature and vibration), control variables (such as torque and speed), and attribute parameters (such as material properties). The model embeds multidisciplinary physical equations (such as partial differential equations in mechanics, thermodynamics, and fluid dynamics) into the neural network using differentiable simulation technology, dynamically correcting parameters to achieve high-fidelity state prediction. For example, for the crack propagation problem in aero-engines, the model dynamically adjusts the crack propagation rate prediction through the interaction of thermodynamic equations and real-time temperature data. The knowledge enhancement module, with LLM as its core, processes unstructured data such as aerospace engineering documents (such as maintenance logs, fault reports, and design specifications) and CAD models. By leveraging a domain knowledge ontology, LLM transforms textual descriptions of fault modes (such as "high temperature causes bearing wear") into quantifiable boundary conditions (such as a temperature threshold of 120°C) and aligns them across modalities with image features (such as crack morphology) to form semantic embeddings. For example, when a maintenance log mentions "vibration frequency exceeding 2kHz indicates bearing failure," LLM maps this description to quantized parameters of spectral features, providing supplementary input to the DT module.

[0098] The output of the fusion processing layer enters the dynamic decision layer, which achieves synergy between physical simulation and knowledge reasoning through a task-adaptive attention mechanism. First, the joint latent representation (fusing physical state embedding and knowledge semantic embedding) is input into the multimodal transformer (LLM), which dynamically allocates the weights of the two modules using self-attention and cross-attention mechanisms. For example, in a fault diagnosis task, if sensor data is complete but textual information is missing, the model increases the weight of the DT module, relying on the physical simulation results; if reasoning requires combining historical cases, it enhances the contribution of the LLM. The decision layer also generates a traceable logical chain, recording the basis for weight allocation and the reasoning path. For example, when the model diagnoses "bearing wear," the logical chain will indicate that the conclusion is based on the physical simulation of the vibration spectrum (weight 60%) and the semantic matching of similar cases in the maintenance log (weight 40%). Furthermore, a dynamic confidence assessment mechanism quantifies the credibility of the physical and knowledge results. When the input data contains noise or is missing, the model outputs multiple confidence results (e.g., "remaining life predicted is 200 hours, confidence level 85%)" for maintenance personnel to refer to. This mechanism is particularly important when dealing with extreme conditions such as polar temperatures. The model can call on the "low-temperature material embrittlement" case in the knowledge base to dynamically correct the physical equations and evaluate the confidence level of the correction, ensuring the reliability and interpretability of the decision.

[0099] A two-layer collaborative working mechanism: The core objective is to address the incompleteness of data input through dynamic activation and inference mechanisms, while balancing physical rigor and fault adaptability through the constraint and release of deviation space. First, the dynamic activation and inference mechanism adjusts the activation state of modules in real time based on the completeness of the input data. When sensor data is complete but engineering documentation is missing, the digital twin module leads the prediction, and the knowledge enhancement module fills in the missing information using a domain knowledge ontology. For example, if vibration data is abnormal but maintenance logs are unavailable, the model automatically calls upon typical features of "bearing wear" from the knowledge base to generate virtual boundary conditions and inject them into the physical equations. Conversely, if sensor data is partially missing (e.g., a temperature sensor malfunction), the knowledge enhancement module uses "high-temperature alarm records" from text logs and structural parameters from the CAD model to infer possible temperature distributions and drive the simulation. When both types of data are incomplete, the dynamic weight allocation mechanism balances the two modules in combination with task requirements: for example, in remaining life prediction, if there is only some vibration data and scattered maintenance records, the model allocates 60% of the weight to the physical simulation of DT (based on known vibration modes) and 40% to the historical fault reasoning of LLM (based on similar working condition cases) through attention routing, and uses embedded knowledge to fill in the missing data (such as estimating the temperature effect through material fatigue curves).

[0100] To realize a two-layer collaborative model framework and a two-layer collaborative working mechanism for integrated processing and dynamic decision-making, this invention proposes the following three innovations: a controllable coupling mechanism based on physical mechanisms and enhanced by multimodal document knowledge (troubleshooting manuals, graphic data, expert rules, etc.) is constructed. This mechanism allows the digital twin to accurately distinguish between new fault traces and random noise while reserving deviation space.

[0101] Innovation Point 1: "Boundary Connection" between Cross-Modal Knowledge and Physical Equations

[0102] One of the core challenges of the fusion processing layer is achieving the "boundary docking" between cross-modal knowledge and physical equations. This process begins with multimodal data preprocessing: sensor data is encoded into physical state embeddings using PINN, and text and image data have semantic features extracted using LLM and CNN, respectively. These features are then aligned to a unified semantic space through contrastive learning or deep canonical correlation analysis (DCCA). For example, "abnormal vibration amplitude" in the text needs to be associated with a specific frequency band of the sensor spectrum. Subsequently, a knowledge retrieval and matching mechanism is activated: when the sensor detects an abnormal temperature surge, the model dynamically retrieves similar fault patterns (such as "high temperature accelerates material fatigue") based on a knowledge graph (integrating maintenance manuals and historical fault cases), and extracts quantified parameters (such as a crack propagation rate coefficient of 0.05) through semantic parsing of LLM. These parameters are dynamically injected into the physical equations, for example, by adding a temperature-sensitive term to the crack propagation equation, or by adjusting the piecewise constraints of the thermodynamic equations through differentiable programming. To ensure the rationality of the corrections, a physical consistency verification mechanism checks core laws such as energy conservation in real time. If the corrections cause errors to exceed limits (such as mass conservation error > 5%), parameter rollback is triggered. Meanwhile, multimodal cross-validation compares the matching degree between simulation results and image features (such as temperature distribution in infrared thermal images), and finally optimizes the knowledge base and alignment strategy through feedback.

[0103] Innovation Point 2: Adaptive Correction of Multimodal Large Models

[0104] The second core challenge of the fusion processing layer is achieving adaptive correction of the multimodal large model. Physical simulation results are generated through a digital twin module, compared with sensor data to calculate residual vectors, and combined with text logs, image, and audio features. A multimodal transformer is used to generate a comprehensive anomaly score, dynamically determining anomaly thresholds and identifying extreme conditions (such as polar cryogenic-high load combinations). Once an anomaly signal is detected, the model initiates knowledge-driven fault inference. Anomaly features are transformed into structured queries using a large language model. Similar fault modes and associated physical laws (such as material embrittlement due to low temperatures) are retrieved from a knowledge graph constructed by integrating maintenance manuals and material failure databases. High-confidence candidates are then selected based on multimodal similarity assessment. Subsequently, the model dynamically corrects the physical equations. For example, retrieved material parameter changes (such as decreased yield strength) are injected into the constitutive equations as differentiable correction terms, or reasonable extension terms are generated through symbolic regression when existing equations cannot explain the anomalies. For extreme conditions (such as icing conditions), additional constraints (such as lift coefficient correction tables) from the knowledge base are invoked and injected into the aerodynamic equations. After correction, multi-level verification is performed to check the consistency of energy conservation indices with cross-modal data (such as alignment between simulated temperature field and infrared thermogram). If the error exceeds the limit, parameter rollback is triggered. After the validity of the correction is determined by confidence score, the corrected parameters that pass the verification are stored in the dynamic knowledge base, and the correction strategy is optimized based on reinforcement learning. At the same time, adversarial data is used to enhance virtual testing to identify potential defects, forming a closed-loop knowledge iteration and model evolution.

[0105] Innovation Point 3: Deviating from the Constraints and Release of Space

[0106] The core innovation of the dual-layer collaborative working mechanism lies in the constraint and release of deviation space, allowing the model to temporarily adapt to unknown faults within a strict physical framework. First, real-time deviation detection is performed by comparing the physical simulation results generated by the digital twin module with sensor data to calculate residuals (such as differences in vibration amplitude). When the residuals continuously exceed limits (such as a 20% deviation from the predicted value), a deviation channel is triggered. Subsequently, the model retrieves historical cases from the knowledge graph (such as the crack length-frequency relationship associated with "turbine blade cracks"), extracts evolutionary features to constrain the deviation direction, and adjusts the deviation magnitude through a dynamic gating mechanism: based on Bayesian inference, the confidence level of the current correction is calculated (such as 70%), and combined with case similarity (such as a 90% match with historical low-temperature embrittlement cases), the maximum allowable deviation is limited (such as a material strength reduction ≤ 25%). For example, when an abnormal material yielding is detected at extreme low temperatures, the model injects a temperature-sensitive correction term into the physical equations and adjusts the range according to the manual's safety threshold. After correction, physical conservation laws are verified (e.g., energy balance error <3%). If successful, the correction is accepted and the knowledge base is updated; if the error exceeds the limit, the model is rolled back to the original equation and a diagnostic report is generated. In the closed-loop feedback, deviation operation data and correction results are stored in a structured case library for reinforcement learning to optimize deviation trigger thresholds and retrieval strategies. Simultaneously, adversarial data (e.g., -60℃ + turbulent disturbances) are used to test the model's response capabilities and identify potential defects (e.g., unmodeled material phase transitions). The knowledge graph is dynamically expanded with new cases (e.g., "icing condition - lift correction"), forming incremental knowledge evolution. This ensures that all corrections are completed under the "dual anchoring" of physical laws and engineering experience, enabling the model to adapt to and controllably evolve in response to new faults.

[0107] Example 1: Flight Data Processing

[0108] Step 1: Data Preprocessing

[0109] After acquiring the raw WQAR flight data, outlier filtering, segmentation, and normalization are performed. A marking mechanism is established for sensor fault points.

[0110] Step 2: Extraction of sensory features during flight phase

[0111] Employing a flight phase perception mechanism, calculate phase perception indicators. ,in and These are flight speed and acceleration, respectively. Add an offset term for the flight phase.

[0112] Step 3: Incremental Information Capture

[0113] Through incremental mechanism Update the feature representation so that the feature vector at each time step accurately reflects the changes in flight state.

[0114] Step 4: Multi-sensor redundancy verification

[0115] When multiple sensors exist for the same physical parameter, calculate the fusion value. and abnormal scoring ,when Greater than the preset threshold The time marker is used as a candidate fault anchor point.

[0116] Example 2: Cross-modal fusion modeling

[0117] Step 1: Cross-modal feature fusion

[0118] A joint embedding space is constructed using a cross-domain attention mechanism to integrate flight data features. Text features and image features They are integrated into a unified feature space.

[0119] Step 2: Calculation of Joint Anomaly Score

[0120] Through the joint determination formula Calculate the cross-modal anomaly score, where These are the confidence coefficients for each modality.

[0121] Step 3: Activate the digital twin module

[0122] A multidisciplinary coupled physical equation simulation model is established based on PINN, with real-time sensor data, control variables and attribute parameters as input, and the parameters are dynamically corrected through differentiable simulation technology.

[0123] Step 4: Collaboration of Knowledge Enhancement Modules

[0124] LLM processes text data such as maintenance logs and fault reports, and transforms text descriptions into quantifiable boundary conditions through a domain knowledge ontology, providing semantic enhancement support for digital twin modules.

[0125] Step 5: Dynamic Decision Generation

[0126] The dynamic decision layer integrates physical simulation and knowledge reasoning results through a task-adaptive attention mechanism, adjusts modal weights according to PHM task requirements, and generates a traceable decision logic chain.

[0127] Example 3: Deviation from Spatial Constraints and Release Mechanism

[0128] Step 1: Real-time Deviation Detection

[0129] The residual is calculated by comparing the simulation results of the digital twin module with the sensor data. When the residual continues to exceed the limit, the deviation channel is triggered.

[0130] Step 2: Knowledge Constraints Deviate from Direction

[0131] Historical cases are retrieved from the knowledge graph, evolutionary features are extracted to constrain the direction of deviation, and the deviation magnitude is controlled through a dynamic gating mechanism.

[0132] Step 3: Verification of physical conservation laws

[0133] After correction, the physical conservation laws such as energy balance are verified. If the error exceeds the limit, the equation is rolled back to the original equation and a diagnostic report is generated.

[0134] Step 4: Closed-loop feedback optimization

[0135] Deviation operation data and correction results are stored in a case library for reinforcement learning to optimize deviation trigger thresholds and retrieval strategies.

[0136] This invention, through a fusion modeling concept of "mechanism as the core, knowledge as the framework, and data as the application," achieves efficient processing and intelligent modeling of multimodal PHM data for civil aircraft, providing reliable technical support for safe civil aviation operations. This method not only solves the technical challenge of fusing multi-source heterogeneous data but also ensures the interpretability and engineering applicability of the model through a physical-knowledge collaborative mechanism.

[0137] Any part of this invention not described in detail can be referred to in the prior art or in the art known to those skilled in the art. This embodiment does not limit such part and will not describe it in detail here.

[0138] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A modeling method for civil aircraft PHM model based on cross-modal coupling, characterized in that, The method achieves efficient integration of multimodal data and physical-knowledge fusion modeling through a cross-modal coupling-holographic knowledge evolution dual-driven data processing mechanism and a fusion processing-dynamic decision-making dual-layer collaborative framework; The steps of the method include: S1. Multimodal data preprocessing: Feature extraction and spatiotemporal / event alignment are performed for three types of modalities: flight time series data, text data, and image data, laying the foundation for subsequent fusion modeling; S2. Cross-modal feature fusion and holographic data sample construction: Through cross-domain fusion and dynamic optimization, a holographic data sample library covering the entire life cycle is constructed; S3, PHM Physics-Knowledge Fusion Integrated Modeling: Constructing a two-layer collaborative framework that combines physical simulation and knowledge reasoning to achieve controllable fusion of physical simulation and multimodal knowledge; The fusion processing-dynamic decision-making two-layer collaborative framework includes: Fusion Processing Layer: Used to achieve boundary docking between cross-modal knowledge and physical equations; includes digital twin module and knowledge enhancement module; The digital twin module, based on homogeneous mapping, embeds multidisciplinary coupled physical equations into a neural network to obtain a physical information neural network, and then inputs real-time sensor data, control variables and attribute parameters to realize physical simulation and output physical state embedding. The knowledge enhancement module parses text data using a pre-trained large language model and outputs domain knowledge embeddings. Dynamic decision-making layer: The weights of the digital twin module and the knowledge enhancement module are dynamically allocated through a task-adaptive attention mechanism to generate a traceable decision logic chain; combined with a dynamic confidence evaluation mechanism, the prediction results with confidence are output.

2. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 1, characterized in that, The feature extraction in step S1 includes: For flight time-series data: a multivariate heterogeneous long-term time-series prediction model is used to extract feature vectors of flight dynamic patterns. V Q Anomaly scores are calculated through multi-sensor redundancy verification. S Q Mark candidate fault anchor points; For text data: Utilizing a domain-knowledge-enhanced large language model, unstructured maintenance work orders and fault reports are parsed into structured event chain feature vectors. V T Anomaly scores were calculated using KL divergence. S T Identify abnormal events; For image data: Based on ResNet-50 and channel attention mechanism, local feature vectors of fault images are extracted. V I ; Calculate anomaly scores using L2 norm S I Identify the areas of significant faults.

3. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 2, characterized in that, The spatiotemporal alignment in step S1 includes: Based on the improved elastic time warping model, the temporal deviation problem of multimodal data is solved by jointly optimizing the consistency of morphological similarity and physical laws. Event alignment includes: A six-tuple key-value mechanism driven by a composite logic chain is designed to realize the association mapping between faults and maintenance records and flight missions, ensuring multimodal characteristics. V Q , V T , V I The precise association between ) 4. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 1, characterized in that, The cross-modal feature fusion in step S2 includes: Employing a cross-domain attention mechanism to integrate flight dynamics pattern feature vectors V Q Structured event chain feature vector V T Local feature vectors of fault images V I Mapped to a unified joint embedding space, joint features containing multi-dimensional information are formed; joint anomaly scores are calculated using a confidence-based computation method to accurately determine fault anchor points.

5. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 1, characterized in that, The content of constructing the holographic data sample in step S2 includes: Aircraft history dimension: Mapping multimodal data to a unified time coordinate system, identifying key nodes, constructing a continuous evolution curve of aircraft health status, and reflecting performance degradation trends; Fault history dimension: The node features and edge weights of the fault evolution chain are optimized by graph neural network, and the alignment error is corrected in real time by closed-loop online feedback mechanism to ensure the accuracy of the fault evolution chain.

6. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 1, characterized in that, The digital twin module dynamically updates the physical equation parameters through an adaptive correction mechanism to achieve high-fidelity physical simulation. The updated physical equation parameters are readjusted through a constraint and release mechanism that deviates from the space. Specifically, if the physical equation deviates after the parameter update, the deviation is limited by a dynamic gating mechanism, and the physical conservation laws are verified. If the verification passes, the current parameters are retained. If the limit is exceeded, parameter rollback is triggered.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the content of the civil aircraft PHM model modeling method based on cross-modal coupling as described in any one of claims 1-6.

8. A computer device, comprising: The memory, the processor, and the computer program stored in the memory and executable on the processor are characterized in that the processor executes the computer program to implement the content of the civil aircraft PHM model modeling method based on cross-modal coupling as described in any one of claims 1-6.

Images