Civil aircraft PHM model modeling method based on cross-modal coupling, medium and equipment

Abstract

The invention discloses a civil aircraft PHM modeling method based on cross-modal coupling, a medium and equipment, and belongs to the technical field of aircrafts. The method comprises the steps of S1, multi-modal data preprocessing: performing feature extraction and time-space / event alignment for three modalities of flight time sequence data, text data and image data, and laying a foundation for subsequent fusion modeling; s2, cross-modal feature fusion and holographic data sample construction: constructing a holographic data sample library covering the whole life cycle through cross-domain fusion and dynamic optimization; and S3, PHM physical-knowledge fusion comprehensive modeling: constructing a double-layer collaborative framework combining physical simulation and knowledge reasoning, and realizing controllable fusion of physical simulation and multi-modal knowledge. The problems that in the prior art, multi-source heterogeneous data alignment is difficult, physical mechanism and data driving fusion is insufficient, and model interpretability is poor can be solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of aircraft, and particularly relates to a civil aircraft PHM model modeling method based on cross-modal coupling, a medium and equipment. BACKGROUND

[0002] Aircraft PHM (Prognostics and Health Management) is a comprehensive system based on multidisciplinary technology, aiming to realize early prediction of faults, health state evaluation and optimization of maintenance decision through real-time monitoring and data analysis of key components and systems of the aircraft, so as to improve flight safety, reduce operating costs and improve maintenance efficiency. Aircraft PHM is a key technology in the field of aviation maintenance, and has become a standard function of modern civil aircraft (such as Boeing 787 and Airbus A350).

[0003] Currently, the civil aircraft PHM system has the following technical problems in multi-modal data processing and modeling:

[0004] 1. Problems at the data processing level:

[0005] The existing civil aircraft maintenance data has problems such as inconsistent time and space reference and insufficient knowledge embedding, making it difficult to build a high-precision digital mirror model;

[0006] Flight data (WQAR), maintenance work orders, fault reports and other multi-source heterogeneous data are difficult to align in time and event dimensions;

[0007] There is a lack of effective cross-modal feature extraction and fusion mechanism, and multi-dimensional information cannot be fully utilized.

[0008] 2. Problems at the modeling method level:

[0009] Simply relying on data-driven methods cannot embed the complex physical mechanisms and engineering constraints in the civil aviation field, resulting in insufficient model interpretability;

[0010] The modeling system dominated by physical equations lacks adaptability to new fault modes and cannot effectively integrate unstructured maintenance knowledge;

[0011] The existing methods have insufficient prediction accuracy and robustness under incomplete data input. SUMMARY

[0012] The present application aims to at least partially solve one of the above technical problems in the related art.

[0013] To this end, the purpose of the present application is to provide a civil aircraft PHM model modeling method, medium and equipment based on cross-modal coupling, which can solve the problems of difficulty in aligning multi-source heterogeneous data, insufficient fusion of physical mechanism and data-driven, poor model interpretability and the like in the prior art.

[0014] To solve the above technical problems, the present application is implemented as follows:

[0015] The embodiment of the present application provides a civil aircraft PHM model modeling method based on cross-modal coupling, which realizes efficient integration and physical-knowledge fusion modeling of multi-modal data through a cross-modal coupling-holographic knowledge evolution dual-driven data processing mechanism and a fusion processing-dynamic decision-making double-layer collaborative framework.

[0016] In addition, the civil aircraft PHM model modeling method based on cross-modal coupling according to the present application can also have the following additional technical features:

[0017] In some embodiments thereof, the steps of the method include:

[0018] S1, multi-modal data preprocessing: for flight time series data, text data and image data of three types of modalities, feature extraction and space-time / event alignment are performed, laying a foundation for subsequent fusion modeling;

[0019] S2, cross-modal feature fusion and holographic data sample construction: through cross-domain fusion and dynamic optimization, a holographic data sample library covering the whole life cycle is constructed;

[0020] S3, PHM physical-knowledge fusion comprehensive modeling: a double-layer collaborative framework combining physical simulation and knowledge reasoning is constructed, realizing controllable fusion of physical simulation and multi-modal knowledge.

[0021] In some embodiments thereof, the content of feature extraction in step S1 includes:

[0022] For flight time series data: a multivariate heterogeneous long time series prediction model is used to extract flight dynamic mode feature vectors ; abnormal scores are calculated through multi-sensor redundancy verification , and candidate fault anchor points are marked;

[0023] For text data: a large language model enhanced by domain knowledge is used to parse unstructured maintenance work orders and fault reports into structured event chain feature vectors ; abnormal scores are calculated through KL divergence , and abnormal events are identified;

[0024] For image data: based on ResNet-50 and channel attention mechanism, local feature vectors of fault images are extracted ; Abnormal score is calculated by L2 norm , and a significant failure region is determined.

[0025] In some embodiments, the content of the spatio-temporal alignment in step S1 includes:

[0026] Based on the improved elastic time warping model, the time sequence deviation problem of multi-modal data is solved by jointly optimizing the morphological similarity and the consistency of physical laws.

[0027] The content of the event alignment includes:

[0028] A six-tuple key-value mechanism driven by composite logic chains is designed to realize the association mapping of failures and maintenance records and flight tasks, and ensure the accurate association of multi-modal features. , , ).

[0029] In some embodiments, the content of the cross-modal feature fusion in step S2 includes:

[0030] The cross-domain attention mechanism is used to map the flight dynamic mode feature vector , the structured event chain feature vector , and the local feature vector of the fault image to a unified joint embedding space to form a joint feature containing multi-dimensional information; and the joint abnormal score is calculated based on the confidence calculation method to accurately determine the fault anchor point.

[0031] In some embodiments, the content of the holographic data sample construction in step S2 includes:

[0032] Aircraft history dimension: mapping multi-modal data to a unified time coordinate system, identifying key nodes, constructing a continuous evolution curve of aircraft health state, and reflecting performance degradation trend;

[0033] Fault history dimension: optimizing the node features and edge weights of the fault evolution chain through graph neural network, and combining the closed-loop online feedback mechanism to correct the alignment error in real time, ensuring the accuracy of the fault evolution chain.

[0034] In some embodiments, the fusion processing-dynamic decision-making two-level collaborative framework includes:

[0035] Fusion processing layer: used for realizing the boundary docking of cross-modal knowledge and physical equations; including a digital twin module and a knowledge enhancement module;

[0036] The digital twin module, based on homology mapping, embeds multi-disciplinary coupled physical equations into a neural network to obtain a physical information neural network, and then inputs real-time sensor data, control variables and attribute parameters to realize physical simulation and output physical state embedding.

[0037] The knowledge enhancement module: analyzing the text data through a pre-trained large language model, and outputting field knowledge embedding;

[0038] The dynamic decision layer: dynamically allocating the weights of the digital twin module and the knowledge enhancement module through a task adaptive attention mechanism, generating a traceable decision logic chain; combining a dynamic confidence evaluation mechanism, outputting a prediction result with confidence.

[0039] In some embodiments, the physical equation parameters in the digital twin module are dynamically updated through an adaptive correction mechanism to achieve high-fidelity physical simulation;

[0040] The updated physical equation parameters are readjusted through the constraint and release mechanism of the deviation space, specifically: if the physical equation after updating the parameters deviates, the deviation amplitude is limited through a dynamic gating mechanism, and the law of conservation of physics is verified; if it is verified, the current parameters are retained; if it is out of limit, the parameter rollback is triggered.

[0041] The embodiment of the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the content of the civil aircraft PHM model modeling method based on cross-modal coupling according to any one of the above.

[0042] The embodiment of the application also provides a computer device, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to realize the content of the civil aircraft PHM model modeling method based on cross-modal coupling according to any one of the above.

[0043] Compared with the prior art, the application has at least the following beneficial effects:

[0044] In the embodiment of the application, the civil aircraft PHM model modeling method based on cross-modal coupling realizes high-fidelity simulation of the degradation process of the key system of the civil aircraft and explainable reasoning of the fault mode through the collaborative architecture of "mechanism as the core and knowledge as the pulse"; the framework not only solves the boundary docking problem of cross-modal knowledge and physical equation, but also reserves a controllable deviation space for new fault modes through a dynamic relaxation mechanism; the output joint features and logic chain provide a structured input base for multi-agent collaborative training, ensuring efficient optimization and adaptive evolution of the model under complex multi-objective constraints;

[0045] In the embodiment of the application, the civil aircraft PHM model modeling method based on cross-modal coupling has the effect of data processing: precise alignment of multi-source heterogeneous data is realized through a cross-modal coupling mechanism, and the problem of inconsistent time and space benchmarks is solved;

[0046] The civil aircraft PHM model modeling method based on cross-modal coupling provided in the embodiment of the application can improve modeling accuracy: fusing physical mechanism and multi-modal knowledge, while ensuring explainability, the prediction accuracy is improved;

[0047] The civil aircraft PHM model modeling method based on cross-modal coupling provided in the embodiment of the application can enhance robustness: the dynamic activation mechanism enables the model to maintain stable performance under incomplete data;

[0048] The civil aircraft PHM model modeling method based on cross-modal coupling provided in the embodiment of the application can improve explainability: a traceable decision logic chain is generated, and a transparent reasoning process is provided.

[0049] Additional aspects and advantages of the application will be described in the following description, some of which will become apparent from the following description, or will be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The civil aircraft PHM multi-modal large model "cross-modal coupling-holographic knowledge evolution" double-drive type data processing mechanism flowchart disclosed for an embodiment of the application;

[0051] Figure 2 The PHM physical-knowledge fusion comprehensive modeling method framework disclosed for an embodiment of the application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the application will be described in detail below with reference to the drawings of the embodiments of the application. Obviously, the described embodiments are some of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0053] The embodiments of the application will be described in detail below with reference to the drawings and specific embodiments and application scenarios.

[0054] Please refer to Figure 1 In some embodiments of the application, a civil aircraft PHM model modeling method based on cross-modal coupling is provided, specifically a civil aircraft predictive health management (PHM) multi-modal large model modeling method fusing flight data, maintenance text and fault images.

[0055] In some embodiments of the application, a civil aircraft PHM model modeling method based on cross-modal coupling adopts a "cross-modal coupling-holographic knowledge evolution" double-drive type data processing mechanism and a "fusion processing-dynamic decision" double-layer collaborative modeling framework. The specific technical solutions are as follows.

[0056] High-quality data is the first step to build a multi-modal large model of civil aircraft PHM. The application proposes a "cross-modal coupling-holographic knowledge generation" dual-drive data processing mechanism, aiming to build a multi-dimensional, multi-level and multi-source PHM data base. By integrating flight data, maintenance records, fault reports and spare parts turnover records and other PHM related multi-source heterogeneous data, the problem of data space-time and event dimension alignment is solved, and finally a computable holographic data sample library covering multiple dimensions such as flight, fault and maintenance is formed, which can not only retain the original characteristics of multi-modal data, but also support deep reasoning of health status, and provides a high-quality data basis for civil aircraft intelligent operation and maintenance.

[0057] In some embodiments of the application, the modeling method first preprocesses multi-modal data, then performs cross-modal feature fusion to construct holographic data samples, and finally performs PHM physical-knowledge fusion comprehensive modeling based on the samples. The following describes the three parts one by one.

[0058] I. Multi-modal data preprocessing: including multi-source data feature extraction and alignment, multi-modal space-time alignment and semantic association.

[0059] In some embodiments of the application, for the continuous time series data (such as WQAR flight data) generated in the civil aircraft operation process, text data (maintenance work order, fault report) and image data (fault inspection record), the application first constructs data cleaning and conversion rules to preprocess and extract features from each modal data. For flight time series data, a multi-variable heterogeneous long time series prediction model is studied to capture long-range dependence and flight stage adaptive segmentation, and a feature vector capable of representing flight dynamic mode is extracted ; for text data, a large language model enhanced by domain knowledge is used to parse unstructured text into structured event chain feature vector , realizing the computable representation of maintenance process and fault description; for image data, a feature enhanced visual processing model is designed to extract and quantify local fault features (such as cracks, ablation, wear) in the image, forming a fault image feature vector . After feature extraction, the data is aligned in time and event dimensions. In the time dimension, a dynamic time warping method is studied to synchronize and calibrate multi-modal data with different sampling rates, so that time series features from different data sources are aligned in the same time coordinate system. In the event dimension, a composite logic chain driven key-value mechanism is studied to associate and map each fault's maintenance record with the key information before the fault, such as flight tasks, to realize the accurate alignment of different modal features , , ), providing unified and traceable basic data for subsequent holographic data sample construction. Based on the completion of data alignment, a cross-domain attention mechanism is studied to , , In the same joint embedding space, fusion is formed to form a joint feature representation containing dimension information. Based on the fusion representation, a holographic data sample set of two dimensions of aircraft history and fault history is generated: in the aircraft history dimension, a time series data archive covering the whole life cycle is established for each aircraft, forming an aircraft holographic history table, recording each flight task stage, maintenance event and key fault node; in the fault dimension, for the common or high maintenance cost faults of civil aircraft key equipment, multiple fault and maintenance records of different aircraft and different time periods are integrated, related multi-modal data is aggregated, and a fault holographic data subset is constructed. The holographic data sample not only comprehensively presents the dimension information of the civil aircraft operation and maintenance process, but also provides standardized and high-dimensional data support for the civil aircraft PHM multi-modal large model, so that the model can assist in interpretation, real-time monitoring and dynamic updating with the help of domain knowledge when performing high-fidelity physical simulation.

[0060] The construction of the civil aircraft PHM multi-modal large model depends on the efficient integration and deep knowledge expression of multi-modal data such as flight data, maintenance work orders and fault images. The current civil aircraft operation and maintenance data has problems such as inconsistent time and space benchmarks and insufficient knowledge embedding, making it difficult to construct a high-precision digital mirror model. The present application proposes a "cross-modal coupling-holographic knowledge evolution" dual-drive data processing mechanism, as shown in Figure 1 , which integrates multi-source heterogeneous data, aiming to adaptively capture and deeply align the "fault anchor points" in heterogeneous data, and finally form holographic data samples covering multiple dimensions. The specific steps of multi-source data feature extraction and alignment include:

[0061] Step 1, flight time series data processing;

[0062] After obtaining the original flight data, abnormal values and sensor fault points are filtered, segmented and normalized. For the long-range dependence characteristics of flight data, flight phase perception mechanism and incremental information capture mechanism are introduced to optimize the feature extraction model based on Transformer. The flight phase perception mechanism dynamically divides the flight phase based on flight parameters and their second derivatives, reflecting the changes in each phase of the flight process. The specific expression is as follows,

[0063]

[0064]

[0065] wherein, is the phase perception indicator, and are the speed and acceleration of the flight, is the additional offset item of the flight phase, The incremental change of flight data is represented. Each time new data is input, the model updates the feature representation according to the information of the previous moment, so that the processing of each step of data can introduce the latest flight information. Through the incremental mechanism, the model can dynamically update the representation of flight data, so that the feature vector at each moment more accurately reflects the changes of the flight state. The feature vector extracted by the incremental information processing Transformer model characterizes the time evolution pattern of the flight state.

[0066] To further enhance the ability of anomaly detection and fault anchor point determination in the case of data loss or sensor failure, a multi-sensor redundancy checking mechanism is introduced. When there are multiple sensors for the same physical parameter, weights are assigned according to the historical noise level of each sensor, and the measurement values are weighted and fused to obtain the fusion value , and the abnormal score is calculated:

[0067]

[0068] When is greater than the preset threshold , it is marked as a candidate fault anchor point.

[0069] Step 2, maintenance text data processing;

[0070] The maintenance text is denoised, and the preliminary cleaning of word segmentation and entity recognition is completed according to the aviation terminology library. Based on the civil aviation maintenance field corpus, the BERT model is fine-tuned in the field, and the fine-tuned BERT model is used to identify maintenance actions, timestamps, component identifiers and fault descriptions in the text. Through semantic role labeling to parse sentence structure, extract "subject-action-object" triplets, and associate context timestamps and component identifiers.

[0071] The triplets are arranged in chronological order to generate maintenance event sequences , and the graph attention network is used to model the causal relationship between events to construct a weighted directed graph , where the edge weight represents the causal strength of event to . The context semantic embedding of the event node is obtained by BERT , and the attention weight is calculated based on the importance of the event to fault diagnosis , and the structured feature vector is generated by weighted aggregation:

[0072]

[0073] And calculate the KL divergence by comparing the distribution difference of the historical normal event chain , when , mark the abnormal event.

[0074] Step 3, fault image data processing;

[0075] For the detection of micro-damage in aerial images, a feature difference quantification method based on ResNet-50 is proposed. First, the image is preprocessed by denoising and brightness standardization to eliminate blurred or irrelevant images. The preprocessed image is input into the ResNet-50 network to extract the last layer of convolution feature map And wherein is the number of channels, And is the spatial size. By calculating the feature difference and introducing the channel attention mechanism, the saliency of the fault area is quantified:

[0076]

[0077] Reflecting the difference contribution of the first channel. The difference map is mapped to the fault image feature vector by global average pooling, and an anomaly score is generated. When , it is determined that the image has a significant fault area.

[0078] In some embodiments of the present application, the multi-modal spatio-temporal alignment and semantic association content includes:

[0079] In the time dimension, there is a time sequence deviation between the WQAR data and the maintenance event record, and the traditional dynamic time warping (DTW) algorithm is difficult to balance the physical consistency. The present application proposes an improved elastic time warping model (IDTW), and the objective function is:

[0080]

[0081] wherein, represents the KL divergence of the local time sequence statistics, is the physical consistency constraint weight. This method realizes the accurate time alignment of multi-modal data by jointly optimizing the morphological similarity and the physical law consistency. By matching the morphological features of the pressure curve and the statistical distribution characteristics of the maintenance event, the two are aligned to the same time window.

[0082] In the event dimension, the present application uses a composite key value mechanism as the semantic association dependency of multi-source data. A six-tuple key space is defined wherein is the aircraft tail number, is the event time window, is the fault ATA code, is the component serial number, For the flight phase, For the fault level. By constructing a multi-dimensional index table through a six-tuple key, different modal data are mapped to a unified logical framework, realizing semantic association and rapid tracing of multi-source data. To improve retrieval efficiency, a dynamic priority queue is designed, and the index weight is automatically adjusted according to the historical query frequency to ensure the response speed in a high-concurrency scenario.

[0083] II. Cross-modal feature fusion and holographic data sample construction

[0084] Cross-modal feature fusion: To realize deep fusion of multi-modal features, a cross-domain attention mechanism is used to construct a joint embedding space, integrating multi-dimensional information into the same feature space to generate a joint feature representation that can fully reflect the health status of the aircraft.

[0085] To fully exploit the advantages of each modality, a cross-modal auxiliary inference mechanism is introduced to fuse the anomaly scores of each modality. The anomaly scores of flight data, maintenance text, and images are , and respectively. is the confidence coefficient of each modality, which is obtained by training historical data. The joint decision formula is:

[0086]

[0087] The cross-modal information is complementary, ensuring that the joint anomaly score exceeds the threshold to confirm the existence of the fault anchor point, thereby effectively aligning the semantic and temporal information of different modalities.

[0088] In some embodiments of the present application, holographic data sample construction: integrate multi-source data to construct a full life cycle archive for aircraft health management and fault prediction, map various information to a unified time coordinate system using time warping technology, identify key event nodes, construct a continuous evolution curve of aircraft health status, and reflect the performance degradation trend of the aircraft in long-term service.

[0089] Based on the preliminary construction of the fault evolution chain, the graph neural network (GNN) is used to iteratively fuse the information of each node in the chain graph and optimize the edge weight. The node update formula is:

[0090]

[0091] where, is the feature vector of node in the first layer, is the neighborhood set of node . At the same time, a closed-loop online feedback mechanism is designed to update the model parameters by comparing real-time data with offline historical data:

[0092]

[0093] wherein, is the time model parameter at time t, is the learning rate, denotes the loss function regarding the joint feature and the true label gradient. This mechanism helps to continuously correct the alignment error during system operation, ensuring the accuracy and robustness of the fault evolution chain.

[0094] III. Integrated modeling method of civil aircraft PHM multi-modal large model

[0095] After the multi-modal holographic data sample library is completed, PHM physical-knowledge fusion integrated modeling will be carried out. In order to solve the problems of civil aviation aircraft key equipment in PHM modeling, such as difficulty in fusion of physical model and engineering knowledge information, poor interpretability of black box model, etc., the present invention studies and proposes a fusion modeling method of "mechanism as core, knowledge as pulse, data as use", proposes a double-layer collaborative framework of "fusion processing-dynamic decision", realizes controllable fusion of physical law and multi-modal knowledge through deep coupling of digital twin module and knowledge enhancement module. The present invention is a hierarchical fusion architecture of large language model (LLM) and digital twin (DT), which realizes effective integration of physical mechanism, real running data, engineering technical data and artificial experience knowledge, overcomes the problems that LLM is not good at numerical calculation and DT is difficult to process multi-source data, and constructs an integrated civil aircraft PHM multi-modal large model which not only has high-fidelity physical simulation capability but also is rich in field knowledge. The model makes full use of the multi-modal data information of civil aircraft PHM, not only can accurately simulate the dynamic change process of equipment health state, but also can automatically combine field knowledge to interpret the output results of the model, providing comprehensive, intelligent and explainable support for equipment state monitoring, fault diagnosis and residual life prediction.

[0096] In some embodiments of the present invention, the PHM physical-knowledge fusion integrated modeling includes a "fusion processing-dynamic decision" double-layer collaborative framework and a double-layer collaborative working mechanism.

[0097] The "fusion processing-dynamic decision" double-layer collaborative framework: the core of the PHM physical-knowledge fusion integrated modeling method is to build a double-layer collaborative model framework of "fusion processing-dynamic decision", such as Figure 2As shown, the framework aims to solve complex problems in civil aircraft PHM modeling through deep integration of physical mechanisms and multi-modal knowledge. The first stage of the framework is the fusion processing layer, which consists of a digital twin (DT) module and a knowledge enhancement module. Both modules complement each other through cross-modal fusion techniques. The digital twin module is based on a physics-informed neural network (PINN) and establishes a simulation model of multi-disciplinary coupled physical equations through a homology mapping method. The input includes real-time sensor data (such as temperature, vibration), control variables (such as torque, speed), and attribute parameters (such as material properties). The model embeds multi-disciplinary physical equations (such as partial differential equations of mechanics, thermodynamics, fluid, etc.) into a neural network through differentiable simulation techniques to dynamically correct parameters and achieve high-fidelity state prediction. For example, for the crack propagation problem of an aircraft engine, the model interacts with real-time temperature data through thermodynamic equations to dynamically adjust the prediction of crack propagation rate. The knowledge enhancement module, with LLM as the core, processes unstructured data such as aviation engineering documents (such as maintenance logs, fault reports, design specifications) and CAD models. Through the domain knowledge ontology library, LLM converts the text description of the fault mode (such as "high temperature causes bearing wear") into quantifiable boundary conditions (such as temperature threshold 120°C), and aligns with image features (such as crack morphology) through cross-modal alignment to form semantic embedding. For example, when the maintenance log mentions "vibration frequency exceeding 2kHz indicates bearing failure", LLM maps this description to quantifiable parameters of spectral features, providing supplementary input for the DT module.

[0098] The output of the fusion processing layer enters the dynamic decision-making layer, which realizes the cooperation of physical simulation and knowledge reasoning through task-adaptive attention mechanism. First, the joint latent representation (fusion of physical state embedding and knowledge semantic embedding) is input into the multi-modal transformer, which dynamically allocates the weights of the two modules using self-attention and cross-attention mechanisms. For example, in the fault diagnosis task, if the sensor data is complete but the text information is missing, the model will increase the weight of the DT module and rely on the physical simulation result; if it needs to combine historical cases for reasoning, the contribution of LLM will be enhanced. The decision-making layer also generates a traceable logic chain to record the weight allocation basis and reasoning path. For example, when the model diagnoses "bearing wear", the logic chain will mark that the conclusion is based on physical simulation of vibration spectrum (weight 60%) and semantic matching of similar cases in the maintenance log (weight 40%). In addition, the dynamic confidence evaluation mechanism quantifies the reliability of physical and knowledge results. When the input data has noise or is missing, the model outputs multiple confidence results (such as "remaining life prediction is 200 hours, confidence 85%"), which can be used as a reference for maintenance personnel. This mechanism is particularly important in handling extreme working conditions (such as extreme low temperature). The model can call the "low-temperature material embrittlement" case in the knowledge base to dynamically correct the physical equation and evaluate the corrected confidence, ensuring the reliability and explainability of the decision.

[0099] Dual-layer collaborative mechanism: The core objective is to cope with the incompleteness of data input through dynamic activation and reasoning mechanisms, while balancing physical rigor and fault adaptability through constraints and releases in the deviation space. First, the dynamic activation and reasoning mechanism adjusts the activation state of the module in real time according to the integrity of the input data. When the sensor data is complete but the engineering document is missing, the digital twin module dominates the prediction, and the knowledge enhancement module completes the missing information through the domain knowledge ontology library. For example, if the vibration data is abnormal but the maintenance log is unavailable, the model automatically calls the "bearing wear" typical characteristics in the knowledge base to generate virtual boundary conditions and inject them into the physical equation. Conversely, if the sensor data is partially missing (such as a temperature sensor failure), the knowledge enhancement module uses the "high temperature alarm record" in the text log and the structural parameters of the CAD model to infer the possible temperature distribution and drive the simulation. When both types of data are incomplete, the dynamic weight distribution mechanism balances the two modules according to the task requirements: for example, in remaining life prediction, if only part of the vibration data and scattered maintenance records are available, the model allocates 60% weight to the DT physical simulation (based on known vibration patterns) and 40% to the LLM historical fault reasoning (based on similar working condition cases), and uses embedded knowledge to complete the missing data (such as estimating the temperature impact through material fatigue curves).

[0100] To realize the dual-layer collaborative model framework and dual-layer collaborative working mechanism of fusion processing-dynamic decision, a set of controllable coupling mechanism is constructed, which takes physical mechanism as the core and can utilize multi-modal document knowledge (troubleshooting manual, graphic data, expert rules, etc.) for reinforcement, so that the digital twin can accurately distinguish new fault traces from random noise while reserving deviation space. The invention proposes the following three innovations.

[0101] Innovation 1: "Boundary docking" of cross-modal knowledge and physical equations

[0102] One of the core challenges of the fusion processing layer is to achieve the "boundary docking" of cross-modal knowledge and physical equations. This process begins with multi-modal data preprocessing: sensor data is encoded into physical state embeddings through PINN, and text and image data extract semantic features through LLM and CNN respectively, and then align to a unified semantic space through contrastive learning or deep canonical correlation analysis (DCCA). For example, the "abnormal vibration amplitude" in the text needs to be associated with a specific frequency band of the sensor frequency spectrum. Subsequently, the knowledge retrieval and matching mechanism is started: when the sensor detects an abnormal temperature rise, the model dynamically retrieves similar fault modes (such as "high temperature accelerates material fatigue") based on the knowledge graph (integrating maintenance manuals and historical fault cases), and extracts quantitative parameters (such as crack propagation rate coefficient 0.05) through LLM semantic analysis. These parameters are dynamically injected into the physical equation, for example, adding a temperature-sensitive term in the crack propagation equation, or adjusting the piecewise constraints of the thermodynamic equation through differentiable programming. To ensure the reasonableness of the correction, the physical consistency verification mechanism checks the core laws such as energy conservation in real time, and if the correction leads to an error exceeding the limit (such as mass conservation error > 5%), the parameter rollback is triggered. At the same time, multi-modal cross-validation compares the simulation results with the matching degree of image features (such as infrared thermal image temperature distribution), and finally optimizes the knowledge base and alignment strategy through feedback iteration.

[0103] Innovation point 2: adaptive correction of multi-modal large models

[0104] The second core challenge of the fusion processing layer is to achieve adaptive correction of multi-modal large models. By generating physical simulation results through the digital twin module, the residual vector is calculated by comparing with sensor data, combined with text logs, image and audio features, and a comprehensive abnormal score is generated using a multi-modal transformer to dynamically determine the abnormal threshold and identify extreme working conditions (such as polar low temperature-high load combination). Once an abnormal signal is detected, the model initiates knowledge-driven fault inference, converts abnormal features into structured queries through a large language model, retrieves similar fault modes and associated physical laws (such as low temperature leading to material embrittlement) in the knowledge graph constructed by integrating maintenance manuals and material failure databases, and selects high-confidence candidates based on multi-modal similarity evaluation. Subsequently, the model dynamically modifies the physical equations, such as incorporating the retrieved material parameter changes (such as yield strength reduction) into the constitutive equation as differentiable correction terms, or generating reasonable extension terms through symbolic regression when existing equations cannot explain the anomaly. For extreme working conditions (such as icing conditions), additional constraints (such as lift coefficient correction table) in the knowledge base are called and injected into the aerodynamic equation. After correction, multi-level verification is performed to check the energy conservation indicators and cross-modal consistency (such as aligning the simulation temperature field with the infrared thermal image), and if the error exceeds the limit, the parameter rollback is triggered; after determining the effectiveness of the correction through confidence scoring, the corrected parameters that pass the verification are stored in the dynamic knowledge base, and the correction strategy is optimized based on reinforcement learning, while the potential defects are identified through adversarial data augmentation in virtual testing, forming a closed-loop knowledge iteration and model evolution.

[0105] Innovation point 3: constraints and release of deviated space

[0106] The core innovation of the double-layer cooperative mechanism is the constraint and release of deviation space, allowing the model to temporarily adapt to unknown faults within a strict physical framework. First, real-time deviation detection is performed, and the residual error (such as the difference in vibration amplitude) is calculated by comparing the physical simulation results generated by the digital twin module with the sensor data. When the residual error continuously exceeds the limit (such as 20% deviation from the predicted value), the deviation channel is triggered. Subsequently, the model retrieves historical cases from the knowledge graph (such as the crack length-frequency rule associated with "turbine blade crack"), extracts evolutionary feature constraints for deviation direction, and regulates the deviation amplitude through a dynamic gating mechanism: based on Bayesian inference, the confidence of the current correction is calculated (such as 70%), and the maximum allowed deviation is limited (such as material strength reduction ≤25%) combined with case similarity (such as 90% match with historical low-temperature embrittlement cases). For example, when detecting material yield anomalies at extremely low temperatures, the model injects temperature-sensitive correction terms into the physical equations and adjusts the range according to the manual safety threshold. After correction, the physical conservation law verification (such as energy balance error <3%) is performed, and if it passes, the correction is accepted and the knowledge base is updated; if the error exceeds the limit, it is rolled back to the original equation and a diagnostic report is generated. In the closed-loop feedback, deviation operation data and correction results are stored in the structured case library for reinforcement learning to optimize deviation trigger thresholds and retrieval strategies, while testing the model's response capability through adversarial data (such as -60°C + turbulent disturbance) to identify potential defects (such as unmodeled material phase transitions). The knowledge graph dynamically expands to add new cases (such as "icing conditions-lift correction"), forming incremental knowledge evolution, ensuring that all corrections are completed within the "double anchoring" of physical laws and engineering experience, and realizing the self-adaptation and controllable evolution of the model to new faults.

[0107] Embodiment 1: Flight data processing

[0108] Step 1: Data preprocessing

[0109] After obtaining the original WQAR flight data, perform outlier filtering, segment identification, and normalization processing. Establish a marking mechanism for sensor fault points.

[0110] Step 2: Flight phase perception feature extraction

[0111] Use flight phase perception mechanism to calculate phase perception indicators where and are the flight speed and acceleration, is the additional offset item in the flight phase.

[0112] Step 3: Incremental information capture

[0113] Update the feature representation through the incremental mechanism so that the feature vector at each time accurately reflects the change in flight state.

[0114] Step 4: Multi-sensor redundancy check

[0115] When there are multiple sensors for the same physical parameter, calculate the fusion value and the anomaly score , when is greater than the preset threshold , it is marked as a candidate fault anchor point.

[0116] Embodiment 2: Cross-modal fusion modeling

[0117] Step 1: Cross-modal feature fusion

[0118] Use cross-domain attention mechanism to construct joint embedding space, and fuse flight data features , text features and image features into a unified feature space.

[0119] Step 2: Joint anomaly score calculation

[0120] Calculate the cross-modal anomaly score by the joint decision formula , where is the confidence coefficient of each modality.

[0121] Step 3: Digital twin module activation

[0122] Based on PINN, a multidisciplinary coupled physical equation simulation model is established, which inputs real-time sensor data, control variables and attribute parameters, and dynamically corrects parameters through differentiable simulation technology.

[0123] Step 4: Knowledge enhancement module cooperation

[0124] LLM processes text data such as maintenance logs and fault reports, and converts text descriptions into quantifiable boundary conditions through a domain knowledge ontology library, providing semantic enhancement support for the digital twin module.

[0125] Step 5: Dynamic decision generation

[0126] The dynamic decision layer fuses the results of physical simulation and knowledge reasoning through a task-adaptive attention mechanism, adjusts the modality weight according to the PHM task requirements, and generates a traceable decision logic chain.

[0127] Embodiment 3: Deviation space constraint and release mechanism

[0128] Step 1: Real-time deviation detection

[0129] Calculate the residual by comparing the simulation results of the digital twin module with the sensor data, and trigger the deviation channel when the residual continuously exceeds the limit.

[0130] Step 2: Knowledge constraint deviation direction

[0131] Search historical cases in the knowledge graph, extract evolution feature constraints to deviate from the direction, and regulate the deviation amplitude through a dynamic gating mechanism.

[0132] Step 3: Physical conservation law verification

[0133] After correction, the energy balance and other physical conservation laws are verified, and if the error is out of limit, the original equation is rolled back and a diagnostic report is generated.

[0134] Step 4: Closed-loop feedback optimization

[0135] Deviation operation data and correction results are stored in the case library for reinforcement learning to optimize deviation trigger thresholds and retrieval strategies.

[0136] The present application realizes efficient processing and intelligent modeling of civil aircraft PHM multi-modal data through the fusion modeling concept of "mechanism as the core, knowledge as the vein, and data as the use", providing reliable technical support for civil aviation safety operation. This method not only solves the technical problem of multi-source heterogeneous data fusion, but also guarantees the interpretability and engineering practicability of the model through the physical-knowledge collaborative mechanism.

[0137] The parts of the present application not described in detail can refer to the prior art or be known to those skilled in the art, and the present embodiment is not limited thereto, and will not be described in detail here.

[0138] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative and not limiting, and those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A civil aircraft PHM modeling method based on cross-modal coupling, characterized by: The method achieves efficient integration of multimodal data and physical-knowledge fusion modeling through a cross-modal coupling-holographic knowledge evolution dual-driven data processing mechanism and a fusion processing-dynamic decision-making two-layer collaborative framework.

2. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 1 is characterized in that: The steps of the method include: S1. Multimodal data preprocessing: Feature extraction and spatiotemporal / event alignment are performed for three types of modalities: flight time series data, text data, and image data, laying the foundation for subsequent fusion modeling; S2. Cross-modal feature fusion and holographic data sample construction: Through cross-domain fusion and dynamic optimization, a holographic data sample library covering the entire life cycle is constructed; S3. PHM physics-knowledge fusion integrated modeling: Construct a two-layer collaborative framework that combines physical simulation and knowledge reasoning to achieve controllable fusion of physical simulation and multimodal knowledge.

3. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 2 is characterized in that: The content of feature extraction in step S1 includes: For flight time series data: a multivariate heterogeneous long time series prediction model is used to extract the flight dynamic mode feature vector V Q ; Calculate anomaly scores through multi-sensor redundancy verification , mark candidate fault anchor points; For text data: Use a large language model enhanced with domain knowledge to parse unstructured maintenance work orders and fault reports into structured event chain feature vectors ; Calculate anomaly score by KL divergence , identify abnormal events; For image data: Based on ResNet-50 and channel attention mechanism, extract the local feature vector of the fault image ; Calculate anomaly score by L2 norm , determine the significant fault area.

4. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 2 is characterized in that: The contents of spatiotemporal alignment in step S1 include: Based on an improved elastic time warping model, the temporal deviation problem of multimodal data is solved by jointly optimizing morphological similarity and consistency of physical laws. Event alignment includes: Design a six-tuple key-value mechanism driven by a composite logic chain to realize the association mapping between faults and maintenance records and flight missions, ensuring multimodal features ( , , )’s precise association.

5. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 2 is characterized in that: The cross-modal feature fusion in step S2 includes: The cross-domain attention mechanism is used to transform the flight dynamic pattern feature vector , structured event chain feature vector , local feature vector of the fault image Mapped to a unified joint embedding space to form a joint feature containing multi-dimensional information; calculated the joint anomaly score through a confidence-based calculation method to accurately determine the fault anchor point.

6. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 2 is characterized in that: The contents of constructing the holographic data sample in step S2 include: Aircraft history dimension: Map multimodal data to a unified time coordinate system, identify key nodes, and construct a continuous evolution curve of the aircraft's health status to reflect performance degradation trends; Fault history dimension: Graph neural networks are used to optimize the node features and edge weights of the fault evolution chain. A closed-loop online feedback mechanism is used to correct alignment errors in real time, ensuring the accuracy of the fault evolution chain.

7. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 1 is characterized in that: The fusion processing-dynamic decision-making two-layer collaborative framework includes: Fusion processing layer: used to achieve boundary connection between cross-modal knowledge and physical equations; including digital twin module and knowledge enhancement module; The digital twin module, based on homologous isomorphic mapping, embeds multidisciplinary coupled physical equations into a neural network to obtain a physical information neural network, then inputs real-time sensor data, control variables, and attribute parameters to achieve physical simulation and output physical state embedding; The knowledge enhancement module parses text data through a pre-trained large language model and outputs domain knowledge embedding; Dynamic decision layer: Dynamically allocates the weights of the digital twin module and the knowledge enhancement module through the task-adaptive attention mechanism to generate a traceable decision logic chain; combined with the dynamic confidence evaluation mechanism, it outputs prediction results with confidence.

8. The civil aircraft PHM modeling method based on cross-modal coupling according to claim 7 is characterized in that: The digital twin module dynamically updates the physical equation parameters through an adaptive correction mechanism to achieve high-fidelity physical simulation; The updated physical equation parameters are readjusted through the constraint and release mechanism of the deviation space. Specifically, if there is a deviation in the physical equation after the updated parameters, the deviation amplitude is limited through the dynamic gating mechanism, and the physical conservation law is verified; if the verification passes, the current parameters are retained; if the limit is exceeded, the parameter rollback is triggered.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the content of the civil aircraft PHM modeling method based on cross-modal coupling as described in any one of claims 1 to 8 is implemented.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the content of the civil aircraft PHM modeling method based on cross-modal coupling as described in any one of claims 1 to 8.

Images