IMA fault monitoring method and device based on multi-modal data

Abstract

According to the IMA fault monitoring method and device based on the multi-modal data, high-coverage fault detection of an IMA system is achieved, and the real-time performance and accuracy of fault recognition are improved. A hierarchical processing mechanism of module recovery-dynamic reconstruction-emergency configuration is adopted, faults are rapidly isolated, system functions are recovered, and the robustness of the system is enhanced. And meanwhile, natural language maintenance suggestions are generated by utilizing the fault maintenance model, so that the maintenance threshold is reduced, and intelligent auxiliary maintenance suggestions are provided for maintenance personnel. The problems of high complexity and low adaptability of a traditional method are solved, the fault detection accuracy and the system resource utilization rate are improved, the aviation safety certification requirement is met, and the method is suitable for avionics system health management of civil airplanes and military aircrafts.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of integrated avionics, and particularly relates to an IMA fault monitoring method and device based on multi-modal data. BACKGROUND

[0002] In the contemporary aviation industry system, the avionics system as the nerve center of the aircraft, its integration degree and strategic value are growing exponentially with the technology iteration. From the early joint architecture to the technology leap of integrated modular avionics (IMA), although the decoupling of functional modules and platform independence are realized, new challenges are derived. Under the IMA architecture, the hardware resource configuration matrix is large in scale, the functions are intertwined to form a complex network, and the fault location is like finding a needle in a haystack, and the field maintenance faces the double pressure of technical barriers and time cost. The wide application of field replaceable unit (LRU) not only gives the system flexible reconfiguration capability, but also puts forward strict requirements on the whole life cycle health management system.

[0003] In the prior art, the health management of the avionics system mainly relies on the artificial rule engine and the static configuration file, which has the following significant defects: 1. High configuration complexity: Modern avionics systems contain thousands of parameters, and manual maintenance cannot adapt to dynamic task requirements. 2. Insufficient fault prediction capability: Traditional methods can only handle single structured data (such as sensor threshold), and cannot integrate unstructured data such as maintenance logs and design documents, so the coverage rate of fault diagnosis is low. 3. Weak real-time and adaptive capability: Static configuration files cannot respond to changes in flight environment, resource utilization is low, and dynamic optimization of configuration files cannot be realized. In the prior art, Chinese patent CN112329875A proposes a health management method for avionics systems, but this method relies on a fixed rule base and cannot realize dynamic optimization of configuration files. Therefore, there is an urgent need for a health management scheme for avionics system resource configuration files based on large models to improve the health management and maintenance level of avionics system resource configuration files. SUMMARY

[0004] The application provides an IMA fault monitoring method and device based on multi-modal data, which realizes online fault monitoring, dynamic reconstruction and auxiliary maintenance of the avionics system, and solves the problems of poor flexibility and weak adaptability of traditional methods.

[0005] The first aspect of the application provides an IMA fault monitoring method based on multi-modal data, comprising the following steps: S1: Collecting historical multi-modal operation data and system maintenance data of the IMA, correlating and extracting the collected data, and performing normalization processing, constructing an IMA fault monitoring model and a fault maintenance model; S2: deploying the fault monitoring model in an independent partition of the IMA, isolated from the functional partition; S3: inputting real-time collected online multi-modal operation data to the fault monitoring model during system operation, and calculating a residual value using a predicted value of the fault monitoring model and an actual state vector; S4: determining whether the residual value is within a preset health threshold: if yes, determining that the avionics system is in a normal state, and continuously monitoring; if not, determining that the avionics system is in a fault state, determining a fault cause according to the actual state vector and system maintenance data, the fault cause including a module fault and a partition fault, and entering S5; S5: if the fault module or the fault partition is not restartable, cutting off a resource sharing and data interaction channel of the fault module or the fault partition; if the fault module or the fault partition is restartable, cutting off the resource sharing and data interaction channel of the fault module or the fault partition, and executing a module or partition level fault recovery function; performing periodic state monitoring to determine whether the system is restored to normal; if yes, returning to S3 for continuous monitoring; if not, entering S6; S6: running a resource reconfiguration algorithm based on a task priority, and loading a reconfigured IMA system configuration file; S7: performing periodic state monitoring on the reconfigured system to determine whether the reconfiguration is successful: if the periodic state monitoring is normal, the reconfiguration is successful, and the avionics system is in a normal state, returning to S4 for continuous monitoring; if the periodic state monitoring is abnormal, the reconfiguration fails, and entering S8; S8: enabling a pre-stored historical effective resource configuration.

[0006] Optionally, the method further comprises: S9: in response to abnormal periodic state monitoring of the system, recording system fault logs in a system configuration pre-defined NVRAM address; inputting the fault logs into a fault maintenance model to generate natural language based auxiliary fault analysis and processing suggestions.

[0007] Optionally, the S1 further comprises: adjusting the fault monitoring model and the fault maintenance model based on IMA field rules.

[0008] Optionally, in the S1, the historical multi-modal operation data includes sensor collected parameters in a flight operation process.

[0009] Optionally, in the step one, the system maintenance data includes crew alarm records, BIT self-checking reports, component replacement records, and maintenance reports.

[0010] Optionally, the periodic state monitoring includes the fault monitoring model, BIT self-checking, watchdog, and heartbeat monitoring.

[0011] Optionally, in the resource reconfiguration algorithm, the resources subjected to distribution and optimization include interface resources, network resources and computing resources.

[0012] The second aspect of the present application provides an IMA fault monitoring device based on multi-modal data, configured to perform the IMA fault monitoring method based on multi-modal data as any one of the first aspect.

[0013] The third aspect of the present application provides a computer readable storage medium, comprising: a memory and a processor; The memory is configured to save executable instructions; The processor is configured to implement the IMA fault monitoring method based on multi-modal data as any one of the first aspect when executing the executable instructions saved by the memory.

[0014] The fourth aspect of the present application provides a computer program product, comprising instructions, which, when executed by a computer, implement the IMA fault monitoring method based on multi-modal data as any one of the first aspect.

[0015] The present application has the following beneficial effects compared with the prior art: The present application provides an IMA fault monitoring method and device based on multi-modal data, which realizes high-coverage fault detection of the IMA system and improves the real-time performance and accuracy of fault identification. In the present application, a hierarchical processing mechanism of "module recovery - dynamic reconfiguration - emergency configuration" is adopted to quickly isolate faults and recover system functions, thereby enhancing the robustness of the system. In the present application, a natural language maintenance suggestion is generated by using a fault maintenance model, thereby reducing the maintenance threshold and providing intelligent assistance for maintenance personnel. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. The drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0017] Fig. 1 A flowchart of a comprehensive modular avionics system fault monitoring and auxiliary maintenance method based on multi-modal data provided by the present application; Fig. 2 In a comprehensive modular avionics system fault monitoring and auxiliary maintenance method based on multi-modal data provided by the present application, a fault recovery function architecture diagram; Fig. 3An IMA system software architecture diagram is provided in a comprehensive modular avionics system fault monitoring and auxiliary maintenance method based on multi-modal data. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, any other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0019] The features and illustrative embodiments of various aspects of the present application will be described below in detail. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one of ordinary skill in the art that the present application can be practiced without some of these specific details. The description of the embodiments below is merely intended to provide a better understanding of the present application by showing examples of the present application. The present application is in no way limited to any specific settings and methods presented below, but covers any improvements, replacements and modifications of structures, methods and devices without departing from the spirit of the present application. In the drawings and the following description, well-known structures and techniques are not shown to avoid unnecessary obscuring the present application.

[0020] In the description of the present application, it should be noted that the directions or positional relationships indicated by "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like are described based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and cannot be understood as limiting the present application. In addition, the ordinal numbers (for example, "first" and "second") are used to distinguish objects, and are not limited to the order, and cannot be understood as indicating or implying relative importance.

[0021] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, "mounting", "connection" and "connection" should be understood in a broad sense, which can be direct connection or indirect connection through an intermediate medium. For a person of ordinary skill in the art, the specific meaning of the above-mentioned terms in the present application can be understood according to the specific circumstances.

[0022] It should be noted that, in the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other, and each embodiment can be mutually referred to and quoted. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0023] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0024] See Figs. 1-3 As shown, this invention provides a method for fault monitoring and auxiliary maintenance of a comprehensive modular avionics system based on multimodal data, comprising the following steps: Step 1: Collect historical multimodal operation data and system maintenance data of the IMA system, perform correlation extraction and normalization processing on the collected data, construct the IMA system fault monitoring model and fault repair model, and fine-tune the model based on the domain rules of the IMA system.

[0025] Step 2: Deploy the fault monitoring model in an independent partition of the airborne IMA system, isolated from the functional partition; during system operation, input the real-time collected online multimodal operation data into the model, and calculate the residual using the model's predicted values ​​and the actual state vector; Step 3: Determine whether the residual is within the preset health threshold: If it is within the health threshold, the avionics system is determined to be in normal condition and is continuously monitored; if it is not within the health threshold, the avionics system is determined to be in fault condition and proceed to Step 4.

[0026] Step 4: Disconnect the resource sharing and data interaction channels of the faulty module or partition, execute the module or partition-level fault recovery function, perform periodic status monitoring, and determine whether the system has returned to normal: if it has recovered, return to Step 2 for continuous monitoring; if it has not recovered, proceed to Step 5.

[0027] Step 5: Generate dynamic refactoring instructions, isolate faulty modules or partitions, run resource refactoring algorithms based on task priorities, and load the refactored IMA system configuration file.

[0028] Step Six: Perform periodic status monitoring on the reconstructed system to determine if the reconstruction was successful: If the periodic status monitoring is normal, the reconstruction is successful, and the avionics system is in a normal state. Return to Step Two for continuous monitoring; if the periodic status monitoring is abnormal, the reconstruction fails, and proceed to Step Seven. Step 7: For any abnormalities detected in the system's periodic status monitoring, record the system fault log in the predefined NVRAM address in the system configuration. Enable the pre-stored historical valid configurations to ensure the normal operation of critical system functions.

[0029] Step 8: Input the fault log into the fault repair model to generate natural language-based auxiliary fault analysis and handling suggestions.

[0030] The process of the integrated modular avionics system fault monitoring and auxiliary maintenance method based on multimodal data described in this embodiment is as follows: Step 1: Data Acquisition and Preprocessing Historical operational data (such as flight altitude, airspeed, bus load, etc.) of the IMA system under different flight phases and mission scenarios are collected via the flight data recorder and avionics bus. Avionics system maintenance data (crew alarm records, BIT self-test reports, component replacement records, maintenance reports) are obtained from the maintenance system. Data cleaning is performed on multi-source data, parameters are normalized to a unified range, a fault monitoring model is built, and the system status is predicted in real time; a fault repair model is built, and fault logs are analyzed to generate auxiliary maintenance suggestions; based on rules in fields such as avionics safety standards and module interaction protocols, the model is fine-tuned to adapt it to the IMA system scenario.

[0031] Step 2: Fault monitoring model deployment and deviation calculation The fault monitoring model is deployed in an independent partition of the airborne IMA system, isolated from other functional partitions, to ensure the independence of the fault monitoring model from the resident applications. During system operation, online multimodal data is collected in real time and input into the model to obtain state prediction values. At the same time, sensor data is collected periodically in real time to obtain the actual state vector and calculate the deviation between the predicted value and the actual value.

[0032] Step 3: IMA System Health Status Assessment Based on the IMA system design requirements and historical fault data, a preset health threshold range is set as a reasonable range for residuals during normal operation. If the deviation is within the range, the system is considered normal, and the process returns to step two for continuous monitoring. If the residual exceeds the range, the system is considered faulty, and step four is executed.

[0033] Step 4: IMA System Troubleshooting and Initial Recovery By cutting off the resource sharing channels of the faulty module or partition through the resource management module, such as memory, the memory access permissions of the faulty module are set to "disabled" to prevent illegal read and write operations; for data interaction channels, such as bus communication links, traffic filtering is performed on the MAC address of the faulty partition in the AFDX switch to cut off data interaction; execute the module or partition-level recovery function; after recovery, check whether the system is normal: if normal, return to step two; if the fault still exists, execute step five.

[0034] Step 5: Dynamic Reconfiguration of the IMA System The system generates dynamic reconfiguration instructions, runs resource reconfiguration algorithms based on task priorities, and reallocates computing, network, and interface resources. Computing resources include application type, execution time, execution cycle, execution priority, the hardware to which the application belongs, and the execution schedule. Network resources include network topology, transmission paths, virtual links, and sub-virtual links. Interface resources include port configurations for each terminal device and the mapping between logical and physical ports. Each device in the IMA system loads its corresponding configuration file, completing the dynamic reconstruction of the system.

[0035] Step six: Periodic state monitoring of the reconfigured system Periodic state monitoring is performed on the reconfigured system to determine whether the system reconfiguration was successful. If the periodic state monitoring is normal, the reconfiguration is successful, and the avionics system is in a normal state, continuous monitoring is performed. If the periodic state monitoring is abnormal, the reconfiguration fails, and step seven is performed. Periodic state monitoring includes a fault monitoring model, BIT self-checking, a watchdog, and heartbeat monitoring.

[0036] Step seven: Fault log recording and emergency handling In response to abnormal periodic state monitoring of the system, system fault logs are recorded, including fault timestamp, fault module ID, fault partition ID, and fault code. Pre-stored historical effective configurations are enabled to ensure normal operation of key system functions.

[0037] Step eight: Generation of fault maintenance assistance suggestions Fault logs are input into a fault maintenance model, and the model generates natural language maintenance suggestions based on historical maintenance data and domain knowledge for reference by maintenance personnel.

[0038] The structure of a comprehensive modular avionics system fault monitoring and auxiliary maintenance method based on multi-modal data is as follows: it includes a data processing and model construction module, a real-time fault monitoring module, a fault preliminary processing module, a dynamic reconfiguration module, an emergency support module, and a maintenance assistance module.

[0039] The data processing and model construction module completes the collection and preprocessing of multi-modal data of the IMA system, as well as the construction and field rule fine-tuning of the fault monitoring model and the fault maintenance model. The real-time fault monitoring module deploys the IMA system fault monitoring model in the independent partition of the airborne IMA system, inputs real-time collected online multi-modal running data, calculates the deviation using the actual state vector and the predicted value; realizes the airborne deployment of the fault monitoring model, as well as real-time deviation calculation and health state determination during system operation. The fault preliminary processing module performs preliminary isolation and recovery attempts on the fault system, executes IMA system module or partition level fault recovery functions, and quickly handles simple faults. The dynamic reconfiguration module performs system level dynamic reconfiguration when preliminary recovery fails, reallocates resources to recover system functions. The emergency support module performs periodic state monitoring on the corrected system to determine whether the system reconfiguration is successful. When the reconfiguration fails, pre-stored historical effective configurations are enabled to ensure system key functions. The maintenance assistance module inputs system fault logs into the IMA system fault maintenance model, and uses the fault maintenance model to provide fault analysis and processing suggestions based on natural language for system maintenance.

[0040] The above detailed description of the application is not intended to limit the application to the specific embodiments described, but is intended to cover all alternatives that fall within the scope of the application.

Claims

1. A method for IMA fault monitoring based on multi-modal data, characterized in that, Comprising the following steps: S1: Collecting historical multi-modal operation data and system maintenance data of IMA, performing correlation extraction and normalization processing on the collected data, constructing IMA fault monitoring model and fault maintenance model; S2: Deploying the fault monitoring model in the independent partition of IMA, isolated from the functional partition; S3: During system operation, inputting the online multi-modal operation data collected in real time to the fault monitoring model, and calculating the residual error using the predicted value of the fault monitoring model and the actual state vector; S4: Determine whether the residual error is within the preset health threshold: if it is within the health threshold, the avionics system is in normal state, and continuous monitoring is performed; If it is not within the health threshold, the avionics system is in a fault state, the fault cause is determined according to the actual state vector and the system maintenance data, the fault cause includes module fault and partition fault, and S5 is entered; S5: If the fault module or fault partition is not restartable, the resource sharing and data interaction channel of the fault module or fault partition is cut off; If the fault module or fault partition is restartable, the resource sharing and data interaction channel of the fault module or fault partition is cut off, and the module or partition level fault recovery function is executed; the periodic state monitoring is performed to determine whether the system is restored to normal; If it is restored, return to S3 for continuous monitoring; if it is not restored, enter S6; S6: Running the resource reconstruction algorithm based on task priority, and loading the reconstructed IMA system configuration file; S7: Performing periodic state monitoring on the reconstructed system to determine whether the reconstruction is successful: if the system periodic state monitoring is normal, the reconstruction is successful, and the avionics system is in normal state, returning to S4 for continuous monitoring; If the system periodic state monitoring is abnormal, the reconstruction fails, and S8 is entered; S8: Enable the pre-stored historical effective resource configuration.

2. The method for IMA fault monitoring based on multi-modal data according to claim 1, characterized in that, Also includes: S9: For system periodic state monitoring abnormal, record system fault log in system configuration pre-defined NVRAM address; Input the fault log into the fault maintenance model to generate natural language based auxiliary fault analysis and processing suggestion.

3. The method for IMA fault monitoring based on multi-modal data according to claim 1, characterized in that, The S1 further comprises: Adjusting the fault monitoring model and the fault maintenance model based on IMA domain rules.

4. The method for IMA fault monitoring based on multi-modal data according to claim 1, characterized in that, In the S1, the historical multi-modal operation data includes sensor collected parameters during aircraft operation.

5. The method for IMA fault monitoring based on multi-modal data according to claim 1, characterized in that, In the step one, the system maintenance data includes crew alarm record, BIT self-check report, component replacement record, and maintenance report.

6. The method for IMA fault monitoring based on multi-modal data according to claim 1, characterized in that, Periodic state monitoring includes fault monitoring model, BIT self-check, watchdog and heartbeat monitoring.

7. The method for IMA fault monitoring based on multi-modal data according to claim 1, characterized in that, In the resource reconstruction algorithm, the distributed and optimized resources include interface resources, network resources and computing resources.

8. An IMA fault monitoring apparatus based on multi-modal data, characterized by, For executing the multi-modal data based IMA fault monitoring method as claimed in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, Comprising: Memory and processor; The memory is configured to save executable instructions; The processor is configured to implement the multi-modal data based IMA fault monitoring method as claimed in any one of claims 1-7 when executing the executable instructions saved by the memory.

10. A computer program product, characterised in that, The computer program product includes instructions that, when executed by a computer, implement the IMA fault monitoring method based on multimodal data as described in any one of claims 1-7.

Images