Intelligent airplane inspection system based on multi-modal large model

Abstract

The invention belongs to the technical field of artificial intelligence and aviation safety, and particularly relates to an intelligent aircraft inspection system based on a multi-modal large model. Comprising a multi-modal data acquisition module, a data preprocessing module, a multi-modal large model analysis module, a real-time fault diagnosis and risk prediction module, a cloud-based data management module and a user interaction and visualization module. Through the powerful processing capability of the large model, comprehensive analysis of various states of the aircraft is realized, and potential problems are found in time, so that the automation level and safety of aircraft inspection are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and aviation safety technology, and in particular to an intelligent aircraft inspection system based on a multi-modal large model. Background Art

[0002] Aircraft safety and reliability are of vital importance to the air transport industry. Therefore, regular aircraft inspection and maintenance are necessary measures to ensure flight safety.

[0003] However, existing aircraft inspection technologies face several challenges and limitations, including the following:

[0004] Low efficiency of manual inspection: Traditional manual inspection relies on technicians to manually inspect the aircraft. Due to the large size and complex components of the aircraft, a comprehensive inspection takes a lot of time. Technicians need to carefully inspect each component, and it is difficult to ensure the thoroughness and consistency of each inspection;

[0005] Strong subjectivity of personnel: During manual inspections, the experience and subjective judgment of technicians will affect the inspection results, which may cause some potential problems to be missed, thereby increasing the risk of failure;

[0006] Limitations of single-mode inspection: Current inspection technologies usually rely on single-mode data, such as using only visual cameras for inspection or relying on acoustic sensors to detect noise. This single-mode data cannot fully capture the aircraft status and may miss faults that can be identified through other modes;

[0007] Limited data processing capabilities: Traditional detection systems face challenges in processing and analyzing large amounts of sensor data. Data processing is often decentralized, lacking effective integration and analysis, making it difficult to extract useful information from complex data.

[0008] Limited application scope: Although artificial intelligence (AI) and machine learning (ML) technologies have been used in the aviation field to a certain extent in recent years, these technologies are mostly used for data analysis or fault prediction and have not yet been widely integrated into actual aircraft inspection systems;

[0009] Insufficient multimodal data fusion: Current AI and ML applications are still insufficient in multimodal data fusion. The comprehensive analysis capabilities of multimodal data (such as visual, infrared, sound waves, etc.) are not strong, making it impossible to fully utilize the potential of these data to improve the accuracy and comprehensiveness of fault detection;

[0010] High cost: Traditional comprehensive inspections require a lot of labor costs and equipment maintenance costs. In addition, high-frequency inspections may interfere with the normal use of aircraft and affect the operating efficiency of airlines;

[0011] Safety risks: Due to the limitations of manual inspection, some hidden faults may not be discovered in time, increasing flight safety risks.

[0012] In summary, the existing aircraft inspection technology faces many problems such as low efficiency, limited detection range, insufficient data processing capacity, high cost and safety risks. To this end, an intelligent aircraft inspection system based on a multimodal large model is proposed to solve the above problems. Summary of the invention

[0013] The purpose of the present invention is to provide an intelligent aircraft inspection system based on a multi-modal large model to solve the problems raised in the above background technology.

[0014] To achieve the above-mentioned object, the present invention provides the following technical solutions: an intelligent aircraft inspection system based on a multimodal large model, comprising a multimodal data acquisition module, a data preprocessing module, a multimodal large model analysis module, a real-time fault diagnosis and risk prediction module, a cloud-based data management module, and a user interaction and visualization module;

[0015] The multimodal data acquisition module is used to integrate historical and production information of multiple sensors and aircraft, with the purpose of comprehensively acquiring and modeling the aircraft status;

[0016] The data preprocessing module is used to perform specific preprocessing on each type of data, and to perform denoising, normalization, data enhancement and other processing on the collected raw data to improve the data quality. This includes but is not limited to processing vibration and acoustic data based on frequency domain filtering and adaptive noise elimination algorithms to reduce environmental noise and signal errors;

[0017] The multi-modal large model analysis module is used to perform fault identification on the multi-modal large model;

[0018] The real-time fault diagnosis and risk prediction module supports real-time processing to meet the needs of rapid feedback during inspections;

[0019] The cloud-based data management module utilizes cloud computing to achieve centralized management and efficient analysis of data;

[0020] The user interaction and visualization module provides visualization of inspection data through an interactive interface to improve user experience.

[0021] Preferably, the data collected by the multimodal data acquisition module include but are not limited to visual data, infrared data, acoustic data, vibration data, aircraft information and time data; the visual data is the image data of the outside and inside of the aircraft collected by a high-resolution camera and a hyperspectral camera; the infrared data is the heat distribution on the surface of the aircraft detected by an infrared sensor to identify potential overheating problems; the acoustic data is the sound wave signal of the aircraft during operation monitored by an acoustic sensor to identify abnormal noise, and the acoustic sensor includes a microphone array; the vibration data is the vibration data of each component of the aircraft collected by a vibration sensor to detect structural problems; the aircraft information includes specific models, production batches, inspection records, warranty and fault records, and aircraft manuals to provide in-depth background data to help large models identify potential problems of specific models and enrich the background information of multimodal data; the time data is equipped with synchronized timestamps for all modal data to ensure the temporal consistency of each modal information for accurate modeling and dynamic analysis.

[0022] Preferably, the functions of the data preprocessing module include feature extraction and data alignment. The feature extraction is to extract edge and texture features from visual data through convolutional neural network (CNN), extract hot spots from infrared data through heat map segmentation, and extract spectrum features from acoustic data through Fourier transform. A dynamic multimodal fusion mechanism is adopted, and a weighting strategy based on the attention mechanism is introduced in the fusion unit, so that the system can adaptively adjust the fusion weight of each modal feature according to the content of real-time data and the importance of related modalities. This dynamic fusion method not only improves the sensitivity of the model to diversified fault features, but also allows the system to automatically adjust the priority of each modal feature in different operating scenarios (such as engine, fuselage, etc.), thereby enhancing the applicability and accuracy of the model.

[0023] The data alignment uses timestamps to synchronize data between modalities and implements multimodal data alignment based on the dynamic time warping (DTW) algorithm to ensure the consistency of model input data in the time dimension. By identifying the temporal relationship between each modality in the historical data, a new time-aligned feature group is generated. This mechanism can capture the potential correlation between sensor, image and sound data during the alignment process.

[0024] Preferably, the functions of the multimodal large model analysis module include time series processing and large model training;

[0025] The time series processing is to introduce a long short-term memory network (LSTM) to process time series data to capture dynamic information, and combine the global context-aware characteristics of the converter architecture to achieve understanding of long-term behavior and improve fault prediction accuracy;

[0026] The large model training is based on the labeled fault data set and the historical fault records of the specific model using self-supervised learning technology to train a deep neural network combining CNN and RNN to improve the generalization ability of fault recognition and prediction of the specific model under multi-modal input.

[0027] Preferably, the functions of the real-time fault diagnosis and risk prediction module include real-time diagnosis and risk prediction;

[0028] The real-time diagnosis is to process multimodal data in real time and dynamically update model parameters through edge computing and incremental learning technology to improve the response speed of real-time fault detection;

[0029] The risk prediction utilizes time series analysis and outlier detection algorithms to identify potential failure trends in advance, assist in predicting possible risk components in the future, and ensure preventive maintenance.

[0030] Preferably, the functions of the cloud-based data management module include cloud storage and sharing, cloud analysis and model updating, and data security;

[0031] The cloud storage and sharing is to store aircraft historical data and production batch information, and to achieve collaborative analysis by different inspection teams through the cloud;

[0032] The cloud-based analysis and model update is to use the high-performance computing resources in the cloud to conduct in-depth data analysis and iterative model updates, and improve the efficiency of data analysis and model optimization through automated tasks;

[0033] The data security is to implement multi-layer data encryption and access control to ensure the security of sensitive data in the cloud.

[0034] Preferably, the functions of the user interaction and visualization module include real-time status display and inspection report generation;

[0035] The real-time status display is to display the real-time status and inspection progress of the aircraft, and to provide instant warning of abnormal situations, so as to support users to respond quickly;

[0036] The inspection report generation is that the system automatically generates an inspection report, which covers faulty components, cause analysis and recommended maintenance measures to facilitate subsequent maintenance and management decisions.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1) Multimodal data integration and processing: The system integrates multiple sensors and combines the specific model information of the aircraft, historical inspection records, warranty and fault records, production batches and operating manuals to form multimodal data, which enhances the depth and breadth of the data, enables large model analysis to have richer contextual support, and ensures more comprehensive diagnosis;

[0039] 2) Accurate multimodal data fusion and analysis: Through the combination of deep neural networks and time series models, the system achieves accurate fusion of multimodal data, effectively improving the accuracy and timeliness of fault identification, and combines time series data for behavioral pattern learning to achieve potential fault prediction for specific models;

[0040] 3) Real-time and edge computing advantages: By utilizing the real-time processing capability and incremental learning mechanism of edge computing, the system can respond to inspection needs in a timely manner, provide instant fault diagnosis and potential risk warning, and effectively reduce manual maintenance time and maintenance costs;

[0041] 4) Powerful cloud data management and sharing: This system uses the cloud to achieve data storage, analysis and sharing, effectively improving the collaboration efficiency between different inspection teams. It also supports continuous optimization and model training of the system to ensure that equipment inspection data is continuously enriched and learning models are continuously updated;

[0042] 5) User-friendly interactive experience: The system provides an intuitive visual interface, allowing users to obtain aircraft status and warning information in real time, quickly generate inspection reports, provide a convenient operation experience, and improve the efficiency of inspection and maintenance decision-making;

[0043] 6) High level of automation: This system greatly reduces manual intervention, reduces the labor intensity of manual inspection, enhances the comprehensiveness and accuracy of fault detection, reduces the risk of missed detection, and helps to improve the safety and economy of aviation operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural diagram of the intelligent aircraft inspection system based on a multi-modal large model;

[0045] Figure 2 It is the system structure diagram;

[0046] Figure 3 Multimodal data acquisition module structure diagram;

[0047] Figure 4 This is the functional structure diagram of the data preprocessing module;

[0048] Figure 5 This is the functional structure diagram of the multimodal large model analysis module;

[0049] Figure 6 This is the functional structure diagram of the real-time fault diagnosis and risk prediction module;

[0050] Figure 7 It is the functional structure diagram of the cloud-based data management module;

[0051] Figure 8 It is the functional structure diagram of the user interaction and visualization module;

[0052] Fig. 9 This is the inspection flow chart. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0055] See also Figure 1-9 , the present invention provides a technical solution:

[0056] An intelligent aircraft inspection system based on a multimodal large model, including a multimodal data acquisition module, a data preprocessing module, a multimodal large model analysis module, a real-time fault diagnosis and risk prediction module, a cloud-based data management module, and a user interaction and visualization module;

[0057] The multimodal data acquisition module is used to integrate historical and production information of multiple sensors and aircraft, with the purpose of comprehensively acquiring and modeling the aircraft status;

[0058] The data preprocessing module is used to perform specific preprocessing on each type of data, and to perform denoising, normalization, data enhancement and other processing on the collected raw data to improve the data quality. This includes but is not limited to processing vibration and acoustic data based on frequency domain filtering and adaptive noise elimination algorithms to reduce environmental noise and signal errors;

[0059] The multi-modal large model analysis module is used to perform fault identification on the multi-modal large model;

[0060] The real-time fault diagnosis and risk prediction module supports real-time processing to meet the needs of rapid feedback during inspections;

[0061] The cloud-based data management module utilizes cloud computing to achieve centralized management and efficient analysis of data;

[0062] The user interaction and visualization module provides visualization of inspection data through an interactive interface to improve user experience.

[0063] The system can integrate multimodal data, including visual images, sounds, sensor data, etc. Through the powerful processing capabilities of large models, it can achieve a comprehensive analysis of various aircraft conditions and discover potential problems in a timely manner, thereby improving the automation level and safety of aircraft inspections.

[0064] The data collected by the multimodal data acquisition module include but are not limited to visual data, infrared data, acoustic data, vibration data, aircraft information and time data; the visual data is the image data of the outside and inside of the aircraft collected by a high-resolution camera and a hyperspectral camera; the infrared data is the heat distribution on the surface of the aircraft detected by an infrared sensor to identify potential overheating problems; the acoustic data is the sound wave signal of the aircraft during operation monitored by an acoustic sensor to identify abnormal noise, and the acoustic sensor includes a microphone array; the vibration data is the vibration data of each component of the aircraft collected by a vibration sensor to detect structural problems; the aircraft information includes specific models, production batches, inspection records, warranty and fault records, and aircraft manuals to provide in-depth background data to help large models identify potential problems of specific models and enrich the background information of multimodal data; the time data is equipped with a synchronized timestamp for all modal data to ensure the temporal consistency of each modal information for accurate modeling and dynamic analysis.

[0065] The functions of the data preprocessing module include feature extraction and data alignment. The feature extraction is to extract edge and texture features from visual data through convolutional neural network (CNN), extract hot spots from infrared data through heat map segmentation, and extract spectrum features from acoustic data through Fourier transform. A dynamic multimodal fusion mechanism is adopted, and a weighting strategy based on the attention mechanism is introduced in the fusion unit, so that the system can adaptively adjust the fusion weight of each modal feature according to the content of real-time data and the importance of related modalities. This dynamic fusion method not only improves the sensitivity of the model to diversified fault features, but also allows the system to automatically adjust the priority of each modal feature in different operating scenarios (such as engine, fuselage, etc.), thereby enhancing the applicability and accuracy of the model.

[0066] The data alignment uses timestamps to synchronize data between modalities and implements multimodal data alignment based on the dynamic time warping (DTW) algorithm to ensure the consistency of model input data in the time dimension. By identifying the temporal relationship between each modality in the historical data, a new time-aligned feature group is generated. This mechanism can capture the potential correlation between sensor, image and sound data during the alignment process.

[0067] The functions of the multimodal large model analysis module include time series processing and large model training;

[0068] The time series processing is to introduce a long short-term memory network (LSTM) to process time series data to capture dynamic information, and combine the global context-aware characteristics of the converter architecture to achieve understanding of long-term behavior and improve fault prediction accuracy;

[0069] The large model training is based on the labeled fault data set and the historical fault records of the specific model using self-supervised learning technology to train a deep neural network combining CNN and RNN to improve the generalization ability of fault recognition and prediction of the specific model under multi-modal input.

[0070] The functions of the real-time fault diagnosis and risk prediction module include real-time diagnosis and risk prediction;

[0071] The real-time diagnosis is to process multimodal data in real time and dynamically update model parameters through edge computing and incremental learning technology to improve the response speed of real-time fault detection;

[0072] The risk prediction utilizes time series analysis and outlier detection algorithms to identify potential failure trends in advance, assist in predicting possible risk components in the future, and ensure preventive maintenance.

[0073] The functions of the cloud-based data management module include cloud storage and sharing, cloud analysis and model updating, and data security;

[0074] The cloud storage and sharing is to store aircraft historical data and production batch information, and to achieve collaborative analysis by different inspection teams through the cloud;

[0075] The cloud-based analysis and model update is to use the high-performance computing resources in the cloud to conduct in-depth data analysis and iterative model updates, and improve the efficiency of data analysis and model optimization through automated tasks;

[0076] The data security is to implement multi-layer data encryption and access control to ensure the security of sensitive data in the cloud.

[0077] The functions of the user interaction and visualization module include real-time status display and inspection report generation;

[0078] The real-time status display is to display the real-time status and inspection progress of the aircraft, and to provide instant warning of abnormal situations, so as to support users to respond quickly;

[0079] The inspection report generation is that the system automatically generates an inspection report, which covers faulty components, cause analysis and recommended maintenance measures to facilitate subsequent maintenance and management decisions.

[0080] Embodiment 1:

[0081] Step 1: System Setup and Initialization

[0082] Hardware deployment: Multimodal sensors such as high-resolution cameras, infrared sensors, microphone arrays, vibration sensors, etc. are placed in key parts of the aircraft, including engines, landing gear, wings and other parts prone to problems. These sensors are connected to edge computing devices via a local area network to facilitate local data processing.

[0083] Edge computing devices: Edge computing devices deployed on the aircraft or ground maintenance center are equipped with sufficient processing power and storage space to support real-time data preprocessing and transmission. After completing basic preprocessing, the edge device uploads the data to the cloud system to ensure the stability and real-time nature of data flow.

[0084] Cloud Server: Connect the aircraft's edge devices and ground data centers to the cloud server. The cloud server is responsible for long-term data storage, in-depth analysis, and system optimization. The cloud has security protocols to protect data from unauthorized access.

[0085] Step 2: Data collection and processing

[0086] Data collection: The system collects data from various sensors in real time, including visual, infrared, acoustic and vibration data, and adds a timestamp to each piece of data to ensure the synchronization of multimodal data. During data collection, additional information such as specific aircraft models, production batches, historical inspection records, warranty and fault records are integrated into the database to form rich contextual information.

[0087] Data preprocessing: During the aircraft inspection, the collected data is transferred to the data processing module. The data preprocessing unit performs denoising, normalization, data enhancement and other processing on the raw data to ensure that the data quality is suitable for subsequent analysis.

[0088] The dynamic multi-modal fusion mechanism introduced here can adaptively adjust the fusion weights of each modal feature, improving the sensitivity of the model to diverse fault features. For details, see the data preprocessing module of the technical solution;

[0089] Multimodal data fusion: The pre-processed data is input into the multimodal data fusion unit, and the fusion algorithm is used to integrate the visual, infrared, acoustic and vibration data into a unified model input to provide more comprehensive aircraft status information.

[0090] Temporal alignment is introduced here to generate new time-aligned feature groups by identifying the temporal relationship between each modality in the historical data. This mechanism can capture the potential correlation between sensor, image and sound data during the alignment process.

[0091] Step 3: Large model training

[0092] Multimodal large model construction: Deep neural network models are trained in the cloud. The model structure includes convolutional layers for visual feature extraction, recurrent layers for processing time series information, and feature fusion layers for integrating data from different modalities. The training data comes from a large amount of annotated historical fault data so that the model can identify the association between multimodal features.

[0093] Model optimization and time series processing: The system adds historical data to the training process for incremental learning. Through time series data, the model learns specific behavior patterns during the operation of the equipment and enhances its ability to perceive potential risks. After training, the model will regularly fine-tune and learn online through new inspection data to continuously optimize the accuracy of fault prediction.

[0094] Model verification: Before actual application, the system conducts verification testing on the model and evaluates the model through a known fault data set to ensure that the model's fault recognition rate and false alarm rate meet the expected standards.

[0095] Step 4: Real-time troubleshooting

[0096] Data flow during the inspection process: The system acquires data in real time during the aircraft inspection process. The data collected by the sensor is processed by the edge device and transmitted to the large cloud model in real time. After each round of data transmission, the system generates a multimodal feature vector and inputs it into the model.

[0097] Real-time analysis and fault identification: After receiving new data, the large model uses a deep learning model to analyze the characteristics of each mode separately. For critical faults, the model uses a weighted fusion strategy to integrate the prediction results of each mode to generate the final fault identification result. The system completes the diagnostic analysis within seconds and determines whether an early warning needs to be generated.

[0098] Automatic warning and notification: If the model identifies an abnormal signal that exceeds the threshold, the system will automatically generate a warning message and push it to the maintenance personnel terminal. The warning message includes the specific component, possible cause of failure and recommended maintenance measures. The system will also automatically update the inspection record to ensure the complete maintenance history.

[0099] The equipment in this step is a cloud computing device. The aircraft collects real-time data, and the cloud server is responsible for processing the data. The cloud server can also combine the past historical records of all aircraft of the same type and model to make fault judgments.

[0100] Step 5: Cloud Data Management and Model Optimization

[0101] Data storage and sharing: All inspection data and diagnostic results are stored in the cloud for easy access and analysis by different maintenance teams, enabling data sharing and knowledge accumulation. The system also supports automated data analysis and learning processes, using new data to fine-tune the model and continuously improve the accuracy of fault detection.

[0102] Continuous learning and model updates: The system has online learning capabilities in the cloud, which can introduce new inspection data to achieve incremental updates and regularly retrain models to ensure improved accuracy and sensitivity of fault prediction. The data management module will perform regular backups and security checks to protect system data security.

[0103] Step 6: User interaction and visualization

[0104] Interface display: Users can view the aircraft's current status, historical inspection records, fault warnings and diagnostic reports through an intuitive graphical interface. The interface supports interactive analysis, and users can click on specific components to view detailed diagnostic information, such as fault type, related sensor data, and potential risks.

[0105] Decision support and report generation: When the system generates a fault diagnosis report, users can choose to archive the inspection data or use it as incremental learning data for the model. The system also supports the generation of printable reports, which contain detailed inspection data, historical fault comparisons, and system recommendations for technicians to refer to and make decisions.

[0106] This embodiment places the processing and analysis of the model in the cloud, and the aircraft side only needs to collect and upload data, thus significantly reducing the computing burden on the aircraft side. Cloud computing provides powerful computing power, which can comprehensively analyze a large amount of historical data of different types of aircraft, identify problems more accurately, and support the reception and transmission of real-time information;

[0107] In the data preprocessing, timestamps and rich aircraft information are added, including specific models, production batches, inspection records, warranty and fault records, and aircraft manual contents. The addition of this information not only facilitates the system to conduct comprehensive analysis, but also improves the model's fault prediction accuracy and decision support level;

[0108] A long short-term memory (LSTM) network is introduced to process time series data to capture dynamic information, and combined with the global context-aware characteristics of the converter architecture, it can achieve an understanding of long-term behavior and improve fault prediction accuracy.

[0109] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention; therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is limited by the attached claims rather than the above description. Therefore, it is intended to include all changes within the meaning and scope of the equivalent elements of the claims in the present invention, and any figure marks in the claims should not be regarded as limiting the claims involved.

[0110] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent aircraft inspection system based on a multimodal large model, including a multimodal data acquisition module, a data preprocessing module, a multimodal large model analysis module, a real-time fault diagnosis and risk prediction module, a cloud-based data management module and a user interaction and visualization module, characterized in that: The multimodal data acquisition module is used to integrate historical and production information of multiple sensors and aircraft, with the purpose of comprehensively acquiring and modeling the aircraft status; The data preprocessing module is used to perform specific preprocessing on each type of data, and to perform denoising, normalization, data enhancement and other processing on the collected raw data to improve the data quality. This includes but is not limited to processing vibration and acoustic data based on frequency domain filtering and adaptive noise elimination algorithms to reduce environmental noise and signal errors; The multi-modal large model analysis module is used to perform fault identification on the multi-modal large model; The real-time fault diagnosis and risk prediction module supports real-time processing to meet the needs of rapid feedback during inspections; The cloud-based data management module utilizes cloud computing to achieve centralized management and efficient analysis of data; The user interaction and visualization module provides visualization of inspection data through an interactive interface to improve user experience.

2. The intelligent aircraft inspection system based on multimodal large model according to claim 1 is characterized in that: The data collected by the multimodal data collection module includes but is not limited to visual data, infrared data, acoustic data, vibration data, aircraft information and time data; the visual data is collected by high-resolution cameras and hyperspectral cameras to collect image data of the outside and inside of the aircraft; The infrared data uses infrared sensors to detect heat distribution on the aircraft surface to identify potential overheating problems; the acoustic data uses acoustic sensors to monitor sound wave signals during aircraft operation to identify abnormal noise, and the acoustic sensors include microphone arrays; the vibration data uses vibration sensors to collect vibration data of various aircraft components to detect structural problems; the aircraft information includes specific models, production batches, inspection records, warranty and fault records, and aircraft manuals to provide in-depth background data to help large models identify potential problems of specific models and enrich the background information of multimodal data; the time data is equipped with synchronized timestamps for all modal data to ensure the temporal consistency of each modal information for accurate modeling and dynamic analysis.

3. The intelligent aircraft inspection system based on multimodal large model according to claim 1 is characterized in that: The functions of the data preprocessing module include feature extraction and data alignment. The feature extraction is to extract edge and texture features from visual data through convolutional neural networks, extract hot spots from infrared data through heat map segmentation, and extract spectrum features from acoustic data through Fourier transform. A dynamic multimodal fusion mechanism is adopted, and a weighting strategy based on the attention mechanism is introduced in the fusion unit, so that the system can adaptively adjust the fusion weight of each modal feature according to the content of real-time data and the importance of related modalities. This dynamic fusion method not only improves the sensitivity of the model to diversified fault features, but also allows the system to automatically adjust the priority of each modal feature in different operating scenarios, thereby enhancing the applicability and accuracy of the model. The data alignment uses timestamps to synchronize data between modalities and realizes multimodal data alignment based on a dynamic time warping algorithm to ensure the consistency of model input data in the time dimension. By identifying the temporal relationship between each modality in historical data, a new time-aligned feature group is generated. This mechanism can capture the potential correlation between sensor, image and sound data during the alignment process.

4. The intelligent aircraft inspection system based on multimodal large model according to claim 1 is characterized in that: The functions of the multimodal large model analysis module include time series processing and large model training; The time series processing is to introduce a long short-term memory network to process time series data to capture dynamic information, and combine the global context awareness characteristics of the converter architecture to achieve understanding of long-term behavior and improve fault prediction accuracy; The large model training is based on the labeled fault data set and the historical fault records of the specific model using self-supervised learning technology to train a deep neural network combining CNN and RNN to improve the generalization ability of fault recognition and prediction of the specific model under multi-modal input.

5. The intelligent aircraft inspection system based on multimodal large model according to claim 1 is characterized in that: The functions of the real-time fault diagnosis and risk prediction module include real-time diagnosis and risk prediction; The real-time diagnosis is to process multimodal data in real time and dynamically update model parameters through edge computing and incremental learning technology to improve the response speed of real-time fault detection; The risk prediction utilizes time series analysis and outlier detection algorithms to identify potential failure trends in advance, assist in predicting possible risk components in the future, and ensure preventive maintenance.

6. The intelligent aircraft inspection system based on multimodal large model according to claim 1 is characterized in that: The functions of the cloud-based data management module include cloud storage and sharing, cloud analysis and model updating, and data security; The cloud storage and sharing is to store aircraft historical data and production batch information, and to achieve collaborative analysis by different inspection teams through the cloud; The cloud-based analysis and model update is to use the high-performance computing resources in the cloud to conduct in-depth data analysis and iterative model updates, and improve the efficiency of data analysis and model optimization through automated tasks; The data security is to implement multi-layer data encryption and access control to ensure the security of sensitive data in the cloud.

7. The intelligent aircraft inspection system based on multimodal large model according to claim 1 is characterized in that: The functions of the user interaction and visualization module include real-time status display and inspection report generation; The real-time status display is to display the real-time status and inspection progress of the aircraft, and to provide instant warning of abnormal situations, so as to support users to respond quickly; The inspection report generation is that the system automatically generates an inspection report, which covers faulty components, cause analysis and recommended maintenance measures to facilitate subsequent maintenance and management decisions.

Images