Unmanned aerial vehicle fault prediction method, device and system based on multi-source data

Abstract

This application discloses a method, apparatus, and system for UAV fault prediction based on multi-source data. The method includes: acquiring multi-source UAV data; performing time-aligned preprocessing on the multi-source UAV data, and then performing weighted fusion and unified feature mapping on the multi-source UAV data to obtain multi-dimensional fused time-series data; based on a time attention mechanism and an adaptive weight mechanism, extracting at least one of local dynamic features, long-term time-series features, and short-term time-series features from the multi-dimensional fused time-series data to perform UAV fault prediction, obtaining UAV fault prediction results, which include fault probability prediction results and interpretability factor prediction results. The method of this application can perform UAV fault prediction based on fused and unified multi-source data, effectively improving the accuracy, real-time performance, and interpretability of UAV fault prediction.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) fault prediction, and in particular to a method, apparatus, and system for UAV fault prediction based on multi-source data. Background Technology

[0002] With the widespread application of unmanned aerial vehicle (UAV) technology in low-altitude economy, intelligent logistics, emergency rescue, geographic surveying and mapping, and military reconnaissance, the reliability and safety of UAV systems have become one of the key factors restricting the industry's development. During long-term operation, UAVs' power systems, energy systems, and flight control systems may experience varying degrees of performance degradation and potential malfunctions. These malfunctions are often covert, gradual, and multi-factor coupled; once they occur, they can lead to mission interruption or, in severe cases, flight crashes, causing serious economic losses and safety risks.

[0003] Current health management and fault prediction technologies for drones mainly rely on single-source data for fault prediction, which is difficult to adapt to the prediction needs of drones in multi-tasking and complex environments, resulting in poor prediction performance. Summary of the Invention

[0004] In view of the above problems, this application provides a method, apparatus and system for UAV fault prediction based on multi-source data, which can perform UAV fault prediction based on fused and unified multi-source data, effectively improving the accuracy, real-time performance and interpretability of UAV fault prediction.

[0005] Firstly, this application provides a method for UAV fault prediction based on multi-source data. The method includes: acquiring multi-source data of the UAV, which includes at least flight control system data, environmental sensor data, power system data, and mission payload data; performing time-aligned preprocessing on the multi-source data of the UAV, and performing weighted fusion and unified feature mapping processing on the multi-source data of the UAV to obtain multi-dimensional fused time-series data; and extracting at least one of local dynamic features, long-term time-series features, and short-term time-series features from the multi-dimensional fused time-series data based on a time attention mechanism and an adaptive weight mechanism to perform UAV fault prediction, thereby obtaining UAV fault prediction results, which include fault probability prediction results and interpretability factor prediction results.

[0006] In the technical solution of this application embodiment, firstly, multi-source data of the UAV is acquired. The multi-source data of the UAV includes at least flight control system data, environmental sensor data, power system data, and mission payload data. Then, the multi-source data of the UAV is preprocessed with time alignment, and weighted fusion and unified feature mapping are performed on the multi-source data of the UAV to obtain multi-dimensional fused time-series data. Finally, based on the time attention mechanism and adaptive weight mechanism, at least one of the local dynamic features, long-term time-series features, and short-term time-series features in the multi-dimensional fused time-series data is extracted to perform UAV fault prediction and obtain UAV fault prediction results. It can perform UAV fault prediction based on fused and unified multi-source data, which effectively improves the accuracy, real-time performance, and interpretability of UAV fault prediction.

[0007] In some embodiments, the method is applied to a UAV fault prediction model, which is obtained by fusing a temporal convolutional network and a gated recurrent unit. Based on a temporal attention mechanism and an adaptive weight mechanism, at least one of local dynamic features, long-term temporal features, and short-term temporal features is extracted from the multi-dimensional fused temporal data to perform UAV fault prediction and obtain UAV fault prediction results. The method includes: inputting multi-dimensional fused temporal data into the UAV fault prediction model; extracting local dynamic features through a temporal convolutional network; and extracting long-term and short-term temporal features through a gated recurrent unit; performing multi-task learning based on the local dynamic features, long-term temporal features, and short-term temporal features; and outputting fault probability prediction results and interpretability factor prediction results.

[0008] In some embodiments, the UAV multi-source data is preprocessed with time alignment, and then weighted fusion and unified feature mapping are performed on the UAV multi-source data to obtain multi-dimensional fused time-series data. This includes: performing unified time axis mapping on the UAV multi-source data using a spatiotemporal alignment algorithm to obtain multi-source aligned data; performing weighted fusion of the multi-source aligned data according to the importance of different data based on a multimodal feature fusion network to obtain multi-source fused data; and mapping the multi-source fused data to a unified feature space based on a feature semantic embedding network to obtain multi-dimensional fused time-series data.

[0009] In some embodiments, based on a time attention mechanism and an adaptive weighting mechanism, at least one of local dynamic features, long-term time series features, and short-term time series features is extracted from multi-dimensional fused time series data to perform UAV fault prediction and obtain UAV fault prediction results, including: performing UAV fault prediction through multiple sub-models based on an ensemble learning and model weighting strategy to obtain multiple sub-prediction results; and performing weighted fusion of the multiple sub-prediction results to obtain the UAV fault prediction result.

[0010] In some embodiments, the method further includes: generating a fault prediction trend curve based on the UAV fault prediction results and historical fault prediction results, and generating visualization information based on at least one of the fault prediction trend curve, the UAV fault prediction results, and the UAV real-time operation data.

[0011] In some embodiments, the method further includes: comparing the UAV fault prediction results with historical fault prediction results to calculate the risk level; and generating an early warning signal when the fault probability prediction result exceeds a preset probability threshold.

[0012] In some embodiments, after extracting at least one of the local dynamic features, long-term time series features, and short-term time series features from the multi-dimensional fused time series data based on the time attention mechanism and the adaptive weight mechanism to perform UAV fault prediction and obtain the UAV fault prediction result, the method further includes: adaptively updating the parameters of the UAV fault prediction model based on the fault feedback data according to the online learning and transfer learning framework.

[0013] On the other hand, this application provides a UAV fault prediction device based on multi-source data. The device includes: an acquisition module for acquiring multi-source data of the UAV, which includes at least flight control system data, environmental sensor data, power system data, and mission payload data; a processing module for performing time-aligned preprocessing on the multi-source data of the UAV, and performing weighted fusion and unified feature mapping processing on the multi-source data of the UAV to obtain multi-dimensional fused time-series data; and a prediction module for extracting at least one of local dynamic features, long-term time-series features, and short-term time-series features from the multi-dimensional fused time-series data based on a time attention mechanism and an adaptive weight mechanism to perform UAV fault prediction and obtain UAV fault prediction results, which include fault probability prediction results and interpretability factor prediction results.

[0014] On the other hand, this application provides a drone health prediction system, the system comprising: an edge device for executing the drone fault prediction method based on multi-source data according to any of the above embodiments to obtain drone fault prediction results; and a cloud server for communicating with the edge device and updating the parameters of the drone fault prediction model based on the drone fault prediction results.

[0015] On the other hand, this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any of the above embodiments.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a UAV fault prediction method based on multi-source data according to an embodiment of this application is shown; Figure 2 This illustration shows a schematic diagram of a UAV fault prediction system based on a multi-source data fusion and time-series prediction model according to an embodiment of this application. Figure 3 A flowchart illustrating the operation of the UAV fault prediction system according to an embodiment of this application is shown. Figure 4 A block diagram of a UAV fault prediction device based on multi-source data according to an embodiment of this application is shown; Figure 5 A schematic diagram of a drone health prediction system according to an embodiment of this application is shown; Figure 6 A block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0020] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0023] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0024] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0025] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0026] With the widespread application of unmanned aerial vehicle (UAV) technology in low-altitude economy, intelligent logistics, emergency rescue, geographic surveying and mapping, and military reconnaissance, the reliability and safety of UAV systems have become one of the key factors restricting the industry's development. During long-term operation, UAVs' power systems, energy systems, and flight control systems may experience varying degrees of performance degradation and potential malfunctions. These malfunctions are often covert, gradual, and multi-factor coupled; once they occur, they can lead to mission interruption or, in severe cases, flight crashes, causing serious economic losses and safety risks.

[0027] Current health management and fault prediction technologies for drones mainly rely on single-source data for fault prediction, which is difficult to adapt to the prediction needs of drones in multi-tasking and complex environments, resulting in poor prediction performance.

[0028] Specifically, current research on UAV fault prediction mainly focuses on the following three aspects: (1) Statistical analysis and threshold determination method based on single-source data These methods primarily rely on structured data from the flight control system (such as speed, voltage, current, and attitude angles) to determine whether anomalies exist in the system by setting thresholds or using simple regression models. Typical methods include: moving average anomaly detection, chi-square test, and Z-score monitoring (outlier detection).

[0029] However, such methods can only detect obvious anomalies, cannot capture potential degradation trends in the system, and are difficult to adapt to multivariate coupling relationships in complex environments.

[0030] (2) Feature extraction and model fitting methods based on signal processing These methods take vibration signals, temperature signals, or motor acoustic signals as the core, extract feature indicators through Fourier transform, wavelet packet decomposition, empirical mode decomposition (EMD), etc., and then use traditional machine learning models (such as support vector machine SVM, random forest, BP neural network) for classification or prediction.

[0031] However, such methods typically target only a single component or a single signal source, making it difficult to integrate multi-source data from flight control, sensors, environment, and mission levels, and also failing to fully model the dynamic evolution of time series data.

[0032] (3) Time series prediction and health assessment model based on deep learning With the development of deep learning, some methods utilize LSTM (Long Short-Term Memory Network), GRU (Gated Recurrent Unit), or CNN (Convolutional Neural Network) models to model flight process data and achieve Remaining Useful Life (RUL) prediction.

[0033] These methods outperform traditional statistical models in capturing temporal features, but still have shortcomings: the models usually rely on single-modal inputs (such as using only flight control log data); the sampling frequency and dimensionality of different data sources vary greatly, and there is a lack of a systematic multi-source fusion mechanism; the model structure is closed, lacking interpretability and online adaptive capability; and it cannot meet the general prediction needs of UAVs under multiple missions, multiple environments, and multiple aircraft types.

[0034] Currently, some methods rely on neural network prediction models based on time-series data from flight control systems. Examples include using LSTM networks to predict the health status of UAV motors and using sensor data to predict UAV faults. Their system architecture typically includes the following components: (1) Data acquisition module: collects basic data such as flight attitude, speed, current and voltage from the flight control system in real time.

[0035] (2) Data preprocessing module: performs filtering, normalization and time series processing.

[0036] (3) Feature extraction module: extracts data segments through a sliding window.

[0037] (4) Prediction model module: The model is trained based on a single deep learning network (such as LSTM) and is used to output the prediction results of health status.

[0038] (5) Result output module: Determine whether there is a potential fault based on the predicted probability.

[0039] These systems work by predicting equipment failures by identifying trends in characteristic changes (such as speed fluctuations and current anomalies) within the flight data stream. However, because they only use a single mode (such as flight control data) and do not comprehensively consider factors such as the external environment (wind speed, air pressure, temperature) and payload status (mission weight, mission type), the accuracy, stability, and generalization of the predictions are insufficient. Furthermore, this method cannot explain the causal mechanism of the failures and lacks engineering interpretability and adaptability.

[0040] In general, existing methods for monitoring the condition of unmanned aerial vehicles (UAVs) and predicting faults often suffer from the following problems: (1) Data level: Multi-source heterogeneous data have not been effectively integrated. The data types, sampling frequencies and semantics of different modules are significantly different, and there is a lack of a unified time series feature modeling framework.

[0041] (2) Model level: lacking the ability to collaboratively model deep temporal features and multimodal semantics, traditional network structures cannot simultaneously take into account local dynamic features and long-term evolutionary patterns.

[0042] (3) Prediction level: The model lacks interpretability and adaptability. The results of the existing model are black box, making it difficult for engineers to understand the reasons for the prediction and unable to automatically adjust the parameters according to changes in external working conditions.

[0043] (4) Application level: It lacks the ability to generalize across different models and multiple scenarios. The model is optimized for specific devices or datasets and is difficult to migrate to different drone platforms or mission scenarios.

[0044] In view of this, this application proposes a method, device and system for UAV fault prediction based on multi-source data, which can perform UAV fault prediction based on fused and unified multi-source data, effectively improving the accuracy, real-time performance and interpretability of UAV fault prediction.

[0045] In the technical solution of this application embodiment, firstly, multi-source data of the UAV is acquired. The multi-source data of the UAV includes at least flight control system data, environmental sensor data, power system data, and mission payload data. Then, the multi-source data of the UAV is preprocessed with time alignment, and weighted fusion and unified feature mapping are performed on the multi-source data of the UAV to obtain multi-dimensional fused time-series data. Finally, based on the time attention mechanism and adaptive weight mechanism, at least one of the local dynamic features, long-term time-series features, and short-term time-series features in the multi-dimensional fused time-series data is extracted to perform UAV fault prediction and obtain UAV fault prediction results. It can perform UAV fault prediction based on fused and unified multi-source data, which effectively improves the accuracy, real-time performance, and interpretability of UAV fault prediction.

[0046] Figure 1 A flowchart of a UAV fault prediction method based on multi-source data according to an embodiment of this application is shown.

[0047] like Figure 1 As shown, the UAV fault prediction method 100 based on multi-source data provided in this application includes steps S110 to S130.

[0048] Step S110: Acquire multi-source data from the UAV. The multi-source data from the UAV includes at least flight control system data, environmental sensor data, power system data, and mission payload data.

[0049] For example, UAV multi-source data may include flight control system data, environmental sensor data, power system data, mission payload data, and other multi-source heterogeneous data. For example, multi-dimensional data can be collected in real time from various sensors of the UAV (including IMU, GPS, barometer, motor speed sensor, battery monitoring module, temperature sensor, etc.) and external data sources (meteorological system, terrain data, mission control system) to obtain UAV multi-source data.

[0050] Step S120, perform preprocessing for time alignment on the multi-source data of the UAV, and perform weighted fusion and unified feature mapping processing on the multi-source data of the UAV to obtain multi-dimensional fusion time-series data.

[0051] Exemplarily, time alignment of the multi-source data of the UAV can be performed through high-frequency sampling and time synchronization mechanisms to ensure that each data stream is strictly aligned on the time axis, providing a consistent time benchmark for subsequent fusion and modeling; and based on the importance of different sensor signals, multi-modal feature extraction and weighted fusion are performed on the heterogeneous data from different sources (multi-source data of the UAV), and unified mapping is performed on the extracted features or fused features, so as to ensure the unity of the time series and feature semantics of the preprocessed multi-source data of the UAV and obtain multi-dimensional fusion time-series data.

[0052] Step S130, based on the time attention mechanism and the adaptive weight mechanism, extract at least one of the local dynamic features, long-term time-series features, and short-term time-series features in the multi-dimensional fusion time-series data for UAV fault prediction to obtain a UAV fault prediction result, where the UAV fault prediction result includes a fault probability prediction result and an interpretability factor prediction result.

[0053] Exemplarily, long-term time-series features and short-term time-series features in the multi-dimensional fusion time-series data can be extracted to capture long-term dependence relationships and short-term dynamic evolution laws from complex flight data, and local dynamic features can be extracted to capture multi-scale local dynamic changes in the flight data. At the same time, a time attention mechanism is introduced to identify "which second of the data in a past flight record can best predict a fault", strengthening the feature learning of key time periods, so as to perform UAV fault prediction based on complex time-series features. Among them, the interpretability and accuracy of the UAV fault prediction result are enhanced through the time attention mechanism and the adaptive weight mechanism. That is, on the one hand, reinforcement learning is performed on the key time-series features in the multi-dimensional fusion time-series data that affect the UAV fault prediction result. For example, if the UAV has a "severe jitter" at the 100th second, and the reason may be a "sudden voltage drop" at the 98th second, then when performing fault prediction, the attention to the data related to the 98th second (high weight) is strengthened, and the data related to the stable flight from the 50th to the 90th second (low weight) is ignored. On the other hand, visual analysis is performed on the main factors affecting the prediction result (such as relevant UAV indicators) to help engineers understand the prediction and quickly locate potential fault sources; the UAV fault prediction result can include a fault probability prediction result and an interpretability factor prediction result, and the interpretability factor prediction result can be the prediction result of the relevant UAV indicators that provide an interpretability analysis for the fault probability prediction result. For example, it can include the prediction result of the health index of each key component of the UAV.

[0054] In the technical solution of this application embodiment, firstly, multi-source data of the UAV is acquired. The multi-source data of the UAV includes at least flight control system data, environmental sensor data, power system data, and mission payload data. Then, the multi-source data of the UAV is preprocessed with time alignment, and weighted fusion and unified feature mapping are performed on the multi-source data of the UAV to obtain multi-dimensional fused time-series data. Finally, based on the time attention mechanism and adaptive weight mechanism, at least one of the local dynamic features, long-term time-series features, and short-term time-series features in the multi-dimensional fused time-series data is extracted to perform UAV fault prediction and obtain UAV fault prediction results. It can perform UAV fault prediction based on fused and unified multi-source data, which effectively improves the accuracy, real-time performance, and interpretability of UAV fault prediction.

[0055] In one example, the UAV fault prediction method based on multi-source data of this application is executed on a UAV fault prediction system based on a multi-source data fusion and time-series prediction model. The system realizes intelligent, interpretable and real-time prediction of UAV operating status by constructing a unified multi-source time-series data fusion framework, a deep adaptive prediction model and an online closed-loop health management module, which will be described in detail below.

[0056] Figure 2 A schematic diagram of a UAV fault prediction system based on a multi-source data fusion and time-series prediction model according to an embodiment of this application is shown.

[0057] like Figure 2 As shown, the UAV fault prediction system based on multi-source data fusion and time series prediction model includes a data acquisition module (data acquisition layer), a fusion module (data fusion layer and time series feature extraction layer), a time series prediction and health assessment module (fault prediction and health assessment layer), a model training and optimization module (feedback and optimization layer), and a fault early warning and visualization module (not shown in the figure).

[0058] Data acquisition module: Responsible for real-time acquisition of multi-dimensional data (UAV multi-source data) from various sensors of the UAV (including inertial measurement unit IMU, positioning system GPS, barometer, motor speed sensor, battery monitoring module, temperature sensor, etc.) and external data sources (meteorological system, terrain data, mission control system). Multi-dimensional data may include flight control system data, environmental sensor data, power system data, mission payload data, and other multi-source heterogeneous data.

[0059] This module ensures that each data stream is strictly aligned on the time axis through high-frequency sampling and time synchronization mechanisms, providing a consistent time reference for subsequent fusion and modeling.

[0060] Fusion Module: This module performs data preprocessing, data fusion, data synchronization and time alignment, and feature mapping on multi-source data from UAVs, and adopts a multi-layer fusion strategy of data-level fusion, feature-level fusion, and decision-level fusion.

[0061] Data-level fusion: Time alignment and noise suppression of raw data from multiple sensors are achieved through time window sliding and feature interpolation.

[0062] Feature-level fusion: An attention mechanism and a multimodal feature extraction and fusion network are introduced to adaptively weight the importance of signals from different sensors.

[0063] Decision-level fusion: By integrating learning and model weighting strategies, the prediction results of multiple sub-models are fused to obtain a more robust failure probability estimate.

[0064] This module extracts multimodal features from flight control data, sensor data, environmental data, and mission data, and performs high-precision fusion, which significantly improves the model's ability to capture features of complex operating states, reduces information loss and noise interference, and realizes cross-modal evolution modeling from "single-source dependence" to "multi-source collaboration".

[0065] The temporal prediction and health assessment module employs a hybrid prediction architecture (UAV failure prediction model) that integrates temporal convolutional networks (TCN) and gated recurrent units (GRU). It can also incorporate temporal attention mechanisms and adaptive weighting mechanisms. The input is a multi-dimensional time series (multi-dimensional fused temporal series data) that has undergone fusion processing, which is used for multi-task learning. This model not only predicts the probability of UAV failures but also predicts the interpretability factors of UAV failures. The output includes: health index of each key component, remaining usable lifespan at the time of failure, failure probability curve, and risk level assessment.

[0066] Meanwhile, the model has online learning and adaptive calibration capabilities, and can automatically update prediction parameters according to flight status and environmental changes.

[0067] This module uses multi-dimensional fused time-series data for UAV fault prediction, effectively improving the accuracy and robustness of fault trend identification. The model can be automatically optimized for different UAV types and different task loads. It enables global modeling and dynamic prediction of complex time series data.

[0068] Model training and optimization module: Based on the feedback mechanism of actual cases and the closed-loop system, it uses historical flight data and failure case data to carry out supervised and self-supervised hybrid training, and adopts comparative learning and anomaly detection mechanism to perform adaptive optimization and parameter update of the model, so that the model has strong generalization and robustness.

[0069] The model supports online dynamic parameter updates and achieves adaptive adjustments to different working conditions and task types through sliding window training and online distillation learning mechanisms. The model parameters are trained through a distributed computing framework to ensure computational efficiency in large-scale data scenarios.

[0070] This module combines transfer learning and domain adaptation algorithms to improve the model's generalization ability under different aircraft models and task conditions, reduce the training cost of the model in new aircraft models and new task scenarios, and realize a generalized UAV health prediction capability across platforms and tasks.

[0071] Fault warning and visualization module: Establish a health status visualization interface to dynamically display the real-time operating status, health index change trends and prediction results of the drone; at the same time, combined with threshold triggering and intelligent early warning mechanism, it automatically generates alarm signals when the predicted failure probability exceeds the set threshold, and supports the interpretable tracing of the cause of the failure.

[0072] This module introduces a visualization layer, enabling technical personnel to view the impact of changes in key parameters on the risk index, significantly enhancing the interpretability and engineering credibility of the model.

[0073] Figure 3 A flowchart illustrating the operation of the UAV fault prediction system according to an embodiment of this application is shown.

[0074] like Figure 3 As shown, the system operates according to the following process in actual operation: 1. Data acquisition stage: During flight, the various sensors of the UAV report data in real time. The timestamps are unified through the time synchronization module to achieve time synchronization and alignment of multi-source data of the UAV.

[0075] 2. Data preprocessing stage: Perform missing value imputation, noise filtering, feature standardization and time window segmentation operations on the multi-source data of UAVs.

[0076] 3. Fusion Modeling Stage: Using a multimodal feature extraction and fusion network, feature extraction, fusion, and adaptive feature adjustment are performed on the fused and time-aligned UAV multi-source data to generate a comprehensive feature vector.

[0077] 4. Temporal prediction stage: Based on the comprehensive feature vector, UAV fault prediction is performed through a hybrid prediction architecture (UAV fault prediction model) that integrates temporal convolutional network (TCN) and gated recurrent unit (GRU) to obtain UAV fault prediction results.

[0078] 5. Early warning and decision-making stage: Compare the drone failure prediction results with relevant data from historical patterns to calculate the risk level and trigger an early warning.

[0079] 6. Adaptive optimization stage: The system updates the model parameters based on the prediction error and actual feedback information to optimize the model, thereby improving long-term accuracy and stability.

[0080] Next, the paper describes in detail how to perform drone fault prediction based on the collected multi-source drone data, through a drone fault prediction system using multi-source data fusion and time-series prediction models.

[0081] For example, preprocessing of UAV multi-source data with time alignment is performed, and weighted fusion and unified feature mapping are performed on the UAV multi-source data to obtain multi-dimensional fused time-series data. For example, a spatiotemporal alignment algorithm is used to perform unified time axis mapping on the UAV multi-source data to obtain multi-source aligned data; and a multi-modal feature fusion network is used to perform weighted fusion of the multi-source aligned data according to the importance of different data to obtain multi-source fused data; and a feature semantic embedding network is used to map the multi-source fused data to a unified feature space to obtain multi-dimensional fused time-series data.

[0082] Specifically, to address the heterogeneity of data from different sensors (such as flight control, sensor, and environmental data), a multi-resolution time-series alignment algorithm (spatiotemporal alignment algorithm) can be used. By sliding the time window and interpolating features, a unified time axis mapping can be performed on data with different sampling frequencies to achieve time alignment and noise suppression of the original data, ensuring the feature synchronization of multi-source data.

[0083] A multimodal feature extraction and fusion network is used to extract and fuse features from time-aligned multi-source UAV data (multi-source aligned data). Simultaneously, an attention weight allocation mechanism is employed to dynamically adjust the contribution of different data sources in the prediction process. For example, for different dimensions of data from various airborne sensors (communication status, battery status, power status, etc.) and meteorological data (temperature, wind speed, etc.) within the multi-source UAV data, the attention weight allocation mechanism processes feature weights. The network can automatically learn relevant weight vectors, suppress irrelevant noise channels (weights approaching 0), amplify key anomaly channels (weights approaching 1), and then obtain a dynamically adaptive weight allocation result. For instance, in windy conditions, data from the air velocity meter and IMU (Inertial Measurement Unit) are crucial for determining attitude stability, automatically increasing the weight of relevant wind speed features. Furthermore, when hovering in calm conditions, the UAV's battery voltage and motor temperature may become core indicators for diagnosing faults (such as overheating or power failure), thus increasing the weight of relevant features. Feature-level weighted fusion of the multi-source aligned data is then performed to obtain multi-source fused data (such as the comprehensive feature vector generated above). Furthermore, a feature semantic embedding network is used to project heterogeneous data (such as multi-source fused data or separately extracted multimodal features) into a unified feature space, so that cross-modal features can be uniformly measured, and finally multi-dimensional fused time series data is obtained to achieve cross-modal feature association learning.

[0084] In the technical solution of this application embodiment, a spatiotemporal alignment algorithm is used to perform unified time axis mapping on UAV multi-source data to obtain multi-source aligned data, ensuring effective collaboration of all data sources; and based on a multimodal feature fusion network, the multi-source aligned data is weighted and fused according to the importance of different data to obtain multi-source fused data, breaking through the limitations of traditional single-sensor or static fusion methods, realizing cross-modal, dynamic adaptive data fusion. The deep fusion of multiple data sources provides more comprehensive and accurate input for fault prediction, solves the data isolation problem in the prior art, and significantly improves the accuracy and real-time performance of prediction; and based on a feature semantic embedding network, the multi-source fused data is mapped to a unified feature space to obtain multi-dimensional fused time series data, realizing cross-modal association learning, significantly improving the feature capture capability of the above-mentioned deep time series model for complex UAV operating states, reducing information loss and noise interference, and realizing cross-modal evolutionary modeling from "single-source dependence" to "multi-source collaboration".

[0085] For example, the method is applied to a UAV fault prediction model, which is obtained by fusing a temporal convolutional network and a gated recurrent unit. Based on a temporal attention mechanism and an adaptive weight mechanism, at least one of the local dynamic features, long-term temporal features, and short-term temporal features from the multi-dimensional fused temporal data is extracted to perform UAV fault prediction and obtain UAV fault prediction results. For example, firstly, the multi-dimensional fused temporal data is input into the UAV fault prediction model, and local dynamic features are extracted through a temporal convolutional network, while long-term and short-term temporal features are extracted through a gated recurrent unit. Then, multi-task learning is performed based on the local dynamic features, long-term temporal features, and short-term temporal features to output fault probability prediction results and interpretability factor prediction results.

[0086] Specifically, the UAV fault prediction model is based on the fusion of a Temporal Convolutional Network (TCN) and a Gated Recurrent Unit (GRU). For example, it can be a temporal deep learning architecture based on the aforementioned temporal attention mechanism and adaptive weight mechanism, which fuses a TCN and a GRU. The TCN module captures multi-scale local dynamic changes (local dynamic features) in multi-dimensional fused temporal data, while the GRU module extracts long-term dependencies (long-term temporal features) and short-term dynamic evolution patterns (short-term temporal features). Furthermore, based on the temporal attention mechanism, the model's learning ability for key time periods is enhanced. Additionally, a feature contribution matrix is ​​constructed within the model to clearly define each input... The system considers the impact of input variables on the prediction results. Then, based on the complex time-series features extracted above (i.e., multi-dimensional fused time-series data), it performs multi-task learning to output failure probability prediction results and interpretability factor prediction results. The failure probability prediction results include the predicted probability value of failure occurring during UAV operation. The interpretability factor prediction results represent the predicted values ​​of relevant indicators that provide interpretability analysis for the failure probability prediction results. For example, the interpretability factor prediction results can include the health index of each key component of the UAV and the remaining lifespan before failure. Thus, by calculating the health index and remaining lifespan, and combining them with a refined description of the causes of failure, the system not only provides failure prediction results but also reveals the underlying mechanisms of their occurrence.

[0087] In the technical solution of this application embodiment, multi-dimensional fused time-series data is first input into the UAV fault prediction model. Local dynamic features are extracted through a temporal convolutional network, and long-term and short-term time-series features are extracted through a gated recurrent unit. Then, multi-task learning is performed based on the local dynamic features, long-term time-series features, and short-term time-series features to output fault probability prediction results and interpretability factor prediction results. By using deep learning methods, the limitations of traditional models in capturing the health degradation process of aircraft are overcome. Through dynamic temporal modeling, the timing and evolution trend of fault occurrence can be accurately predicted, and potential risks can be warned in advance, significantly improving the accuracy of fault prediction and the system's adaptability. The accuracy and robustness of fault trend identification are also improved. At the same time, by introducing an interpretable AI framework, the problem caused by the "black box prediction model" in the prior art is solved. The interpretability of the prediction results enables UAV maintenance personnel to respond quickly to potential risks, improves the timeliness of fault handling, and enhances the transparency and credibility of the model, which helps to promote and use the model in practical engineering applications. This technical framework is not only applicable to UAV health prediction, but can also be extended to the field of fault diagnosis and prediction of other intelligent devices, improving domain adaptability.

[0088] For example, based on the time attention mechanism and the adaptive weighting mechanism, at least one of the local dynamic features, long-term time series features and short-term time series features in the multi-dimensional fused time series data is extracted to perform UAV fault prediction and obtain UAV fault prediction results. For example, firstly, based on the ensemble learning and model weighting strategy, UAV fault prediction is performed through multiple sub-models to obtain multiple sub-prediction results; then, the multiple sub-prediction results are weighted and fused to obtain the UAV fault prediction result.

[0089] Specifically, decision-level fusion of multi-source UAV data can be achieved through a multi-source data fusion module. For example, multiple sub-models are first trained, and then multi-dimensional fused temporal data is input into a temporal deep learning architecture derived from a fusion temporal convolutional network (TCN) and gated recurrent units (GRU) based on temporal attention and adaptive weighting mechanisms. The confidence weights of the TCN and GRU sub-models are calculated in real-time based on the degree of local mutation and long-term evolution trend of the features at the current moment. This allows for UAV fault prediction through multiple sub-models, resulting in multiple sub-prediction results. This enables automated dynamic decision switching for different fault modes (sudden / gradual). The sub-prediction results are then weighted and fused based on the weights of each sub-model to obtain the fused prediction result, i.e., the UAV fault prediction result. For example, when a UAV encounters a sudden gust of wind (short-term disturbance), the TCN sub-model's prediction is more accurate, increasing the weight of the TCN result (sub-prediction result); when the UAV battery slowly ages (long-term degradation), the GRU's prediction is more accurate, increasing the weight of the GRU result (sub-prediction result), thus obtaining a more robust fault probability estimate.

[0090] In the technical solution of this application embodiment, firstly, based on the ensemble learning and model weighting strategy, multiple sub-models are used to predict UAV faults, resulting in multiple sub-prediction results. Then, the multiple sub-prediction results are weighted and fused to obtain the UAV fault prediction result. This breaks through the limitations of traditional single-sensor or static fusion methods. Through deep fusion of multiple data sources, the accuracy and real-time performance of the prediction results are greatly improved.

[0091] For example, a health status visualization interface can also be established to dynamically display the real-time operating status, health index change trends and prediction results of the drone. For instance, a fault prediction trend curve can be generated based on the drone fault prediction results and historical fault prediction results, and visualization information can be generated based on at least one of the fault prediction trend curve, drone fault prediction results and drone real-time operating data.

[0092] Specifically, the UAV fault prediction results include interpretable factor prediction results such as fault probability prediction results and health index prediction results of key components. For example, a fault probability curve can be generated based on real-time fault prediction probabilities and historical fault prediction probabilities. Alternatively, a health index trend curve (fault prediction trend curve) can be obtained based on the real-time and historical health index prediction results of key components, thus yielding the fault prediction trend curve. The health index prediction result can be a predicted value of 0 to 1 representing the health status of the UAV, such as 0.85 representing a current predicted health level of 85%. Then, the real-time operating status of the UAV (relevant data on real-time UAV operation), the health index trend curve, the fault probability curve, or the real-time UAV fault prediction results are dynamically displayed through a health status visualization interface. UAV maintenance personnel can quickly respond to potential risks, improving the timeliness of fault handling.

[0093] In the technical solution of this application embodiment, a fault prediction trend curve is generated based on the UAV fault prediction result and the historical fault prediction result, and visualization information is generated based on at least one of the fault prediction trend curve, the UAV fault prediction result, and the UAV real-time operation data. This enables UAV operation and maintenance personnel to respond quickly to potential risks, improves the timeliness of fault handling, enhances the transparency and credibility of the model, and helps to promote and use the model in practical engineering applications.

[0094] For example, early warnings can also be issued based on the drone fault prediction results. For instance, first, the drone fault prediction results are compared with historical fault prediction results to calculate the risk level; then, when the fault probability prediction results exceed a preset probability threshold, a warning signal is generated.

[0095] Specifically, the predicted probability of drone failure can be compared with historical prediction results to assess the current risk level. Then, by combining threshold triggering and intelligent early warning mechanisms, an alarm signal can be automatically generated when the predicted failure probability (failure probability prediction result) exceeds a set threshold (preset probability threshold) to issue an early warning of failure.

[0096] In the technical solution of this application embodiment, the failure prediction result of the UAV is first compared with the historical failure prediction result to calculate the risk level. Then, when the failure probability prediction result exceeds the preset probability threshold, an early warning signal is generated. It can perform health assessment and failure prediction in real time during flight, thereby accurately predicting the timing and evolution trend of failure and giving early warning of potential risks.

[0097] For example, based on the time attention mechanism and the adaptive weight mechanism, at least one of the local dynamic features, long-term time series features, and short-term time series features in the multi-dimensional fused time series data is extracted to perform UAV fault prediction. After obtaining the UAV fault prediction result, the prediction model can be optimized through a feedback mechanism based on the fault warning result. For example, based on the online learning and transfer learning framework, the parameters of the UAV fault prediction model are adaptively updated according to the fault feedback data.

[0098] Specifically, online learning and transfer learning frameworks can be introduced into the UAV fault prediction model to continuously adjust model parameters based on real-time fault feedback data and new data. For example, the system can automatically update the health status model based on fault warning results and optimize the prediction model through a feedback mechanism, thereby improving the prediction accuracy and stability of the model in long-term operation. Fault feedback data can be, for example, prediction error data based on the UAV's operating status at the next moment. Simultaneously, the model can utilize historical flight data and fault case data for supervised and self-supervised hybrid training, employing comparative learning and anomaly detection mechanisms to maintain stable performance under different mission, environmental, and load conditions, and to achieve continuous learning and improvement in new operating conditions.

[0099] In the technical solution of this application embodiment, based on an online learning and transfer learning framework, the parameters of the UAV fault prediction model are adaptively updated according to fault feedback data. This enables dynamic adjustment of the prediction model based on real-time data feedback during flight, ensuring continuous updating and optimization of the prediction model. Through real-time feedback and online continuous learning mechanisms, the system maintains high accuracy and robustness throughout long-term operation and can operate effectively under various UAV types and flight mission scenarios. It not only provides prediction results but also continuously updates health records, forming a closed-loop intelligent system of "data acquisition—prediction—early warning—feedback—optimization". This provides technical support for the safety assurance of the entire life cycle of UAVs and forms a closed-loop management system for the entire life cycle of UAV health prediction, ensuring the complete process of the system from data acquisition to fault prediction to model optimization.

[0100] The following describes a specific example of fault prediction using a temporal deep learning architecture obtained by fusing TCN and GRU based on multi-source data from UAVs: Step 1: Receive multi-dimensional fused time-series data after raw data (multi-source data from UAVs) collection, time alignment, missing value imputation, and data fusion processing.

[0101] Step 2: Map the multidimensional fused time series data into a high-dimensional feature vector through the Embedding layer and output it.

[0102] Step 3: Receive the high-dimensional feature vector obtained in the previous step, and perform global information compression on the vector in sequence (perform global average pooling in the time dimension to obtain the global statistical features of each feature), and perform feature importance learning on the MLP network (fully connected neural network) consisting of two fully connected layers (construct different feature weight vectors, and multiply the generated weights back into the original input feature vector (high-dimensional feature vector) through a broadcast mechanism to suppress noise and achieve the effect of feature weighting).

[0103] Step 4: Further process local features using TCN, employing convolutional networks to capture short-term, local waveform features (such as current spikes, voltage drops, and sudden vibration changes). For example, a sudden drop in voltage or a microsecond-level spike in current are often early signs of drone malfunctions. The input to this step is the feature vector weighted by the feature dimensions from the previous step, and the output is a feature vector containing a "local waveform feature sequence" (representing microscopic changes at a local moment).

[0104] For example, a stacked causal dilated convolutional network architecture can be adopted. By introducing causal padding, the convolution operation is ensured to strictly follow the forward logic of the time series, and future information leakage is strictly prohibited. At the same time, the receptive field is effectively expanded by using the inflation coefficient that increases exponentially with the number of layers. Combined with the residual connection structure, the accurate capture and deep encoding of transient waveform changes and high-frequency dynamic features can be achieved within the local time window.

[0105] Step 5: Health status assessment. The feature vector containing the "local waveform feature sequence" obtained in the previous step is used as input, and then connected to a small fully connected neural network to output a specific health index numerical sequence (for example, a numerical sequence of 0 to 1, where 0.85 represents the current health level of 85%).

[0106] Step 6: Feature fusion. The feature information output from Step 4 and Step 5 is further merged, that is, the features containing the "local waveform feature sequence" are merged with the "index obtained from health status assessment", and a merged feature vector with more dimensions (more information) is output.

[0107] Step 7: Further process the long-term temporal evolution features using GRU. Use a unidirectional multilayer GRU to process the merged feature vector obtained in Step 6, focusing on capturing the degradation trend of long-term features (such as the process of slow decrease in battery voltage), and output the hidden state feature vector containing complete historical evolution information.

[0108] Step 8: Based on the time attention mechanism and adaptive weighting mechanism, automatically identify the "trigger moment" that caused the failure from past time windows, ignoring irrelevant data during smooth flight. For example, in the past 100 seconds, only the jitter at the 98th second may be fatal, while the rest of the time is spent in smooth flight. In this case, the feature weight of the 98th second is increased (see the relevant description in the above embodiment for details).

[0109] The input to this step is the feature vector of a certain period output by the GRU layer in step 7. The output is a weighted aggregated feature vector obtained by adaptively assigning feature weights to each time point from the temporal dimension after strengthening the "trigger moment" and weakening the "other moments". After this step, the time dimension is compressed and aggregated, and the output is a feature vector focusing on the key time window, extracting the most critical fault symptoms in the entire history.

[0110] Step 9: Final Prediction and Decision Layer, aiming at multi-perspective decision fusion. Based on different sub-models TCN and GRU, the weights for the final decision are adaptively allocated, integrating local mutation details, long-term evolutionary trends, and anomalies at critical moments, combining the cases of abrupt and long-term failures, to output the UAV failure prediction result. The input consists of the complete feature vector obtained in Step 8 and the local features of the TCN branch obtained in Step 4, and the output is the final prediction result (e.g., UAV failure prediction probability, remaining available time).

[0111] Figure 4 A block diagram of a UAV fault prediction device based on multi-source data according to an embodiment of this application is shown.

[0112] like Figure 4 As shown, this application provides a UAV fault prediction device 400 based on multi-source data. The device 400 includes: The acquisition module 410 is used to acquire multi-source data of the UAV, which includes at least flight control system data, environmental sensor data, power system data, and mission payload data.

[0113] The processing module 420 is used to perform time-aligned preprocessing on the multi-source data of the UAV, and to perform weighted fusion and unified feature mapping processing on the multi-source data of the UAV to obtain multi-dimensional fused time-series data.

[0114] The prediction module 430 is used to extract at least one of the local dynamic features, long-term time series features, and short-term time series features from the multi-dimensional fused time series data based on the time attention mechanism and the adaptive weight mechanism, so as to perform UAV fault prediction and obtain UAV fault prediction results. The UAV fault prediction results include fault probability prediction results and interpretability factor prediction results.

[0115] For example, the method is applied to a UAV fault prediction model, which is obtained by fusing a temporal convolutional network and a gated recurrent unit. The prediction module 430 is further configured to: input multi-dimensional fused temporal data into the UAV fault prediction model, extract local dynamic features through a temporal convolutional network, and extract long-term and short-term temporal features through a gated recurrent unit; perform multi-task learning based on the local dynamic features, long-term temporal features, and short-term temporal features, and output fault probability prediction results and interpretability factor prediction results.

[0116] For example, the processing module 420 is further configured to: perform unified timeline mapping on the UAV multi-source data through a spatiotemporal alignment algorithm to obtain multi-source aligned data; and perform weighted fusion of the multi-source aligned data according to the importance of different data based on a multimodal feature fusion network to obtain multi-source fused data; and map the multi-source fused data to a unified feature space based on a feature semantic embedding network to obtain multi-dimensional fused time-series data.

[0117] For example, the prediction module 430 is further configured to: perform UAV fault prediction through multiple sub-models based on ensemble learning and model weighting strategies to obtain multiple sub-prediction results; and perform weighted fusion of the multiple sub-prediction results to obtain the UAV fault prediction result.

[0118] For example, the device 400 further includes a first generation module, configured to: generate a fault prediction trend curve based on the UAV fault prediction result and historical fault prediction results, and generate visualization information based on at least one of the fault prediction trend curve, the UAV fault prediction result, and the UAV real-time operation data.

[0119] For example, the device 400 further includes a second generation module, used to: compare the UAV fault prediction result with the historical fault prediction result to calculate the risk level; and generate a warning signal when the fault probability prediction result exceeds a preset probability threshold.

[0120] For example, after extracting at least one of the local dynamic features, long-term time series features, and short-term time series features from the multi-dimensional fused time series data based on the time attention mechanism and adaptive weight mechanism to perform UAV fault prediction and obtain the UAV fault prediction result, the device 400 further includes a parameter update module for: adaptively updating the parameters of the UAV fault prediction model based on the fault feedback data according to the online learning and transfer learning framework.

[0121] Figure 5 A schematic diagram of a drone health prediction system according to an embodiment of this application is shown.

[0122] like Figure 5As shown, this application provides a drone health prediction system 500, which includes an edge device 501 and a cloud server 502.

[0123] Edge device 501 is used to execute the UAV fault prediction method based on multi-source data according to any of the above embodiments to obtain UAV fault prediction results; cloud server 502 is connected to edge device 501 for updating parameters of UAV fault prediction model based on UAV fault prediction results.

[0124] For example, edge device 501 is used to collect multi-source data from the UAV in real time and perform lightweight prediction to achieve rapid initial judgment during flight. Cloud server 502 is used to train the UAV fault prediction model, perform global data analysis and strategy distribution, and update model parameters based on real-time prediction results and feedback data to achieve continuous model iteration and optimization. The systems can be dynamically synchronized through adaptive communication protocols and model version control mechanisms to form an intelligent closed loop of "prediction, verification, and optimization". Thus, through the closed-loop health prediction system architecture of "edge-cloud collaboration", intelligent health management of the entire life cycle of the UAV is realized. Prediction results can be transmitted back in real time and support early warning of faults and optimization of flight scheduling, forming an automated and sustainable evolution system from data collection to model optimization.

[0125] In the technical solution of this application embodiment, drone fault prediction is performed through edge devices to obtain drone fault prediction results. Based on the drone fault prediction results, the parameters of the drone fault prediction model are updated by a cloud server, thereby constructing an "end-to-end closed-loop" drone health prediction system. This forms a closed loop of the entire process of real-time data acquisition, fault prediction, health assessment, early warning feedback, and model optimization. Through the collection and fusion of real-time sensor data, a complete flight data record is formed. Based on a deep learning model, the current flight status is predicted in real time, and fault warnings are issued in advance. Based on the fault warning results, the health status model is automatically updated, and the prediction model is optimized through a feedback mechanism.

[0126] In summary, the aforementioned UAV fault prediction method, device, and UAV health prediction system based on multi-source data, using UAV multi-source data and a UAV fault prediction model that integrates temporal convolutional networks and gated recurrent units, achieve the following significant technical effects: 1. Significantly improved prediction accuracy: By using a hybrid architecture that integrates temporal convolutional networks and gated recurrent units for UAV fault prediction, the prediction accuracy is significantly improved compared to the traditional single-source LSTM model.

[0127] 2. Enhanced early warning capability: Compared with existing technologies that identify obvious faults, this method can identify potential faults several flight cycles in advance.

[0128] 3. Strong cross-platform transfer capability: Combining transfer learning and domain adaptation algorithms, the model can maintain stable performance in multi-platform and multi-task environments.

[0129] 4. High level of system intelligence: An "edge cloud collaboration" drone health prediction system has been built, which supports online learning, real-time monitoring and optimization, forming an intelligent closed loop.

[0130] 5. Strong engineering feasibility: The architecture is compatible with UAV airborne systems and cloud-based operation and maintenance platforms, making it easy to integrate and promote.

[0131] Next, the irreplaceable nature of the method in this application will be explained. Without changing the overall multi-source data fusion and end-to-end prediction framework, the hybrid prediction architecture model fusing TCN and GRU can be replaced with other deep neural network combinations possessing temporal modeling capabilities, such as: 1. Combination of multilayer temporal convolutional networks (TCN) and bidirectional recurrent neural networks.

[0132] 2. Introduce a hierarchical recurrent network structure with multi-scale time windows.

[0133] This scheme achieves joint modeling of short-term dynamic features and long-term dependencies by combining different network modules, and has equivalent technical effects to the preferred implementation scheme in terms of prediction accuracy, real-time performance and stability.

[0134] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0135] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0136] Figure 6 A block diagram of an electronic device provided in an embodiment of this application.

[0137] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0138] like Figure 6 As shown, for ease of understanding, an embodiment of this application illustrates a specific electronic device 600.

[0139] Electronic device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 600 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0140] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded into random access memory (RAM) 603 from storage unit 608. RAM 603 may also store various programs and data required for the operation of electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 606 is also connected to bus 604.

[0141] Multiple components in electronic device 600 are connected to I / O interface 605. These components include: input unit 606, such as a keyboard or mouse; output unit 607, such as various types of displays or speakers; storage unit 608, such as a disk or optical disk; and communication unit 609, such as a network interface card (NIC), modem, or wireless transceiver. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0142] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods described above. For example, in some embodiments, any one or more of the methods described above can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of any one or more of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform any one or more of the methods described above by any other suitable means (e.g., by means of firmware).

[0143] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0144] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for predicting UAV faults based on multi-source data, characterized in that, The method includes: Acquire multi-source data from the UAV, which includes at least flight control system data, environmental sensor data, power system data, and mission payload data; The UAV multi-source data is preprocessed with time alignment, and then weighted fusion and unified feature mapping are performed on the UAV multi-source data to obtain multi-dimensional fused time-series data. Based on the time attention mechanism and adaptive weight mechanism, at least one of the local dynamic features, long-term time series features and short-term time series features in the multi-dimensional fused time series data is extracted to perform UAV fault prediction and obtain UAV fault prediction results. The UAV fault prediction results include fault probability prediction results and interpretability factor prediction results.

2. The UAV fault prediction method based on multi-source data according to claim 1, characterized in that, The method is applied to a UAV fault prediction model, which is obtained by fusing a temporal convolutional network and a gated recurrent unit. Based on a temporal attention mechanism and an adaptive weight mechanism, at least one of the following is extracted from the multi-dimensional fused temporal data: local dynamic features, long-term temporal features, and short-term temporal features, to perform UAV fault prediction and obtain the UAV fault prediction result, including: The multidimensional fused time-series data is input into the UAV fault prediction model, the local dynamic features are extracted through the temporal convolutional network, and the long-term and short-term time-series features are extracted through the gated recurrent unit. Multi-task learning is performed based on the local dynamic features, the long-term time-series features, and the short-term time-series features to output the fault probability prediction result and the interpretability factor prediction result.

3. The UAV fault prediction method based on multi-source data according to claim 1, characterized in that, The preprocessing of time alignment of the multi-source data from the UAV, followed by weighted fusion and unified feature mapping, yields multi-dimensional fused time-series data, including: A spatiotemporal alignment algorithm is used to perform unified timeline mapping on the multi-source data from the UAV to obtain multi-source aligned data; and Based on a multimodal feature fusion network, the multi-source aligned data is weighted and fused according to the importance of different data to obtain multi-source fused data; and Based on the feature semantic embedding network, the multi-source fused data is mapped to a unified feature space to obtain the multi-dimensional fused time series data.

4. The UAV fault prediction method based on multi-source data according to claim 1, characterized in that, The method, based on time attention and adaptive weighting, extracts at least one of the local dynamic features, long-term time-series features, and short-term time-series features from the multi-dimensional fused time-series data to perform UAV fault prediction, obtaining UAV fault prediction results, including: Based on ensemble learning and model weighting strategies, multiple sub-models are used to predict UAV faults, resulting in multiple sub-prediction results. The multiple sub-prediction results are weighted and fused to obtain the UAV fault prediction result.

5. The UAV fault prediction method based on multi-source data according to any one of claims 1-4, characterized in that, The method also includes: A fault prediction trend curve is generated based on the UAV fault prediction results and historical fault prediction results, and visualization information is generated based on at least one of the fault prediction trend curve, the UAV fault prediction results, and UAV real-time operation data.

6. The UAV fault prediction method based on multi-source data according to claim 5, characterized in that, The method also includes: The predicted failure results of the UAV are compared with the historical failure prediction results to calculate the risk level; When the predicted failure probability exceeds a preset probability threshold, an early warning signal is generated.

7. The UAV fault prediction method based on multi-source data according to claim 1, characterized in that, After extracting at least one of the local dynamic features, long-term time-series features, and short-term time-series features from the multi-dimensional fused time-series data based on the time attention mechanism and adaptive weight mechanism to perform UAV fault prediction and obtain the UAV fault prediction result, the method further includes: Based on online learning and transfer learning frameworks, the parameters of the UAV fault prediction model are adaptively updated according to fault feedback data.

8. A UAV fault prediction device based on multi-source data, characterized in that, The device includes: The acquisition module is used to acquire multi-source data from the UAV, which includes at least flight control system data, environmental sensor data, power system data, and mission payload data. The processing module is used to perform time-aligned preprocessing on the multi-source data of the UAV, and to perform weighted fusion and unified feature mapping on the multi-source data of the UAV to obtain multi-dimensional fused time-series data. The prediction module is used to extract at least one of the local dynamic features, long-term time series features, and short-term time series features from the multi-dimensional fused time series data based on the time attention mechanism and the adaptive weight mechanism, so as to perform UAV fault prediction and obtain UAV fault prediction results. The UAV fault prediction results include fault probability prediction results and interpretability factor prediction results.

9. A health prediction system for unmanned aerial vehicles (UAVs), characterized in that, The system includes: An edge device is used to execute the UAV fault prediction method based on multi-source data according to any one of claims 1-7, and obtain UAV fault prediction results; A cloud server communicates with the edge device to update the parameters of the UAV fault prediction model based on the UAV fault prediction results.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

Images