Intelligent diagnosis method and system for flight control fault of unmanned aerial vehicle based on digital-awareness fusion

Abstract

The invention relates to the technical field of fault diagnosis, in particular to an intelligent diagnosis method and system for a flight control fault of an unmanned aerial vehicle based on digital-awareness fusion, and the method comprises the steps: S1, obtaining the multi-modal data of the flight state of the unmanned aerial vehicle through an airborne sensor; s2, carrying out preprocessing on the multi-modal data; s3, constructing a flight control system intelligent fault diagnosis model based on the pre-training large model; s4, performing entity identification and relation extraction of the flight control fault of the unmanned aerial vehicle by using natural language processing; s5, constructing a knowledge graph of the flight control fault of the unmanned aerial vehicle based on the multi-modal data and the entity recognition result; and S6, fusing the fault identification result and the knowledge graph reasoning result to generate a fault diagnosis report. According to the method, the quantitative state recognition result output by the intelligent diagnosis model based on the large model is fused with the qualitative knowledge of the fault knowledge graph, so that the accuracy and the interpretability of the fault diagnosis result of the unmanned aerial vehicle flight control system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fault diagnosis, and specifically to an intelligent diagnosis method and system for UAV flight control faults based on digital and knowledge fusion. Background Art

[0002] As the application fields of UAVs become more and more extensive. Among them, the flight control system is a core component of UAVs. Once a fault occurs, it will seriously affect the flight safety of UAVs and may lead to disasters such as crashes and secondary accidents. Fault diagnosis technology can analyze flight data to timely identify faults and significantly improve the autonomous guarantee ability of UAVs.

[0003] For UAV flight control fault diagnosis, existing research has successively proposed methods based on analytical models, knowledge, and data-driven. Among them, the method based on the analytical model detects faults by comparing the constructed physical model with the output of the actual system, the knowledge-based method uses expert experience to discriminate faults, and the data-driven method uses machine learning and deep neural network models to automatically learn fault features from flight state data to achieve automatic fault identification. However, since UAV flight control is a typical complex system with multi-component coupling, it is far from enough to only obtain quantitative identification results of fault states. Existing methods ignore the further excavation of deep fault causes and are difficult to adopt effective and appropriate maintenance methods. Summary of the Invention

[0004] The present invention provides an intelligent diagnosis method and system for UAV flight control faults based on digital and knowledge fusion, which is used to realize intelligent fault diagnosis of UAV sensors, control surfaces, and servos. The present invention significantly improves the accuracy and interpretability of the fault diagnosis results of the UAV flight control system by fusing the quantitative state identification results output by the intelligent diagnosis model based on the large model with the qualitative knowledge of the fault knowledge graph.

[0005] The present application provides the following technical solutions: An intelligent diagnosis method for UAV flight control faults based on digital and knowledge fusion, comprising the following steps: S1. Use on-board sensors to obtain multi-modal data of the UAV flight state, and the on-board sensors include a global positioning system, a barometer, a gyroscope, and an airspeed indicator; S2. Preprocess the multi-modal data; S3. Construct an intelligent fault diagnosis model for the flight control system based on a pre-trained large model; S4. Use natural language processing to perform entity recognition and relationship extraction of UAV flight control faults; S5. Construct a knowledge graph of UAV flight control faults based on the multi-modal data and the entity recognition results; S6. Integrate the fault identification result and the knowledge graph reasoning result to generate a fault diagnosis report.

[0006] Technical principle: Collect multi-modal data during the flight of the UAV. After preprocessing, the fault identification result is obtained by an intelligent fault diagnosis algorithm based on a pre-trained large model. At the same time, according to the summary of UAV fault knowledge, a knowledge graph of flight control faults is established. Combine the fault identification result and the knowledge graph for fusion reasoning to obtain the UAV fault diagnosis report.

[0007] Beneficial effects: The present invention constructs an innovative intelligent fault diagnosis algorithm for the UAV flight control system based on a large model, optimizes for unstructured data, and can identify potential fault modes from complex multi-sensor data. Through deep learning technology, it realizes real-time monitoring and anomaly detection of the UAV operation status.

[0008] At the same time, the present invention constructs a fault diagnosis knowledge graph for the UAV flight control system, which transcends the limitation of traditional technologies that can only provide quantitative identification of fault states. By integrating multi-source heterogeneous data, this graph can deeply explore the underlying causes of faults and recommend the optimal maintenance strategy and historical reference cases. This method can enhance the understanding and analysis ability of complex faults, improve the maintenance efficiency and accuracy, and provide comprehensive knowledge support and technical guarantee for UAV maintenance.

[0009] In addition, the intelligent fault diagnosis method for UAV flight control faults of the present invention can realize the association mapping between the quantitative identification result output by the intelligent diagnosis large model and the qualitative fault instances in the fault diagnosis knowledge graph, further improving the accuracy and intelligence of UAV flight control system fault diagnosis.

[0010] Further, the intelligent fault diagnosis model of the flight control system includes a first fault diagnosis model and a second fault diagnosis model, and the S3 includes: S31. Establish a unified representation of multi-modal data; S32. Use the first fault diagnosis model based on reinforcement learning to identify UAV sensor faults; S33. Use the second fault diagnosis model based on contrast learning to identify faults of UAV control surfaces and servo bearings.

[0011] Further, the S32 includes: Select the Transformer structure to establish the first fault diagnosis model; Unify the encoding of the preprocessed time-series flight data; Define the state space in reinforcement learning; Define the action set according to the control system of the UAV; Design a reward and punishment function and a Q-value function to guide the first fault diagnosis model to learn the optimal strategy; Adjust the hyperparameters of the first fault diagnosis model according to the task scale and hardware resources; Adopt the Adam optimization algorithm combined with the adaptive learning rate adjustment mechanism to train the first fault diagnosis model; Package the trained first fault diagnosis model as a plugin and deploy it.

[0012] Beneficial effects: The fault diagnosis algorithm based on reinforcement learning not only improves the adaptability and learning ability of the model compared with traditional algorithms, but also does not require a large amount of manually labeled data as in traditional supervised learning.

[0013] Furthermore, the S33 includes: Select the Multimodal Transformer structure suitable for multimodal data to establish the second fault diagnosis model; Set the hyperparameters of the second fault diagnosis model according to the task requirements; Use the preprocessed multimodal data to construct positive and negative sample pairs for contrastive learning; Design a loss function to guide the second fault diagnosis model to learn to distinguish positive and negative samples; Adopt the Adam optimization algorithm combined with the self-learning rate warm-up and cosine annealing strategy to train the second fault diagnosis model; Package the trained second fault diagnosis model as a plugin and deploy it.

[0014] Beneficial effects: The fault diagnosis algorithm based on contrastive learning can improve the accuracy and robustness of fault diagnosis in the case of lack of labeled data, unbalanced data and unknown fault patterns through self-supervised learning, effective feature extraction and enhanced generalization ability.

[0015] Furthermore, the S4 includes: S41. Obtain the text data of the UAV flight control fault; S42. Extract key entities from the text data through natural language processing technology; S43. Extract the semantic relationships between entities; S44. Map the identified entities and relationships to the knowledge graph to form a complete graph structure.

[0016] Beneficial effects: By identifying entities and establishing relationships, the fault knowledge data is transformed into structured knowledge, providing support for intelligent applications.

[0017] Furthermore, the S6 includes: S61. Parse the key information in the flight log through natural language processing and pattern recognition technologies; S62. Obtain the graph reasoning result through the knowledge graph; S63. Map the inference result to the fault instance in the knowledge graph; S64. Use the inference result of the knowledge graph as the supplementary input of the intelligent fault diagnosis model for the flight control system to generate a final diagnosis report, where the diagnosis report includes fault phenomena, diagnostic analysis, maintenance solutions, and historical cases.

[0018] Beneficial effects: By working together with the knowledge graph and the large model, their respective advantages are fully utilized. The combination of the two can significantly improve the accuracy and efficiency of fault diagnosis.

[0019] Further, in S63, a semantic mapping function based on a mathematical model and a conceptual model is established to realize the mapping between the different fault intelligent recognition results of multiple components in the flight control system and the fault instances in the knowledge graph. The mathematical model is constructed by extracting fault features from flight state data to identify different types of faults:

[0020] In the formula, MR represents the recognition result output by the data-driven intelligent diagnosis model, X is the collected fault data, F is the fault feature vector automatically extracted by the model from X and is the intelligent diagnosis algorithm used; The conceptual model uniformly represents the knowledge graph of the faults of the UAV flight control system:

[0021] Among them, C, R, A, I respectively represent the fault diagnosis class set, relationship set, function operation, axiom set, and instance; The knowledge graph stores the structured knowledge in the following form:

[0022] Among them and respectively represent the entity set and the relationship set, The semantic mapping function is defined as follows:

[0023] Among them, is an element of the model recognition result; is an element of the fault phenomenon instance.

[0024] On the other hand, the present invention also provides an intelligent diagnostic system for the flight control failure of a data-aware integrated unmanned aerial vehicle (UAV), which includes a data acquisition module, a data storage module, a data preprocessing module, a fault identification module, a knowledge graph module, a fault fusion reasoning module, and a system management module. Specifically: The data acquisition module is used to acquire multi-modal flight data and transmit the acquired data to the data storage module; The data storage module is used to establish a database for the acquired multi-modal flight data, provide data management capabilities for the system, and support data sharing and analysis at the same time; The data preprocessing module preprocesses the acquired flight data; The fault identification module constructs an intelligent fault diagnosis model for the flight control system by using a large-scale pre-trained model based on Transformer, extracts features from the preprocessed data, and realizes rapid identification and accurate judgment of fault modes. It includes a multi-modal data conversion sub-module, a first fault identification module, and a second fault identification module. The multi-modal data conversion sub-module is used to establish a unified representation of multi-modal data; the first fault identification module uses a fault diagnosis algorithm based on reinforcement learning to detect UAV sensor faults in real time; the second fault identification module uses a fault diagnosis algorithm based on contrast learning to detect faults of UAV control surfaces and servo bearings in real time; The knowledge graph module is used to construct a knowledge graph of UAV flight control faults, including an entity recognition sub-module, a knowledge graph construction sub-module, and a knowledge graph storage sub-module. The entity recognition sub-module uses natural language processing to perform entity recognition and relationship extraction of UAV flight control faults; the knowledge graph construction sub-module constructs all entities and relationships into a structured knowledge network; the knowledge graph storage sub-module imports the structured knowledge network into ArangoDB, defines indexes, provides query scripts written based on AQL, supports dynamic updates at the same time, and provides an API interface for integration with other systems; The fault fusion reasoning module uses the knowledge graph to enhance the intelligent fault diagnosis model of the flight control system for knowledge reasoning and fault diagnosis. The knowledge reasoning and fault diagnosis include: first, parsing the key information in the flight log through natural language processing and pattern recognition technologies to automatically identify potential fault sources; then, further deriving the causal relationship between the fault phenomenon and the fault location through the structured relationships in the knowledge graph; feeding back the graph reasoning result to the intelligent fault diagnosis model of the flight control system, and generating a final diagnostic report by integrating the results of fault identification and graph reasoning. The diagnostic report includes fault phenomena, diagnostic analysis, repair solutions, and historical cases; The system management module is used to establish a data security guarantee system and a standardization system, and realize the secure storage of data and the standardized operation of the system. It mainly includes a user management unit, a system log unit, and a device management unit. The user management unit assigns corresponding permissions to different roles, and the roles include operators, system administrators, and UAV maintenance personnel; the system log unit establishes the standardized management of data and real-time monitors the operation logs and usage of the system; the device management unit formulates corresponding management documents for UAVs of different models and configurations.

[0025] Furthermore, the first fault identification module selects the Transformer structure to establish the first fault diagnosis model, uniformly encodes the preprocessed time-series flight data in the database, defines the state space in reinforcement learning, defines the action space according to the control system of the UAV, designs the reward and punishment function and Q-value function for guiding the model to learn the optimal strategy, adjusts the hyperparameters of the Transformer model according to the task scale and hardware resources, trains the first fault diagnosis model using the Adam optimization algorithm combined with the adaptive learning rate adjustment mechanism, and packages the trained first fault diagnosis model as a plugin and deploys it to the system background.

[0026] Furthermore, the second fault identification module selects the Multimodal Transformer model structure to establish the second fault diagnosis model, sets the hyperparameters of the second fault diagnosis model according to the task requirements, constructs positive and negative sample pairs for contrast learning using the preprocessed multimodal data, designs the loss function for guiding the model to learn to distinguish positive and negative samples, and uses the Adam optimization algorithm combined with the self-learning rate warm-up and cosine annealing strategy to optimize the contrast loss and the Transformer model parameters; packages the trained second fault diagnosis model as a plugin and deploys it. Description of the Drawings

[0027] Figure 1 It is a schematic diagram of a method for intelligent diagnosis of UAV flight control faults with digital and knowledge fusion; Figure 2 It is a schematic diagram of the construction process of an intelligent fault diagnosis model for a flight control system based on a pre-trained large model; Figure 3 It is a schematic diagram of an entity recognition and relationship extraction method; Figure 4 It is a schematic diagram of the construction result of a knowledge graph; Figure 5 It is a schematic diagram of the process of a fusion inference fault diagnosis method; Figure 6 It is a schematic diagram of the architecture of a system for intelligent diagnosis of UAV flight control faults with digital and knowledge fusion. Detailed Implementation Modes

[0028] Typical faults of the UAV flight control system include sensor faults, control surface faults, and servo faults. Among them, the on-board sensors of the UAV work in a harsh service environment and usually have various fault forms; the control surfaces of the UAV respond to the control instructions of the flight control computer frequently and in real time during flight, and will inevitably fail under the influence of external environments such as atmospheric turbulence; the servos are mainly used to control the attitude and direction of the UAV, with a relatively complex structural composition and various fault forms such as electrical faults and mechanical faults.

[0029] The present invention provides a method and system for intelligent diagnosis of UAV flight control faults based on data fusion, which is used to realize the intelligent fault diagnosis of UAV sensors, control surfaces, and servos, can significantly reduce the adverse impact of limited fault data on the performance of the intelligent diagnosis model, and solves the problems of insufficient intuitiveness and low level of hierarchy in UAV fault diagnosis.

[0030] The following is a further detailed description through specific embodiments: Embodiment 1 This embodiment provides a method for intelligent diagnosis of UAV flight control faults based on data fusion, as Figure 1 shown, including the following steps: S1. Use on-board sensors to acquire and store multi-modal data of the UAV flight state; Various on-board sensors of the UAV include the Global Positioning System (GPS), barometer, gyroscope, airspeed indicator, etc. These on-board sensors collect various flight state parameters such as position type (e.g., altitude, longitude, and latitude), speed type (e.g., airspeed and ground speed), and angle type (e.g., pitch angle, heading angle, and roll angle) in real time during flight. These data are stored through the on-board data recorder or transmitted to the ground station in real time through wireless communication for display and storage.

[0031] The ground station establishes a database for the collected multi-modal flight data, providing reliable and efficient data management capabilities for the system, and at the same time supporting data sharing and analysis. It includes various methods such as reading the on-board data recorder and receiving the collected data in real time through wireless communication.

[0032] S2. Preprocess the multi-modal data; The multi-modal data contains various types of signals (such as vibration, temperature, pressure, speed, angle, etc.) from different sensors or devices. These signals have different characteristics, sampling rates, and noise values. In order to effectively extract available information and perform subsequent analysis, signal processing methods are needed to preprocess the multi-modal data.

[0033] The signal preprocessing method includes multiple steps such as data cleaning, time-domain / frequency-domain analysis, correlation analysis, normalization, and dimensionality reduction. Among them, the data cleaning step is used to identify and correct errors and noises in the data, including methods such as removing outliers, filling in missing values, and repairing outliers. The time-domain / frequency-domain analysis step extracts time characteristics and frequency characteristics from multi-modal data to reveal the dynamic behavior of the signal. For example, wavelet transform or short-time Fourier transform is used to extract local time-domain features for time series signals such as speed and angle, and fast Fourier transform is used to decompose vibration signals into frequency components. The time-domain and frequency-domain features of different modal signals can be combined for correlation analysis of trend features. Correlation analysis can screen out features that contribute more to fault mode recognition and help understand the interaction between multi-modal data, including intra-modal correlation analysis methods and inter-modal correlation analysis methods. For example, autocorrelation analysis is performed on vibration signals to detect periodic features, and cross-correlation analysis is used to determine the time lag relationship between different modal signals. The normalization step scales different numerical ranges of multi-modal data to a unified specific range, usually using min-max normalization or Z-score standardization analysis methods. The dimensionality reduction step reduces the data dimension by reducing the number of features and tries to retain important feature information, which can reduce the computational cost and avoid overfitting. Usually, the principal component analysis method can be used.

[0034] S3. Build an intelligent fault diagnosis model for the flight control system based on a pre-trained large model.

[0035] The intelligent fault diagnosis model of the flight control system includes a first fault diagnosis model and a second fault diagnosis model. The first fault diagnosis model is used to identify faults in the UAV sensors, and the second fault diagnosis model is used to identify faults in the UAV control surfaces and servo bearings. As Figure 2 shown, S3 includes the following specific steps: S31. Establish a unified representation of multi-modal data, including: Use WordPiece of BERT to convert the processed multi-modal data into tokens to achieve a unified representation of multi-modal data, providing a basis for pre-training; Map the tokens to the indices in the vocabulary, providing a basis for cross-modal learning of the model; Adopt masked language modeling. By masking part of the input data, the model can learn bidirectional context information and enhance the model's ability to understand semantics.

[0036] S32. Use the first fault diagnosis model based on reinforcement learning to identify faults in the UAV sensors, including: Select the Transformer structure to build the first fault diagnosis model.

[0037] Uniformly encode the pre - processed time - series flight data in the database, capture the dependency relationships of sensor data through the multi - layer attention mechanism of the Transformer, and output a unified vector representation.

[0038] Define the state space in reinforcement learning, and clarify the information observed by the model at each time step: According to the flight mission requirements, select key flight parameters as state variables, take the encoded results output by the Transformer as part of the state space, and combine other auxiliary information (such as weather conditions, target positions) to construct a complete state vector, and determine the dimension and range of the state space.

[0039] Define the action space: Define the action set according to the control system of the UAV, ensure that the action space covers all possible control options, and avoid redundant actions to improve training efficiency.

[0040] Design the reward - punishment function and Q - value function to guide the model to learn the optimal policy and select the optimal action.

[0041] Select the hyperparameters of the first fault diagnosis model: Adjust the hyperparameters of the model according to the task scale and hardware resources to ensure the balance between the generalization ability and training efficiency of the model. The hyperparameters include the number of layers, the dimension of the hidden layer, the number of attention heads, etc. Among them, the number of layers of the model determines the depth of the model; the dimension of the hidden layer, that is, the number of neurons in each layer, affects the expression ability of the model; the number of attention heads represents the parallel computing units of the self - attention mechanism, which can enhance the model's ability to focus on different sub - spaces.

[0042] Train the first fault diagnosis model: Use the Adam optimization algorithm, combined with the adaptive learning rate adjustment mechanism, to accelerate the convergence of the model. Gradually increase the learning rate at the beginning of training to avoid model instability caused by too high an initial learning rate; then use a linear decay or cosine annealing strategy to reduce the learning rate to further optimize the model performance. The Adam optimization algorithm and the learning rate warm - up strategy work together to ensure a smooth and efficient training process. Continuously optimize the parameters through iteration to improve the decision - making ability of the model.

[0043] After training is completed, save the model weights and related configuration files, package the first fault diagnosis model into a standard plug - in format (such as Python package, API interface, etc.) and upload it to the system background, and test the performance of the plug - in in the actual flight environment to ensure its stability and reliability.

[0044] S33. Use the second fault diagnosis model based on contrastive learning to identify faults in the UAV's control surface and servo bearing: Select the Multimodal Transformer structure suitable for multimodal data to establish the second fault diagnosis model; Set the hyperparameters of the model according to the task requirements, including the number of layers, the dimension of hidden units, the number of attention heads, the input embedding dimension, etc.; Use the preprocessed multi-modal data in the database for training, including: Construct positive and negative sample pairs for contrastive learning, including generating positive sample pairs from the normal state data collected by the same drone at different time points, and generating negative sample pairs from the data of different drones or different fault states. The positive and negative sample pairs should each have similar semantic information, such as flight conditions, fault modes, etc.

[0045] Design a loss function to guide the model to learn to distinguish positive and negative samples: The contrastive loss function can be measured using the InfoNCE loss function or cosine similarity.

[0046] Adopt the Adam optimization algorithm for gradient descent, combine the self-learning rate warm-up and cosine annealing strategies, and dynamically adjust the learning rate. Input the positive and negative sample pairs into the model to calculate the contrastive loss, calculate the gradient according to the loss function, backpropagate and update the model parameters. Monitor the change trend of the contrastive loss value, and record the model to ensure its gradual convergence. Adjust the model parameters through the optimization process to reach the optimal, and improve the feature extraction ability and fault recognition accuracy of the model.

[0047] After the training is completed, save the model weights and related configuration files, package the second fault diagnosis model into a standard plugin format (such as Python package, API interface, etc.) and upload it to the system background, and test the performance of the plugin in the actual flight environment to ensure its stability and reliability.

[0048] S4. Use natural language processing for entity recognition and relationship extraction of UAV flight control faults, such as Figure 3 shown.

[0049] The knowledge graph provides a basic framework for the organization, sharing, and reasoning of knowledge by formally defining concepts, attributes, and relationships within a domain. Its functions include two aspects: one is to serve as a unified knowledge representation method to facilitate the transfer and use of domain knowledge between different applications; the other is to provide a logical reasoning mechanism that enables the computer to automatically reason and query new knowledge in the domain.

[0050] Entities are the basic units in the knowledge graph, representing objects or concepts in the real world. Relationships describe the associations between entities and are the core part of the knowledge graph. By identifying entities and establishing relationships, the knowledge graph can transform massive data into structured knowledge and provide support for intelligent applications.

[0051] The specific steps of S4 include: S41. Obtain the text data of UAV flight control faults; Obtain the text data of the UAV flight control failure through multiple channels, including flight logs, maintenance records, user feedback, technical documents, expert reports, online communities and forums, historical databases, etc.

[0052] S42. Extract key entities from the text data through NLP technology; Use NLP technology to identify key entities from the text data of UAV flight control failures, such as component names, failure types, operating parameters, etc. It includes specific steps such as named entity recognition, constructing a domain dictionary, extracting entities through regular expressions or keyword matching, and eliminating ambiguity through context analysis.

[0053] S43. Extract the semantic relationships between entities; By extracting the relationships between entities, the root cause of the failure and its propagation path can be revealed. The specific process includes named entity recognition, constructing the features of entity relationships, classifying relationships through machine learning methods, and post-processing the extracted relationships according to application requirements. This step can be completed through the support vector machine algorithm or deep learning method that relies on manually designed features.

[0054] S44. Map the identified entities and relationships to the knowledge graph to form a complete graph structure.

[0055] Take each entity as a node in the knowledge graph and label its category; Create edges between entities according to the extracted semantic relationships and label the relationship types;

[0056] Align and fuse the newly extracted entities and relationships with the existing knowledge graph; Finally, check whether the graph is complete and eliminate redundant or incorrect connections.

[0057] S5. Construct and save the knowledge graph of UAV flight control failures; The construction of the knowledge graph depends on graph modeling, and its main process is divided into defining entities (i.e., nodes) and establishing relationships (i.e., edges). The graph modeling process includes the following steps: S51. Establish entity classification in the knowledge graph: According to the core elements in the analysis of flight control system failure knowledge, four main categories are established: failure phenomenon, failure location, failure cause, and failure repair. Among them, the meaning of the failure phenomenon category is the phenomenon directly observed when the failure occurs; the failure location category describes the part where the failure occurs; the failure cause category describes the reason for the failure; the failure repair category describes the repair measures that need to be taken to eliminate the failure.

[0058] S52. Define relationships according to the interaction or influence between entities: The defined relationship types include causal relationship, inclusion relationship, location relationship, and solution relationship.

[0059] Causal relationship: Describes the causal association between the cause of a fault and the fault phenomenon.

[0060] Inclusion relationship: Indicates that an entity is a part of another entity.

[0061] Location relationship: Specifies the occurrence location of the fault phenomenon or cause.

[0062] Solution relationship: Connects the fault phenomenon or cause with the corresponding repair measures.

[0063] S53. Add attributes to classes or instances to describe the characteristics and properties between them; S54. Use ArangoDB to store the constructed knowledge graph.

[0064] The multi-model feature of ArangoDB is very suitable for storing and managing knowledge graphs, and can support efficient querying and analysis.

[0065] Import the extracted entities and relationships into ArangoDB and define indexes. You can write query scripts using AQL, and it also supports dynamic updates and provides API interfaces for integration with other systems.

[0066] The finally constructed knowledge graph is as Figure 4 shown.

[0067] S6. Integrate the fault identification results and the knowledge graph reasoning results to generate a fault diagnosis report.

[0068] The knowledge graph provides precise structured knowledge and causal reasoning capabilities, including the repair solutions and historical cases of corresponding faults, while the large model has accurate fault identification capabilities. The combination of the two can significantly improve the accuracy of the fault diagnosis report. The process is as Figure 5 shown and includes the following steps: S61. Parse the key information in the flight log through natural language processing and pattern recognition technologies to automatically identify potential fault sources; This step is to extract key information from the flight log and automatically identify potential fault sources.

[0069] First, use natural language processing (NLP) technology to parse the unstructured text data in the flight log, extract keywords, parameter anomaly records, and event descriptions; then analyze the time series data in the flight log to discover abnormal patterns or metrics that deviate from the normal range; finally, combine the relevant working principles in the UAV field and match the extracted information with known fault characteristics to initially locate possible fault sources.

[0070] S62. Obtain the graph reasoning result through the knowledge graph.

[0071] This step utilizes the structured relationships in the knowledge graph to further deduce the causal relationship between the fault phenomenon and the fault location. First, search for entities (such as components, functional modules) related to potential faults and their associated relationships in the knowledge graph. Second, based on the causal chain in the graph, deduce possible fault propagation paths. Finally, apply logical reasoning algorithms (such as path-based reasoning or rule-based reasoning) to verify the rationality of different hypotheses. Eventually, generate the graph reasoning result, clearly reflecting the causal relationship between the fault phenomenon and the fault location.

[0072] S63. Map the reasoning result to the fault instance in the knowledge graph; By establishing a mathematical model and a conceptual model, and establishing a semantic mapping function for the mathematical model and the conceptual model, the mapping between the different fault intelligent recognition results of multiple components in the flight control system and the fault instances in the knowledge graph is realized.

[0073] Among them, the mathematical model is constructed by extracting fault features from flight state data to identify different types of faults, and its specific definition is as follows:

[0074] Among them, MR represents the recognition result output by the data-driven intelligent diagnosis model; X is the collected fault data; F is for the model to X automatically extract the fault feature vector from; is the intelligent diagnosis algorithm used.

[0075] The conceptual model uses the five elements of the ontology to uniformly represent the knowledge graph of the UAV flight control system faults:

[0076] Among them, C, R, A, I respectively represent the fault diagnosis class set, relationship set, function operation, axiom set, and instance.

[0077] The knowledge graph stores the structured knowledge in the following form:

[0078] Among them and respectively represent the entity set and the relationship set.

[0079] Based on the above-established mathematical model and conceptual model, a semantic mapping function :

[0080] Among them, is an element of the model recognition result; is an element of the fault phenomenon instance.

[0081] Under this mapping rule, there may be five different mapping relationships between the model recognition result and the fault instance: mapping failure, unique mapping, one recognition result mapping multiple instances, multiple instances mapping one recognition result, and multiple recognition results mapping multiple instances. Therefore, in the actual operation process, corresponding measures such as adding constraints and updating the recognition method need to be taken to ensure the accuracy of the final mapping S64. Use the graph reasoning result as the supplementary input of the flight control system intelligent fault diagnosis model, and comprehensively generate the final diagnosis report based on the fault recognition result and the graph reasoning result.

[0082] In this step, the reasoning result of the knowledge graph is used as the supplementary input and provided to the large model. The large model combines its own ability to understand the context, evaluates and integrates the fault recognition result and the graph reasoning result, and generates a more accurate conclusion. For example, the priority order of the fault cause list can be adjusted according to the reasoning result to highlight the most likely fault cause. Or historical cases and maintenance experience references can be extracted according to the reasoning result.

[0083] Comprehensively combine the fault recognition result and the graph reasoning result, compare multiple possible fault hypotheses, select the optimal solution as the final conclusion, and generate the final diagnosis report. Clearly display the fault phenomenon, possible causes, diagnosis basis, and repair plan in the diagnosis report, and provide additional information such as historical similar cases and maintenance experience references to help users better understand and perform the repair work.

[0084] Embodiment 2 This embodiment provides a digital and knowledge fusion intelligent diagnosis system for UAV flight control faults, covering the complete process from data collection, storage, analysis to system management. This system is developed using the Python language, runs on the Windows operating system, and uses PyQt5 for the graphical user interface design. In terms of the database, a lightweight embedded database SQLite is used to store the flight status parameters collected on the same day, and at the same time, an ArangoDB multi-model system is used to store the historical monitoring data of all devices and other relevant information.

[0085] The intelligent diagnosis system for UAV flight control faults includes a data collection module, a data storage module, a data preprocessing module, a fault recognition module, a knowledge graph module, a fault fusion reasoning module, and a system management module. Its structure is as Figure 6 shown.

[0086] The data acquisition module is used to acquire multi-modal flight data and transmit the acquired data to the data storage module. Multiple flight state parameters such as position (e.g., altitude, longitude, and latitude), speed (e.g., airspeed and ground speed), and angle (e.g., pitch angle, heading angle, and roll angle) of the UAV during flight are obtained through various on-board sensors. These data are stored by the on-board data recorder or transmitted to the ground station in real time via wireless communication for display and storage. The on-board sensors include a Global Positioning System (GPS), a barometer, a gyroscope, an airspeed indicator, etc., which are respectively used to acquire flight parameters such as the position information, barometric altitude, angular velocity, and indicated airspeed of the UAV.

[0087] The data storage module is used to establish a database for the acquired multi-modal flight data, providing the system with efficient and secure data management capabilities, and at the same time supporting data sharing and analysis. The data storage module includes various methods such as reading the on-board data recorder and receiving the acquired data in real time via wireless communication.

[0088] The data preprocessing module preprocesses the acquired flight data to improve data quality and effectively simplify the complexity of subsequent calculations. It includes a data cleaning unit, a time-domain / frequency-domain analysis unit, a correlation analysis unit, a normalization unit, and a dimensionality reduction unit. Among them, the data cleaning unit is used to identify and correct errors and noise in the data; the time-domain / frequency-domain analysis unit is used to extract time characteristics and frequency characteristics from multi-modal data to reveal the dynamic behavior of the signal; the correlation analysis unit is used to screen out features that contribute more to fault mode recognition to help understand the interaction between multi-modal data; the normalization unit is used to scale different numerical ranges of multi-modal data to a unified specific range; the dimensionality reduction unit is used to reduce the number of features to reduce the data dimension and try to retain important feature information, reducing the computational cost and avoiding overfitting.

[0089] The fault identification module constructs an intelligent fault diagnosis model for the flight control system using a large-scale pre-trained model based on Transformer, extracts features from the preprocessed data, and realizes the rapid identification and accurate judgment of fault modes, including a multi-modal data conversion sub-module, a first fault identification module, and a second fault identification module. Among them: The multi-modal data conversion sub-module realizes the unified representation of multi-modal data.

[0090] The first fault identification module uses a fault diagnosis algorithm based on reinforcement learning to detect UAV sensor faults in real time.

[0091] The second fault identification module uses a fault diagnosis algorithm based on contrast learning to detect faults in the UAV's control surfaces and servo bearings in real time.

[0092] The construction and training process of the fault identification module is as described in Example 1.

[0093] The knowledge graph module is used to construct a knowledge graph of UAV flight control faults, which can effectively improve the efficiency of fault diagnosis and repair, and enhance the safety and intelligence level of the system. It includes an entity recognition sub-module, a knowledge graph construction sub-module, and a knowledge graph storage sub-module.

[0094] The entity recognition sub-module uses natural language processing technology to identify key entities from the text data describing UAV flight control faults, such as component names, fault types, operating parameters, etc. Extract the semantic relationships between entities, map the identified entities and relationships into the knowledge graph, and form a complete graph structure.

[0095] The knowledge graph construction sub-module stores all entities and relationships in the graph structure as nodes and edges, then creates a graph structure in the graph database and defines necessary attributes for each node and edge; then maps specific instances such as fault phenomena, locations, causes, repair methods, etc. to classes to form a structured knowledge network.

[0096] The knowledge graph storage sub-module imports the structured knowledge network into ArangoDB and defines indexes, provides query scripts written based on AQL, and also supports dynamic updates and provides API interfaces for integration with other systems.

[0097] The fault fusion inference module includes a knowledge graph inference module and a large model inference module. Its role is to establish and apply inference rules, fuse the fault recognition results and the fault instances in the knowledge graph, establish rules formulated by mathematical models and conceptual models to satisfy the mapping of different fault intelligent recognition results of multiple components in the flight control system to the fault instances in the knowledge graph, and deduce potential fault causes and corresponding repair methods. Among them: By extracting fault features from flight state data to identify different types of faults, the following general mathematical model is constructed:

[0098] Among them, MR represents the recognition result output by the intelligent diagnosis model driven by small sample data; X is the collected fault data; F is the fault feature vector automatically extracted by the model from X ; f(X) is the intelligent diagnosis algorithm used.

[0099] The conceptual model uniformly represents the faults of the UAV flight control system:

[0100] Among them, C, R, O, A, Irespectively represent the fault diagnosis class set, the relationship set, the function operation, the axiom set, and the instance.

[0101] Considering the diversity of UAV flight missions and service conditions, in order to meet the mapping of the identification results of different faults of multiple components in the flight control system and the corresponding fault instances, based on the established mathematical model and conceptual model, a semantic mapping function is defined :

[0102] Among them, is an element of the model identification result; is an element of the fault phenomenon instance.

[0103] With the support of the data security guarantee system and the standardization system, the system management module realizes the secure storage of data and the standardized operation of the system, and mainly includes a user management unit, a system log unit, and a device management unit. The user management unit assigns corresponding permissions to different roles to ensure the security of system information. The roles mainly include operators, administrators specified by the system, and maintenance personnel of UAVs. The system log unit standardizes the editing, deletion, and storage of data and monitors the operation logs and usage of the system in real time to ensure the normal operation and maintenance of the system. The device management unit formulates corresponding management documents for UAVs of different models and configurations to meet the needs of various UAV flight control fault diagnoses.

[0104] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this embodiment case, and common knowledge such as the specific structures and characteristics known in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicability of the patent. The protection scope required by this application should be subject to the content of its claims, and the specific implementation manners described in the specification can be used to interpret the content of the claims.

Claims

1. A method for intelligent diagnosis of UAV flight control faults based on data-information fusion, characterized by: The steps include: S1. Acquire multimodal data of the flight status of the UAV using airborne sensors, wherein the airborne sensors include a global positioning system, a barometer, a gyroscope, and an airspeed meter; S2, preprocessing multimodal data; S3. Build an intelligent fault diagnosis model for the flight control system based on the pre-trained large model; S4. Use natural language processing to perform entity recognition and relationship extraction of UAV flight control failures; S5. Construct a knowledge graph of UAV flight control failures based on multimodal data and entity recognition results; S6. Integrate the fault identification results and knowledge graph reasoning results to generate a fault diagnosis report.

2. According to the method of intelligent diagnosis of UAV flight control faults based on data-information fusion according to claim 1, it is characterized by: The flight control system intelligent fault diagnosis model includes a first fault diagnosis model and a second fault diagnosis model, and S3 includes: S31. Establish a unified representation of multimodal data; S32, identifying UAV sensor faults using a first fault diagnosis model based on reinforcement learning; S33. Use a second fault diagnosis model based on contrastive learning to identify faults of the UAV rudder surface and servo bearings.

3. The method for intelligent diagnosis of UAV flight control faults based on data-information fusion according to claim 2 is characterized by: The S32 includes: Select the Transformer structure to establish the first fault diagnosis model; Uniformly encode the pre-processed time-series flight data; Define the state space in reinforcement learning; Define a set of actions based on the drone’s control system; Design reward and punishment functions and Q-value functions to guide the first fault diagnosis model to learn the optimal strategy; Adjusting hyperparameters of the first fault diagnosis model according to task scale and hardware resources; The first fault diagnosis model is trained by using the Adam optimization algorithm combined with an adaptive learning rate adjustment mechanism; The trained first fault diagnosis model is packaged as a plug-in and deployed.

4. The method for intelligent diagnosis of UAV flight control faults based on data-information fusion according to claim 2 is characterized by: The S33 includes: Select the Multimodal Transformer structure suitable for multimodal data to establish the second fault diagnosis model; Setting hyperparameters of the second fault diagnosis model according to task requirements; Use preprocessed multimodal data to construct positive and negative sample pairs for contrastive learning; Design a loss function that guides the second fault diagnosis model to learn to distinguish between positive and negative samples; The Adam optimization algorithm is used in combination with the self-learning rate warm-up and cosine annealing strategy to train the second fault diagnosis model; The trained second fault diagnosis model is packaged as a plug-in and deployed.

5. The method for intelligent diagnosis of UAV flight control faults based on data-information fusion according to claim 1 is characterized by: The S4 includes: S41, obtaining text data of the UAV flight control failure; S42. Extract key entities from text data through natural language processing technology; S43, extracting semantic relations between entities; S44. Map the identified entities and relationships into the knowledge graph to form a complete graph structure.

6. The method for intelligent diagnosis of UAV flight control faults based on data-information fusion according to claim 1 is characterized by: The S6 includes: S61. Analyze key information in flight logs through natural language processing and pattern recognition technology; S62. Obtain graph reasoning results through the knowledge graph; S63, mapping the reasoning result to the fault instance in the knowledge graph; S64. Using the graph reasoning result as a supplementary input of the intelligent fault diagnosis model of the flight control system to generate a final diagnosis report, wherein the diagnosis report includes the fault phenomenon, diagnosis analysis, maintenance plan and historical cases.

7. The method for intelligent diagnosis of UAV flight control faults based on data-information fusion according to claim 6 is characterized by: The S63 establishes a semantic mapping function based on a mathematical model and a conceptual model to achieve mapping of different fault intelligent identification results of multiple components in the flight control system with fault instances in the knowledge graph. The mathematical model is constructed by extracting fault features from the flight status data to identify different types of faults: In the formula, MR represents the recognition result output by the data-driven intelligent diagnosis model, X is the collected fault data, and F is the fault feature vector automatically extracted by the model from X. Intelligent diagnostic algorithms for use; The conceptual model uses the five elements of the ontology to uniformly represent the knowledge graph of UAV flight control system failures: Among them, C, R, A, and I represent the fault diagnosis class set, relation set, function operation, axiom set, and instance respectively; The knowledge graph stores structured knowledge in the following form: in and Represent entity sets and relationship sets respectively; The semantic mapping function The definition is as follows: in, Identify elements of results for the model; An element that is an instance of a fault phenomenon.

8. A data-information-integrated UAV flight control fault intelligent diagnosis system, characterized by: It includes data acquisition module, data storage module, data preprocessing module, fault identification module, knowledge graph module, fault fusion reasoning module and system management module, among which: The data acquisition module is used to collect multi-modal flight data and transmit the collected data to the data storage module; The data storage module is used to establish a database for the collected multi-modal flight data, provide data management capabilities for the system, and support data sharing and analysis; The data preprocessing module preprocesses the collected flight data; The fault identification module uses a large-scale pre-trained model based on Transformer to build an intelligent fault diagnosis model for the flight control system, extracts features from pre-processed data, and realizes rapid identification and accurate judgment of fault modes. It includes a multimodal data conversion submodule, a first fault identification module, and a second fault identification module. The multimodal data conversion submodule is used to establish a unified representation of multimodal data. The first fault identification module uses a fault diagnosis algorithm based on reinforcement learning to detect UAV sensor faults in real time. The second fault identification module uses a fault diagnosis algorithm based on contrastive learning to detect UAV rudder and steering gear bearing faults in real time. The knowledge graph module is used to construct a knowledge graph of UAV flight control failures, including an entity recognition submodule, a knowledge graph construction submodule, and a knowledge graph storage submodule. The entity recognition submodule uses natural language processing to perform entity recognition and relationship extraction of UAV flight control failures; the knowledge graph construction submodule constructs all entities and relationships into a structured knowledge network; the knowledge graph storage submodule imports the structured knowledge network into ArangoDB and defines indexes, provides query scripts written based on AQL, and also supports dynamic updates and provides API interfaces for integration with other systems; The fault fusion reasoning module uses the knowledge graph to enhance the intelligent fault diagnosis model of the flight control system to perform knowledge reasoning and fault diagnosis. The knowledge reasoning and fault diagnosis include: firstly, parsing the key information in the flight log through natural language processing and pattern recognition technology to automatically identify the potential root cause of the fault; then, through the structured relationship in the knowledge graph, further deriving the causal relationship between the fault phenomenon and the fault location; feeding back the graph reasoning results to the intelligent fault diagnosis model of the flight control system, integrating the results of fault identification and graph reasoning, and generating a final diagnosis report, which includes the fault phenomenon, diagnostic analysis, maintenance plan and historical cases; The system management module is used to establish a data security system and a standard normalization system to achieve secure data storage and standardized operation of the system. It mainly includes a user management unit, a system log unit and a device management unit. The user management unit assigns corresponding permissions to different roles, including operators, system administrators and drone maintenance personnel; the system log unit establishes standardized data management and monitors the system's operation logs and usage in real time; the device management unit formulates corresponding management documents for drones of different models and configurations.

9. The intelligent diagnosis system for UAV flight control faults based on data-information fusion according to claim 8 is characterized by: The first fault identification module selects the Transformer structure to establish a first fault diagnosis model, uniformly encodes the pre-processed time-series flight data in the database, defines the state space in reinforcement learning, defines the action space according to the control system of the UAV, designs the reward and punishment function and Q-value function to guide the model to learn the optimal strategy, adjusts the hyperparameters of the Transformer model according to the task scale and hardware resources, and uses the Adam optimization algorithm combined with the adaptive learning rate adjustment mechanism to train the first fault diagnosis model. The trained first fault diagnosis model is encapsulated as a plug-in and deployed to the system background.

10. The intelligent diagnosis system for UAV flight control faults based on data-information fusion according to claim 8 is characterized by: The second fault identification module selects the Multimodal Transformer model structure to establish a second fault diagnosis model, sets the hyperparameters of the second fault diagnosis model according to task requirements, uses preprocessed multimodal data to construct positive and negative sample pairs for contrast learning, designs a loss function to guide the model to learn to distinguish between positive and negative samples, and uses the Adam optimization algorithm combined with self-learning rate preheating and cosine annealing strategy to optimize the contrast loss and Transformer model parameters; encapsulates the trained second fault diagnosis model as a plug-in and deploys it.

Images