Knowledge graph-based intelligent fault inquiry system and method for energy power device

Abstract

The invention belongs to the technical field of aviation energy design, and particularly provides an aviation energy power device fault intelligent inquiry system and method based on a knowledge graph. The method comprises the following steps: firstly, showing a complex data structure and a knowledge relationship to a user in a visual and dynamic manner through knowledge extraction, knowledge graph database construction, knowledge graph visualization and other methods, and constructing a knowledge graph basic framework; according to the method, the depth information of the target object can be extracted from the images from different angles, the fault characteristics of the equipment can be intuitively displayed, and the fault state of the equipment can be visually represented by extracting the depth information of the target object from the images from different angles. The evolution of the equipment state can be tracked through the change of the three-dimensional space; a conversion algorithm of the knowledge graph and the Bayesian network is realized through the Bayesian network, and rapid reasoning of fault causes is realized in combination with a Bayesian network reasoning algorithm; and finally, based on an intelligent inquiry system of a large model, a deep learning technology and a large-scale training model are utilized, and previous fault data and professional knowledge of the aviation energy power device are combined to provide intelligent auxiliary diagnosis support.

Description

Technical Field

[0001] The present invention belongs to the technical field of aviation energy system design, and specifically is an intelligent diagnosis system for aviation energy power plant faults based on knowledge graph technology and fault three-dimensional model reconstruction technology. Background Art

[0002] Aviation energy power plant fault diagnosis technology plays an important role in the fields of industry, transportation, aviation, etc. Its main purpose is to timely detect and eliminate faults in the operation of aviation energy power plants to ensure the stability and safety of the equipment. Traditional aviation energy power plant fault diagnosis methods mainly rely on manual experience and rule judgment. Maintenance personnel analyze the operating data (such as vibration, temperature, emissions, etc.) to judge the health status of aviation energy power plants. However, with the increasing complexity of aviation energy power plant systems, the types of faults are diversified and hidden, making traditional empirical methods inefficient and error-prone in diagnosing complex faults.

[0003] In traditional energy and power plant fault diagnosis, it is usually based on expert experience and rule-based reasoning methods, which are often limited by the scope of expert knowledge and experience, and may be inefficient when dealing with multi-source data and complex faults. Traditional energy and power plant fault detection methods, such as spectrum analysis and time-frequency analysis, mainly rely on vibration, temperature and other signals collected by sensors, which can reflect the overall operating status of energy and power plants, but it is difficult to accurately locate the specific fault location or to display it visually. Summary of the invention

[0004] Purpose of the invention: The invention aims to provide an intelligent fault diagnosis system for aviation energy power plant based on knowledge graph. The system adopts multimodal knowledge extraction algorithm and knowledge graph visualization, combined with Bayesian network reasoning algorithm and fault three-dimensional point cloud model reconstruction technology, to better meet the intelligent fault diagnosis and maintenance needs of aviation energy power plant in multiple scenarios.

[0005] Technical solution of the present invention: To achieve the above-mentioned purpose, the present invention adopts the following technical solution:

[0006] On the one hand, the present invention proposes an intelligent diagnosis method for energy power device faults based on knowledge graph, and the method comprises the following steps:

[0007] Step S1: extract knowledge from multi-source data (such as text, images, audio, video, etc.) using a multimodal knowledge extraction algorithm, establish a correlation relationship between electromechanical system behaviors, and construct a comprehensive knowledge graph;

[0008] Step S2: further visualize the knowledge graph established in step S1. Knowledge graph visualization can make the graph more intuitive through multi-dimensional display and layout optimization. By combining deep learning and graph neural network (GNN), the reasoning path is visualized to make the reasoning process transparent, thereby enhancing the application value of knowledge graph in intelligent analysis.

[0009] Step S3: Bayesian network reasoning, by constructing nodes and dependencies, using conditional probability tables to perform fault diagnosis; combining known observation data to perform forward or backward reasoning to infer the cause of the fault;

[0010] Step S4, reconstructing the three-dimensional point cloud model of the fault, using a camera to collect a two-dimensional image or point cloud data of the object; then, converting the data into a three-dimensional point cloud through feature matching and triangulation, and using an algorithm (such as Delaunay triangulation) to reconstruct a three-dimensional surface model.

[0011] Furthermore, in step S1, the multimodal knowledge extraction algorithm specifically includes the following steps: data collection preprocessing, unimodal feature extraction, cross-modal feature alignment, entity recognition and relationship extraction, multimodal fusion and knowledge representation, knowledge reasoning verification and knowledge graph update.

[0012] Furthermore, the multimodal knowledge extraction algorithm for constructing the aviation energy power plant fault diagnosis knowledge graph mainly includes:

[0013] Unimodal feature extraction: Unimodal feature extraction extracts representative features from a single type of data, including text features, image features, audio features, point cloud features, and time series features. These feature extraction methods can capture important information in the data and provide a basis for subsequent analysis and modeling.

[0014] Cross-modal feature alignment: Cross-modal feature extraction aligns and fuses data features of different modalities (such as text, image, audio, etc.) to extract features in a shared semantic space.

[0015] Multimodal fusion and knowledge representation: Multimodal fusion integrates different data modalities through early, late, and mid-term fusion, and uses joint embedding, graph neural networks and other methods to unify knowledge representation to enhance semantic understanding and reasoning capabilities.

[0016] Furthermore, in step S2, the knowledge graph visualization processing specifically includes the following steps: data preparation and graph construction, interactive function design, search function, multi-dimensional display, knowledge reasoning and path display, visualization optimization and user feedback.

[0017] Furthermore, in step S3, the Bayesian network reasoning algorithm is mainly used to perform fault diagnosis by constructing nodes and dependencies and using conditional probability tables. Through forward or backward reasoning, the probabilities of other nodes are calculated in combination with known observation data to infer the cause of the fault. It can dynamically update node probabilities and is suitable for fault diagnosis and uncertainty management of complex systems, with good interpretability and flexibility.

[0018] Furthermore, in step S4, the three-dimensional point cloud model of the fault is reconstructed, which specifically includes the following steps: point cloud data collection of the target object, feature extraction and matching, three-dimensional point cloud generation, point cloud alignment and fusion, three-dimensional surface reconstruction Delaunay triangulation, and fault feature mapping.

[0019] Furthermore, the aviation energy power plant fault three-dimensional point cloud model reconstruction technology mainly includes:

[0020] Collection and representation of point cloud data: Point cloud data can be collected through a variety of methods such as LiDAR, camera, structured light scanning, etc. LiDAR uses laser beams to scan the environment and measures the reflection time to calculate the distance of the object. Point cloud data is usually represented as a set of points P = {p i}, each point p i Contains its three-dimensional coordinate information.

[0021] Point cloud registration: Point cloud registration refers to aligning point cloud data from multiple sources or multiple perspectives to generate a complete 3D model. Common point cloud registration methods include the iterative closest point (ICP) algorithm. The ICP algorithm is used to align the source point cloud to the target point cloud. The basic idea is to minimize the square distance between the points of the two sets of point clouds. The formula is as follows:

[0022]

[0023] Among them, p i and q i are the corresponding points in the two point clouds; T is the transformation matrix, T·p i Indicates that the point p in the source point cloud i After the coordinates are transformed by T (including rotation and translation), the two point clouds are aligned by minimizing the distance; ||·|| represents the Euclidean distance, and the Euclidean distance calculation formula is: argmin T : It means finding the transformation matrix T that minimizes the error.

[0024] Further, feature extraction: Normal vector calculation is used to describe the directionality of the point cloud surface, and the plane is fitted based on the neighborhood of the point. The point cloud normal vector fitting formula is:

[0025]

[0026] Among them, n is the normal vector to be solved, argmin n It means finding the normal vector n so that the error is minimized; Yes i field; p i is the centroid of the neighborhood points; p j represents the jth point in the neighborhood.

[0027] On the other hand, the present invention also proposes an intelligent diagnosis system for aviation energy power plant faults, including a fault type retrieval detection module, a knowledge graph fault information retrieval module, and a fault reasoning solution output module. The fault type detection module uses a multimodal knowledge extraction algorithm to extract knowledge from multi-source heterogeneous data, establishes an association relationship between electromechanical system behaviors, and constructs a comprehensive knowledge graph;

[0028] The knowledge graph fault information retrieval module further visualizes the knowledge graph established by the fault type detection module, completes single-modal feature extraction and establishes associations between different modalities, and performs cross-modal feature fusion processing;

[0029] The fault reasoning scheme output module constructs nodes and dependencies and uses conditional probability tables to perform fault diagnosis; performs forward or backward reasoning in combination with known observation data to infer the cause of the fault and updates the database according to the cause of the fault; and also uses a camera to collect two-dimensional images and point cloud data of the object, converts the data into three-dimensional point clouds through feature matching and triangulation, and reconstructs a three-dimensional surface model.

[0030] Beneficial effects of the present invention: The present invention introduces knowledge graph technology into the field of aviation energy power plant fault diagnosis. The knowledge graph converts structured and unstructured knowledge into a graph structure, represents entities and their relationships in the form of nodes and edges, and thus associates information such as the structure, components, sensor data, and failure modes of the aviation energy power plant. The knowledge graph is particularly suitable for representing the complex cause-and-effect relationships between different components in the aviation energy power plant system, which helps to intuitively understand the background and interaction of failures. The introduction of the knowledge graph also makes the system more scalable and self-learning. When new failure modes or new components appear, the knowledge graph can be updated by continuously adding new knowledge to maintain the continuous improvement of diagnostic capabilities. In addition, by combining machine learning and big data analysis technology, the system can automatically learn from new fault data, gradually expand the content of the knowledge graph, and improve the ability to respond to new faults.

[0031] Three-dimensional point cloud model reconstruction is a technology that reconstructs the three-dimensional shape of an object through point cloud data (such as laser scanning, structured light camera, etc.). The present invention cleverly applies the three-dimensional point cloud model reconstruction technology to the fault diagnosis of energy and power devices, which can be used to achieve three-dimensional visualization of energy and power device components and accurately locate faults. Compared with traditional energy and power device fault detection methods such as spectrum analysis, time-frequency analysis, etc., which mainly rely on vibration, temperature and other signals collected by sensors, and can reflect the overall operating status of energy and power devices, there is a defect that it is difficult to accurately locate the specific fault location or to display it visually. The present invention uses three-dimensional point cloud model reconstruction technology to obtain three-dimensional point cloud data of various components of the energy and power device through laser scanning and other means, and uses reconstruction algorithms to generate three-dimensional models, which provides an effective means for the refined inspection of energy and power devices.

[0032] The present invention applies knowledge graph and 3D point cloud model reconstruction technology to energy power plant fault diagnosis, cleverly combining artificial intelligence technology and 3D visualization technology, greatly improving the accuracy and intuitiveness of diagnosis. The knowledge graph provides systematic and interpretable fault analysis through structured representation and reasoning functions, while the point cloud model provides accurate visualization of the fault location through fine 3D reconstruction. The combination of the two can effectively solve the fault diagnosis problem in complex systems and provide efficient intelligent maintenance and repair support for aviation energy power plant systems, which has great engineering application significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is the flow chart of the intelligent diagnosis system for aviation energy power plant faults based on knowledge graph;

[0034] Figure 2 This is the architecture diagram of the intelligent diagnosis system for aviation energy power plant failures. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Combined with Figure 1 , 2 In the specific implementation process of the present invention, a method for intelligent diagnosis of aviation energy power plant faults based on knowledge graph is provided, which specifically includes:

[0037] 1. Multimodal knowledge extraction algorithm: Multimodal knowledge extraction algorithm can extract knowledge from multi-source data such as text, images, audio, video, etc. to build a comprehensive knowledge graph;

[0038] The step 1 specifically includes:

[0039] Step 1.1, data collection and preprocessing: Collect information from multiple data sources, including text, images, videos, audio, etc. Preprocess each data type, such as word segmentation for text, object detection for images, frame extraction for videos, speech-to-text processing for audio, etc.

[0040] Step 1.2, single modal feature extraction: extract features from each modality of data. Text features: use pre-trained models such as BERT to encode text and extract semantic features; image features: use convolutional neural networks (CNNs) to extract visual features of images and identify objects and scenes; video features: extract key frame features of videos through deep learning networks.

[0041] Step 1.3, cross-modal feature alignment: Map data from different modalities to a common semantic space so that features such as text, images, and videos can be aligned and fused. For example, by matching objects in an image with text descriptions, associations between different modalities are established. A cross-modal attention mechanism is used to learn the associations between features from different modalities, such as matching descriptions in text with visual features in an image.

[0042] Step 1.4, entity recognition and relationship extraction: Entity recognition: In text, use named entity recognition (NER) technology to identify entities; in images, use object detection technology to identify objects. For the language part of audio or video, extract meaningful entity information; relationship extraction: extract the relationship between entities from text and visual modalities.

[0043] Step 1.5, multimodal fusion and knowledge representation: Integrate entities and relationships of different modalities into a unified knowledge representation, and build a multimodal knowledge graph containing text, visual, audio and other information; perform structured representation of the fused knowledge through graph neural networks, and learn embedded representations of each entity and relationship for better reasoning and retrieval.

[0044] Step 1.6: Knowledge reasoning and verification are combined with knowledge graph to perform reasoning, verify the correctness of the extraction results, and ensure the accuracy and reliability of the knowledge graph;

[0045] Step 1.7, knowledge graph update: Dynamically update the knowledge graph based on new data to ensure that the graph content evolves over time and continues to improve.

[0046] 2. Knowledge graph visualization: Knowledge graph visualization can make the graph more intuitive through multi-dimensional display and layout optimization. Combining deep learning and graph neural networks, visualizing the reasoning path makes the reasoning process transparent and enhances the application value of knowledge graphs in intelligent analysis;

[0047] The step 2 specifically includes:

[0048] Step 2.1, data preparation and graph construction: extract structured and unstructured data from the knowledge base and convert them into nodes and edges of the knowledge graph; nodes represent different entities, such as equipment, fault type, etc., and edges represent the relationship between different entities.

[0049] Step 2.2, interactive function design: provide interactive operations (such as node dragging, zooming, clicking to view details, etc.) so that users can flexibly operate the graph and view information of interest.

[0050] Step 2.3, search function: supports users to search nodes and quickly locate specific entities or relationships.

[0051] Step 2.4, multi-dimensional display: Combine multi-dimensional display methods such as timeline and heat map to provide the ability to analyze knowledge graphs from different angles; use dynamic display technology to visualize data changes over time to reflect the update and change of knowledge.

[0052] Step 2.5, knowledge reasoning and path display: Combined with the reasoning engine, the reasoning path and logic are displayed on the map to help users understand the reasoning process of complex knowledge; visualize the reasoning process so that users can clearly see the reasoning logic from initial conditions to conclusions.

[0053] Step 2.6, visualization optimization and user feedback: Based on user interactions and feedback, the layout and rendering effects of the graph are continuously optimized to improve user experience; algorithms such as graph neural networks are used for automatic learning and layout optimization to enhance the visualization and expressiveness of the knowledge graph.

[0054] 3. Bayesian network inference algorithm: Bayesian network reasoning constructs nodes and dependencies and uses conditional probability tables to perform fault diagnosis. It combines known observation data for forward or backward reasoning to infer the cause of the fault;

[0055] 4. Reconstruction of the fault 3D point cloud model: The reconstruction of the fault 3D point cloud model uses the camera to collect the 2D image and point cloud data of the object. Then, the data is converted into a 3D point cloud through feature matching and triangulation, and the 3D surface model is reconstructed using algorithms (such as Delaunay triangulation);

[0056] The step 4 specifically includes:

[0057] Step 4.1, data acquisition: obtain multi-source data of the device through image acquisition (camera) and sensors (such as vibration, temperature, stress, etc.). The pictures and videos taken by the camera can generate point cloud data of the device, and the multi-view camera can take multi-angle two-dimensional images of the device, providing a basis for subsequent three-dimensional model reconstruction.

[0058] Step 4.2, feature extraction and matching: Feature point extraction, extracting key feature points of the device from the image, such as edges, corners, etc. Commonly used methods include SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), etc.; Feature matching, matching the feature points of images from different perspectives to find the corresponding points of the same object and establish the association between the images. The formula is expressed as:

[0059] argmin i，j d(f i , f j )

[0060] d(f i , f j ) represents the feature point f i and f j The distance between (such as the calculation method of Euclidean distance: argmin i，j Represents the feature point f i and f j The distance is minimized to achieve feature matching.

[0061] Step 4.3, 3D point cloud generation: triangulation method, after matching the feature points, use triangulation method to calculate the position of the 3D point. Assume that there are two cameras observing the same feature point P, and the camera projection matrix is ​​M 1 and M 2 , the coordinates of this point in the image are p 1 and p 2 , then the position of the three-dimensional point P is determined by the following formula:

[0062] P=trian gulate(M 1 , p 1 , M 2 , p 2 )

[0063] The triangulate function represents a function that uses a triangulation method to find the three-dimensional coordinates of the same point in two images. P is the three-dimensional coordinate of the output target point. 1 and M 2 is the two-dimensional coordinate of the pixel under two different camera view coordinates, p 1 and p 2They are the projection matrices of the two cameras (generally 3×4 projection matrices).

[0064] Step 4.4, point cloud alignment and fusion: The iterative closest point (ICP) algorithm is used to align point clouds obtained from different perspectives. The iterative closest point (ICP) algorithm is used to minimize the error between two point clouds, thereby aligning point cloud data from different perspectives. Its basic formula is:

[0065]

[0066] Among them, p i and q i are the corresponding points in the two point clouds. T is the transformation matrix, T·p i Indicates that the point p in the source point cloud i After the coordinates are transformed by T (including rotation and translation), the two point clouds are aligned by minimizing the distance. ||·|| represents the Euclidean distance, and the Euclidean distance calculation formula is: argmin T : It means finding the transformation matrix T that minimizes the error.

[0067] Step 4.5, 3D surface reconstruction Delaunay triangulation: Convert the point cloud into a 3D surface and generate a 3D mesh structure by triangulating the points in the point cloud. Delaunay triangulation ensures that the maximum internal angle of the triangle is minimized to generate a more reasonable 3D mesh.

[0068] Step 4.6, Poisson surface reconstruction: used to generate a continuous surface model, smooth the noise in the point cloud and generate a smoother three-dimensional surface.

[0069] Step 4.7, fault feature mapping: Map the abnormal features of sensor data (such as vibration, temperature, etc.) to the corresponding positions of the 3D model to highlight the fault location. For example, if the reading of the vibration sensor on a certain component exceeds the threshold, the component will be marked in the 3D model.

[0070] See attached Figure 2 ,The aviation energy power plant fault intelligent inquiry system proposed in the present invention includes a fault type detection module, a knowledge graph fault information retrieval module, and a fault reasoning solution output module;

[0071] The fault type detection module is used to collect sensor data such as images and videos and perform preprocessing, and uses a fault target detection algorithm and a signal processing method to complete fault type detection and identification.

[0072] During the specific implementation process, when collecting image, video and other data, for different types of sensors, after the analog image is collected and the analog image sampling is divided and quantized, the collected video image information is obtained according to the video transmission protocol, and the transmitted video stream is intercepted by frame to obtain the quantized digital image matrix.

[0073] When performing preprocessing such as image segmentation and image enhancement, the threshold segmentation method is used and the appropriate threshold is set to achieve target area segmentation and extraction. At the same time, image noise reduction filtering is achieved through algorithms such as Gaussian filtering, and image edge detection is performed through operators such as Sobel to enrich the details of the edges and contours in the image and achieve image enhancement processing.

[0074] In some feasible cases, the fault target detection algorithm is used. Through the residual connection of the backbone feature extraction network, the model convergence is accelerated and the model's expression ability is enhanced. At the same time, through the feature fusion method of the path aggregation network, the feature transfer from the lower layer to the upper layer is realized, multi-scale feature information is obtained, more complex feature expressions are captured, and the location, size and other information of the fault are accurately identified and located.

[0075] At the same time, a variety of sensor data signal processing algorithms are used for feature extraction and processing. For the various sequence signals collected, feature information is extracted through time domain feature analysis, frequency domain feature analysis, complexity feature analysis and other aspects.

[0076] A multi-source data fusion method based on convolutional neural network is adopted to construct a fault diagnosis model, and fault diagnosis is performed by combining image, vibration signal and other information.

[0077] The knowledge graph fault information retrieval module is used for the collection, cleaning and preprocessing of fault knowledge data, and extracts multi-source heterogeneous knowledge data through a knowledge extraction method based on multimodal feature fusion, combines the knowledge graph database for knowledge representation and storage, and intuitively displays the knowledge graph through a visual interactive interface.

[0078] During the implementation process, fault knowledge data is collected. According to the fault data characteristics of the components of the energy power device, the definitions of fault entity types and fault relationship types are determined, and the knowledge graph ontology is constructed to design the knowledge base system architecture. Structured data such as fault diagnosis data, component fault data, system environmental parameter data, and unstructured data such as equipment manuals and fault analysis reports are collected and sorted.

[0079] When cleaning and preprocessing data, the original text is first sliced ​​according to the set length, some useless text information is filtered out, special marks are added to the beginning and end of each sentence, and then the sentence is converted into code and then into vector using the training model. At the same time, the relationship classification is converted into code, unlabeled, abnormal and unstructured text fragments are filtered out, and the sentence text code, entity code and relationship code are used as input data for the knowledge extraction model.

[0080] When performing multimodal feature fusion knowledge extraction, for the knowledge extraction method of single-modal feature extraction, this module uses a named entity recognition method based on BERT-BiLSTM-CRF to extract entities from fault texts, and a relation extraction method based on BERT-BiLSTM-Attention to extract relations between fault entities. Through a two-step knowledge extraction operation, the knowledge data of fault entities and fault relations used to construct a knowledge graph is finally obtained.

[0081] When representing and storing knowledge, the knowledge graph Neo4j database is used for knowledge representation and storage.

[0082] The knowledge graph visualization interactive interface design can visualize the knowledge graph contents such as fault causes, fault phenomena, maintenance methods, inspection items, etc. according to the query information.

[0083] The fault reasoning scheme output module generates node failure probabilities and constructs conditional probability tables by constructing nodes and dependency relationships and combining known observation data; performs forward or backward reasoning to infer specific fault causes and updates the database based on the fault causes; and also uses cameras to collect two-dimensional images and point cloud data of objects, converts the data into three-dimensional point clouds through feature matching and triangulation, and reconstructs three-dimensional surface models.

[0084] Retrieval and generation of query subgraphs: When a certain fault phenomenon occurs in the device, the system will automatically query the Neo4j graph database through Cypher query statements to retrieve all fault information associated with the fault phenomenon and obtain a knowledge graph query subgraph containing fault nodes, fault relationships, fault probabilities, and fault association strengths.

[0085] Bayesian network construction: Based on the knowledge graph query subgraph obtained from the above query, including fault nodes, fault relations, fault probabilities, and fault correlation strengths, the Noisy-Or method is used to establish associations between nodes. Combined with statistical data and domain expert knowledge, the prior probability of the fault cause and the conditional probability of each node in the Bayesian network are determined.

[0086] Bayesian network reasoning: After the query subgraph of the knowledge graph is retrieved and generated and the Bayesian network is constructed based on the query subgraph, the Bayesian network is used for joint probability reasoning to sort the fault node probabilities obtained by reasoning and obtain the fault cause that is most likely to cause the fault phenomenon.

[0087] Fault repair plan output: According to the fault cause nodes sorted by probability of occurrence, multi-level queries are further performed in the fault knowledge graph of the Neo4j graph database through Cypher query statements to obtain the corresponding repair plans for each fault cause.

[0088] The above is only a specific embodiment of the present invention, and the present invention is described in detail. The unexplained part is a conventional technology. However, the protection scope of the present invention is not limited to this. Any changes or substitutions that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An intelligent diagnosis method for energy power device faults based on knowledge graph, characterized in that: The method comprises the following steps: Step S1, extracting knowledge from multi-source data using a multimodal knowledge extraction algorithm, establishing associations between electromechanical system behaviors, and constructing a comprehensive knowledge graph; Step S2: further visualize the knowledge graph established in step S1. Knowledge graph visualization can make the graph more intuitive through multi-dimensional display and layout optimization; combine deep learning and graph neural network to visualize the reasoning path to make the reasoning process transparent, thereby enhancing the application value of knowledge graph in intelligent analysis; Step S3: Bayesian network reasoning, by constructing nodes and dependencies, using conditional probability tables to perform fault diagnosis; combining known observation data to perform forward or backward reasoning to infer the cause of the fault; Step S4, reconstruction of the three-dimensional point cloud model of the fault, using a camera to collect a two-dimensional image or point cloud data of the object; then, the data is converted into a three-dimensional point cloud through feature matching and triangulation, and an algorithm is used to reconstruct a three-dimensional surface model.

2. The method for intelligent diagnosis of energy power device faults based on knowledge graph as claimed in claim 1, characterized in that: In step S1, the multimodal knowledge extraction algorithm specifically includes the following steps: data collection preprocessing, unimodal feature extraction, cross-modal feature alignment, entity recognition and relationship extraction, multimodal fusion and knowledge representation, knowledge reasoning verification and knowledge graph update.

3. The method for intelligent diagnosis of energy power device faults based on knowledge graph as claimed in claim 2, characterized in that: The multimodal knowledge extraction algorithm for constructing the knowledge graph of aviation energy power plant fault diagnosis mainly includes: Unimodal feature extraction: Unimodal feature extraction extracts representative features from a single type of data, including text features, image features, audio features, point cloud features, and time series features; these feature extraction methods can capture important information in the data and provide a basis for subsequent analysis and modeling; Cross-modal feature alignment: Cross-modal feature extraction aligns and fuses data features of different modalities to extract features in a shared semantic space; Multimodal fusion and knowledge representation: Multimodal fusion integrates different data modalities through early, late, and mid-term fusion, and uses joint embedding, graph neural networks and other methods to unify knowledge representation to enhance semantic understanding and reasoning capabilities.

4. The method for intelligent diagnosis of energy power device faults based on knowledge graph as claimed in claim 3, characterized in that: In step S2, the knowledge graph is visualized, which specifically includes the following steps: data preparation and graph construction, interactive function design, search function, multi-dimensional display, knowledge reasoning and path display, visualization optimization and user feedback.

5. The method for intelligent diagnosis of energy power device faults based on knowledge graph as claimed in claim 4, characterized in that: In step S3, the Bayesian network reasoning algorithm is mainly used to diagnose faults by constructing nodes and dependencies and using conditional probability tables; by forward or backward reasoning, the probabilities of other nodes are calculated in combination with known observation data to infer the cause of the fault.

6. The method for intelligent diagnosis of energy power device faults based on knowledge graph as claimed in claim 5, characterized in that: In step S4, the fault three-dimensional point cloud model is reconstructed, which specifically includes the following steps: Point cloud data collection of target objects, feature extraction and matching, 3D point cloud generation, point cloud alignment and fusion, 3D surface reconstruction Delaunay triangulation, and fault feature mapping.

7. The method for intelligent diagnosis of energy power device faults based on knowledge graph as claimed in claim 6, characterized in that: The 3D point cloud model reconstruction technology for aviation energy power plant faults includes: Collection and representation of point cloud data: Point cloud data is collected through various methods such as lidar, camera, structured light scanning, etc. Lidar uses laser beams to scan the environment and measures the reflection time to calculate the distance of objects. Point cloud data is usually represented as a set of points P = {p i }, each point p i Contains its three-dimensional coordinate information; Point cloud registration: Point cloud registration refers to aligning point cloud data from multiple sources or multiple perspectives to generate a complete 3D model. The ICP algorithm is used to align the source point cloud to the target point cloud. The basic idea is to minimize the square distance between the points of the two sets of point clouds. The formula is as follows: Among them, p i and q i are the corresponding points in the two point clouds; T is the transformation matrix, T·p i Indicates that the point p in the source point cloud i After the coordinates are transformed by T, the two point clouds are aligned by minimizing the distance; ||·|| represents the Euclidean distance, and the Euclidean distance calculation formula is: argmin T : It means finding the transformation matrix T that minimizes the error.

8. The method for intelligent diagnosis of energy and power device faults based on knowledge graph as claimed in claim 7 is characterized by feature extraction: normal vector calculation is used to describe the directionality of the point cloud surface, a plane is fitted based on the neighborhood of the point, and the point cloud normal vector fitting formula is: in, n is the normal vector to be solved, argmin n It means finding the normal vector n so that the error is minimized; Yes i field; p i is the centroid of the neighborhood points; p j represents the jth point in the neighborhood.

9. An intelligent diagnosis system for aviation energy power plant faults, characterized in that: It includes a fault type retrieval detection module, a knowledge graph fault information retrieval module, and a fault reasoning solution output module. The fault type detection module uses a multimodal knowledge extraction algorithm to extract knowledge from multi-source heterogeneous data, establishes a correlation relationship between electromechanical system behaviors, and constructs a comprehensive knowledge graph; The knowledge graph fault information retrieval module further visualizes the knowledge graph established by the fault type detection module, completes single-modal feature extraction and establishes associations between different modalities, and performs cross-modal feature fusion processing; The fault reasoning scheme output module constructs nodes and dependencies and uses conditional probability tables to perform fault diagnosis; performs forward or backward reasoning in combination with known observation data to infer the cause of the fault and updates the database according to the cause of the fault; and also uses a camera to collect two-dimensional images and point cloud data of the object, converts the data into three-dimensional point clouds through feature matching and triangulation, and reconstructs a three-dimensional surface model.

Images