Wind turbine generator maintenance method based on multi-modal data fusion and knowledge graph

Abstract

The invention belongs to the technical field of wind power equipment fault diagnosis, and relates to a wind turbine generator maintenance method based on multi-modal data fusion and a knowledge graph. Comprising the steps of obtaining text description data, component image data and equipment operation time sequence data of a wind turbine generator; predicting the residual life of the component based on the equipment operation time sequence data, and generating a residual life prediction value; performing feature extraction on the data to obtain corresponding features; performing fusion processing on the features to obtain joint feature representation; analyzing a fault evolution time sequence mode of the wind turbine generator based on the joint feature representation; performing matching retrieval in a historical case library according to a fault evolution time sequence mode; and inputting a matching retrieval result into the dynamic knowledge graph for reasoning, generating a fault traceability result, and generating a maintenance decision scheme including a maintenance priority list in combination with the residual life prediction value. According to the method, accurate fault diagnosis, automatic source tracing of root causes and dynamic optimization of maintenance strategies are realized, the operation and maintenance efficiency is remarkably improved, and the cost is reduced.

Description

Technical Field

[0001] This invention belongs to the field of wind power equipment fault diagnosis technology, and relates to a wind turbine maintenance method based on multimodal data fusion and knowledge graph. Background Technology

[0002] With the rapid growth of wind power installed capacity, the failure problems of key components of wind turbines (such as drive trains, blades, gearboxes, etc.) are becoming increasingly prominent. However, existing fault diagnosis technologies still have many limitations.

[0003] First, the current fault diagnosis process relies heavily on the experience and judgment of maintenance personnel, which is not only highly subjective and has a long diagnosis cycle, but also makes it difficult to effectively integrate multi-source heterogeneous data such as SCADA monitoring data, vibration signals, and infrared images, resulting in low fault identification accuracy.

[0004] Secondly, existing fault diagnosis systems are limited in function, mostly only capable of issuing alarms for abnormalities without being able to deeply analyze the fault mechanisms, and lacking the ability to adapt to new fault modes. More importantly, the industry has not yet established a correlation model between system failure phenomena and root causes, making the process of tracing the source from surface fault characteristics to deep-seated failure causes lack a scientific basis.

[0005] Furthermore, in terms of maintenance decision-making, there is a widespread phenomenon of "over-maintenance," often employing extensive maintenance strategies such as complete turbine replacement, which wastes resources and affects power generation efficiency. Meanwhile, maintenance recommendations rely primarily on simple rule bases, failing to incorporate historical cases for intelligent recommendations or conduct risk assessments based on the actual condition of the equipment, severely restricting the operation and maintenance efficiency and economic viability of wind farms. These issues have become key bottlenecks hindering cost reduction and efficiency improvement in the wind power industry. Summary of the Invention

[0006] To address the problems in existing technologies, this invention provides a wind turbine maintenance method based on multimodal data fusion and knowledge graphs, which enables accurate fault diagnosis, automatic root cause tracing, and dynamic optimization of maintenance strategies, significantly improving operation and maintenance efficiency and reducing unplanned downtime losses.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides a wind turbine maintenance method based on multimodal data fusion and knowledge graph, comprising the following steps: Acquire text description data, component image data, and equipment runtime sequence data of the wind turbine; Based on the device's runtime sequence data, the remaining lifespan of the components is predicted, and a predicted remaining lifespan value is generated. Feature extraction is performed on the text description data, component image data, and device runtime sequence data respectively to obtain corresponding text features, image features, and time sequence features; The text features, image features, and temporal features are fused to obtain a joint feature representation; Based on the joint feature representation, the fault evolution time sequence pattern of the wind turbine is analyzed; according to the fault evolution time sequence pattern, a matching search is performed in the historical case database; The matching retrieval results are input into the dynamic knowledge graph for reasoning, generating fault tracing results including the probability distribution of fault causes and causal reasoning paths; Based on the probability distribution of fault causes and causal reasoning paths in the fault tracing results, a maintenance decision scheme including a maintenance priority list is generated by combining the remaining life prediction value.

[0008] Preferably, the text description data includes failure phenomena described in natural language; the component image data includes infrared thermal images and surface defect images of the components; and the device operation sequence data includes vibration spectra and temperature curves collected by the SCADA system and CMS system.

[0009] Preferably, the feature extraction step includes: The BERT model is used to generate semantic vectors for text description data, and key fault entities are labeled using entity recognition technology; an improved ResNet network is used to extract visual features for component image data; and Fourier transform is performed on the equipment runtime sequence data to extract spectral features.

[0010] Preferably, the fusion processing step includes: By using a cross-modal attention mechanism, text features, image features, and time-series features are weighted and fused to establish a feature mapping relationship, generating a unified multimodal feature matrix that serves as the joint feature representation.

[0011] Preferably, the step of analyzing the fault evolution time sequence mode of the wind turbine includes: A bidirectional LSTM network is used to model the temporal dependency of the joint feature representation, and the temporal evolution features of the fault state sequence are output.

[0012] Preferably, the dynamic knowledge graph includes: The ontology layer constructs a triplet structure that includes the entity relationships between design parameters, process standards, and failure modes; The data layer integrates a structured database and a graph database; the structured database stores wind turbine design documents, FMEA libraries, and maintenance cases; the graph database stores the entity relationship topology. The inference layer integrates a path ranking algorithm to generate causal inference paths and uses a Bayesian network to calculate the probability distribution of fault causes.

[0013] Preferably, the generated maintenance decision scheme includes: Based on the probability distribution of the causes of failures, a mapping relationship between the causes of failures and maintenance measures is established to generate a target maintenance strategy. Based on the component nodes in the causal reasoning path, recommended maintenance solutions for similar failure scenarios are retrieved from the historical case library. The target maintenance strategy and recommended maintenance solutions are fused together, and a risk assessment is performed in conjunction with the predicted remaining life of the equipment. A maintenance decision plan including a maintenance priority list is then output.

[0014] Secondly, the present invention provides a wind turbine maintenance system based on multimodal data fusion and knowledge graph, comprising: Data acquisition module: used to acquire text description data, component image data, and equipment operation sequence data of wind turbine units; Lifetime prediction module: used to predict the remaining lifetime of components based on the device's runtime sequence data, and generate a remaining lifetime prediction value; Feature extraction module: used to extract features from the text description data, component image data and device runtime sequence data respectively, to obtain corresponding text features, image features and time sequence features; Feature fusion module: used to fuse the text features, image features and temporal features to obtain a joint feature representation; Fault Evolution Module: Used to analyze the fault evolution time sequence pattern of wind turbine based on the joint feature representation; and to perform matching and retrieval in the historical case database according to the fault evolution time sequence pattern; Knowledge graph module: Used to input matching retrieval results into a dynamic knowledge graph for reasoning, generating fault tracing results including fault cause probability distribution and causal reasoning path; Maintenance decision module: This module is used to generate a maintenance decision scheme, including a maintenance priority list, based on the probability distribution of fault causes and causal reasoning paths in the fault tracing results, combined with the remaining life prediction value.

[0015] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the above-described method.

[0017] Compared with the prior art, the present invention has the following beneficial effects: By integrating textual description data, component image data, and equipment runtime sequence data, this invention can comprehensively capture the multi-dimensional features of faults. Textual features extract key semantic information about the faults, image features accurately locate abnormal areas, and time-series features reveal equipment degradation patterns through spectral analysis. The dynamic fusion of multi-modal features forms a high-information-density joint feature representation, expanding the coverage of fault diagnosis from a single physical quantity to multi-source associated features, effectively solving the diagnostic blind spots caused by isolated data in traditional methods. Based on the joint feature-driven fault evolution time-series pattern analysis, combined with matching retrieval from a historical case database, quantitative modeling of fault development patterns and intelligent matching of similar cases are achieved. The reasoning mechanism of the dynamic knowledge graph further links surface-level fault phenomena with deep-seated root causes, and the generated fault cause probability distribution and causal reasoning path significantly improve the accuracy and interpretability of fault tracing. Finally, the maintenance decision scheme generated by combining remaining life prediction values, through quantitative assessment of fault urgency and maintenance resource matching, ensures that the formulation of the maintenance priority list not only conforms to the actual health status of the equipment but also meets the optimal allocation of on-site maintenance resources. Compared with traditional experience-based decision-making methods, this reduces unnecessary downtime and lowers maintenance costs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0025] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0026] The present invention will now be described in further detail with reference to the accompanying drawings: The first objective of this invention is to provide a wind turbine maintenance method based on multimodal data fusion and knowledge graph, such as... Figure 1 As shown, it includes the following steps: Acquire text description data, component image data, and equipment runtime sequence data of the wind turbine; Predict the remaining lifespan of components based on equipment runtime sequence data, and generate a predicted remaining lifespan value; Feature extraction was performed on text description data, component image data, and device runtime sequence data to obtain corresponding text features, image features, and time sequence features. Text features, image features, and temporal features are fused to obtain a joint feature representation; Based on joint feature representation, the fault evolution time sequence pattern of wind turbine units is analyzed; according to the fault evolution time sequence pattern, a matching retrieval is performed in the historical case database; The matching retrieval results are input into the dynamic knowledge graph for reasoning, generating fault tracing results including the probability distribution of fault causes and causal reasoning paths; Based on the probability distribution of fault causes and causal reasoning paths in the fault tracing results, a maintenance decision scheme including a maintenance priority list is generated by combining the remaining life prediction value.

[0027] This invention presents a wind turbine maintenance method based on multimodal data fusion and knowledge graphs. By integrating multi-source heterogeneous data and a dynamic knowledge reasoning mechanism, it significantly improves the accuracy of fault diagnosis and the scientific nature of maintenance decisions. Specifically: text descriptions, component images, and equipment runtime sequence data provide comprehensive input to the system, covering natural language descriptions of fault phenomena, visual defect features, and temporal changes in operating status, laying the foundation for multi-dimensional analysis; based on equipment runtime sequence data, it predicts the lifespan degradation trend of key components, providing a time-dimensional basis for prioritizing maintenance; key features (such as fault entities in text, structural anomalies in images, and vibration spectrum characteristics in time series) are extracted from text, image, and time series data respectively, preserving the uniqueness of each modality; heterogeneous features are weighted and integrated through a cross-modal attention mechanism to generate a unified joint feature representation, overcoming the limitations of a single data source and enhancing the completeness of state representation; based on the fused features, it analyzes the dynamic laws of fault development (such as vibration intensification and temperature rise). The system identifies potential paths for fault evolution through correlation analysis; it uses an improved similarity algorithm to filter similar fault scenarios in a case library, providing valuable experience for handling current problems; dynamic knowledge graph reasoning utilizes entity relationships defined in the ontology layer (such as the causal chain of "seal failure → oil leakage") and maintenance cases stored in the data layer, combined with Bayesian networks to calculate the probability distribution of fault causes and generate traceable causal reasoning paths, improving the interpretability of diagnostic results; finally, maintenance decisions generate comprehensive fault cause probabilities, causal paths, and remaining life predictions, constructing a priority list including maintenance steps, spare parts lists, and risk warnings, achieving a closed-loop decision-making process from "phenomenon diagnosis" to "management optimization," and maintenance result data can flow back to the knowledge graph-driven system for self-evolution, updating knowledge graph nodes and edges.

[0028] In one embodiment of the present invention, the text description data includes failure phenomena described in natural language. For example, maintenance personnel can input a description of the failure phenomenon (such as "periodic abnormal noise from the gearbox accompanied by a sudden rise in oil temperature") via a mobile terminal or SCADA system human-machine interface, and the data supports voice-to-text conversion to adapt to voice command recognition in noisy environments. Component image data includes infrared thermal images and surface defect images of components. The infrared thermal images are obtained by an infrared camera capturing images of key components such as the gearbox and generator bearings. The surface defect images are acquired using an industrial endoscope, focusing on capturing microscopic defects such as pitting and cracks on the gearbox tooth surfaces. Equipment operation sequence data includes vibration spectra and temperature curves acquired by the SCADA system and CMS system.

[0029] In one embodiment of the present invention, the feature extraction step includes: generating semantic vectors from text description data using a BERT model and labeling key fault entities using entity recognition technology; extracting visual features from component image data using an improved ResNet network; and extracting spectral features from device runtime sequence data using Fourier transform.

[0030] This invention employs the BERT model to process textual description data, utilizing its bidirectional Transformer architecture to capture the contextual semantic relationships of fault descriptions and generate high-dimensional semantic vectors (such as context-related representations of phrases like "abnormal vibration" and "sudden rise in oil temperature"). Simultaneously, it accurately labels key fault entities (such as "gearbox" and "sealing ring") through entity recognition, providing precise input for subsequent knowledge graph association. For component image data, the improved ResNet network effectively extracts temperature gradient patterns (such as the shape of local high-temperature areas) from infrared thermal images and microscopic texture features (such as crack direction and oil stain diffusion morphology) from surface defect images through residual connections, overcoming the sensitivity of traditional visual models to industrial noise. Fourier transform (FFT) is applied to the equipment runtime sequence data to convert time-domain signals such as vibration and temperature into frequency-domain features (such as sideband energy of gear meshing frequency and amplitude of bearing fault characteristic frequencies), accurately quantifying the physical characteristics of the fault.

[0031] In one embodiment of the present invention, the fusion processing step includes: weighted fusion of text features, image features and time-series features through a cross-modal attention mechanism, establishing a feature mapping relationship, and generating a unified multimodal feature matrix as a joint feature representation.

[0032] The cross-modal attention mechanism establishes an interactive weight allocation model for textual features (such as fault entity descriptions), image features (such as thermal imaging temperature gradients), and temporal features (such as vibration spectra). It dynamically allocates the contribution weights of each modality, giving higher weight to key information (such as hot spots on tooth surface cracks corresponding to high-frequency vibrations), while suppressing redundant or noisy features (such as temperature fluctuations caused by environmental interference). This solves the modal conflict problem that traditional splicing or weighted averaging methods struggle to handle. The feature mapping relationship construction projects heterogeneous features onto a unified semantic space, generating a unified multimodal feature matrix.

[0033] In one embodiment of the present invention, the step of analyzing the time-series pattern of fault evolution of wind turbine units includes: using a bidirectional LSTM network to model the time-series dependency relationship of the joint feature representation and outputting the time evolution features of the fault state sequence.

[0034] The bidirectional LSTM network, through a dual-channel LSTM structure (forward and backward), simultaneously captures the historical cumulative effect of fault characteristics (such as the gradual process of bearing wear) and future potential impacts (such as the precursors of cascading damage to subsequent components due to temperature anomalies). This solves the problem of delayed early fault characteristic perception caused by the unidirectional flow of information in traditional unidirectional LSTM. The time evolution features it generates not only quantify the dynamic changes in fault state (such as the diffusion path of spectral energy over time), but also encode the stage characteristics of fault development (latency period, acceleration period, and failure period) through implicit state vectors, providing interpretable temporal dimensions for maintenance decisions.

[0035] For example, dynamic knowledge graphs include: The ontology layer constructs a triplet structure that includes the entity relationships between design parameters, process standards, and failure modes; The data layer integrates structured databases and graph databases; the structured database stores wind turbine design documents, FMEA libraries, and maintenance cases; the graph database stores entity relationship topology. The inference layer integrates a path ranking algorithm to generate causal inference paths and uses a Bayesian network to calculate the probability distribution of fault causes.

[0036] The ontology layer constructs a standardized expression of domain knowledge through triplet relationships between design parameters, process standards, and failure modes, providing a rigorous logical framework for fault reasoning. The data layer adopts a heterogeneous storage mode of structured database and graph database, which not only preserves the integrity and query efficiency of traditional data such as design documents and FMEA libraries, but also achieves efficient traversal of complex associations through the topological relationship storage of graph database, solving the scalability problem of large-scale relational networks. The reasoning layer integrates path ranking algorithms and Bayesian networks. The former generates interpretable causal chains (such as "process deviation → assembly stress concentration → sealing failure") through graph path search, while the latter quantifies the probabilistic dependencies between nodes. The combination of the two ensures the domain logic consistency of the reasoning process and outputs probabilistic results that can support risk decision-making.

[0037] In one embodiment of the present invention, generating a maintenance decision scheme includes: establishing a mapping relationship between fault causes and maintenance measures based on the probability distribution of fault causes, and generating a target maintenance strategy; retrieving recommended maintenance schemes for similar fault scenarios from a historical case library based on component nodes in the causal reasoning path; fusing the target maintenance strategy and recommended maintenance schemes, combining them with the predicted remaining life of the equipment for risk assessment, and outputting a maintenance decision scheme including a maintenance priority list.

[0038] The maintenance decision generation mechanism of this invention significantly improves the scientific nature and operability of operation and maintenance decisions through triple intelligent integration: First, it accurately maps the probability distribution of fault causes to the maintenance measures knowledge base, ensuring the matching degree between maintenance strategies and fault root causes, and avoiding over-maintenance or under-maintenance caused by empiricism; Second, it locates key component nodes through causal reasoning paths and matches them with historical case libraries, realizing the intelligent reuse of expert experience and solutions for similar scenarios, effectively making up for the limitations of a single reasoning model; Finally, it integrates target maintenance strategies, historical recommended solutions, and remaining life prediction values ​​for multi-dimensional risk assessment, so that the output maintenance priority list considers both the probability of fault occurrence and the time urgency of component failure, forming a dynamic optimization scheme that takes into account maintenance costs, safety margins, and equipment life cycle management, realizing a leap from traditional periodic maintenance to a precise maintenance model based on risk prediction.

[0039] In addition, the maintenance decision-making scheme adopts a multimodal report format that combines text and graphics. It uses a combination of fault tree diagrams and text descriptions to intuitively show the complete reasoning chain from surface phenomena to root causes, such as the hierarchical decomposition of "abnormal gearbox vibration → bearing wear → lubrication failure → oil contamination". Each node is marked with the confidence level of the knowledge graph derivation.

[0040] In summary, this invention employs a cross-modal attention network to achieve deep collaborative analysis of text descriptions, visual features, and SCADA / CMS maintenance data, overcoming the information fragmentation problem inherent in traditional single-modal diagnostics. It develops a self-evolving domain knowledge graph that continuously absorbs new fault modes and maintenance experience through online incremental learning, enabling the system to dynamically update its knowledge. It integrates causal reasoning and case-based reasoning engines, generating interpretable causal chains through path ranking algorithms in the knowledge graph and providing empirical support through historical case matching, significantly improving the credibility of diagnostic results. Finally, it constructs a maintenance decision-making model based on full lifecycle data, integrating multi-dimensional parameters such as real-time monitoring data, fault probability distribution, and component remaining life to generate preventative maintenance solutions that combine risk prediction and cost optimization, achieving a shift from passive maintenance to proactive health management.

[0041] Example A wind turbine maintenance method based on multimodal data fusion and knowledge graph includes: S1. Obtain text description data, component image data, and equipment runtime sequence data of the wind turbine.

[0042] Text description data: Oil leakage and abnormal noise have occurred at the high-speed shaft end cover of the wind turbine gearbox, and the vibration has increased significantly recently; Component image data: Infrared images show oil stains and an abnormal temperature gradient at the end cap connection. Equipment operation sequence data: The SCADA system detected an axial vibration peak of 8.5 mm / s (exceeding the safety threshold of 5 mm / s), and the temperature data showed that the lubricating oil temperature rose by 15% within 10 days.

[0043] S2. Based on the device's runtime sequence data, predict the remaining lifespan of the components and generate a predicted remaining lifespan value.

[0044] Analysis based on the vibration degradation curve using the Weibull life prediction model shows that the remaining life of the high-speed shaft is estimated to be only 120±20 hours.

[0045] S3. Extract features from the text description data, component image data, and device runtime sequence data respectively to obtain the corresponding text features, image features, and time sequence features.

[0046] The semantic vectors generated based on the BERT model successfully identified key entities: "gearbox high-speed shaft" (component), "oil leakage" (phenomenon), "abnormal noise" (phenomenon), and "intensified vibration" (temporal correlation feature). Named entity recognition (NER) associated "oil leakage" with the seal failure mode, and "abnormal noise" with potential bearing or gear wear failure. Meanwhile, the improved ResNet network extracted high-temperature areas (>80℃) from visual features, which were concentrated at bolted connections, indicating uneven stress on the sealing surface. The radial diffusion pattern of oil stains pointed to end cover flatness defects. Fourier transform spectrum analysis further showed a significant increase in the amplitude of the 1st shaft frequency, accompanied by harmonic sideband frequencies, suggesting poor axial alignment or abnormal bearing clearance.

[0047] S4. The text features, image features and temporal features are fused to obtain a joint feature representation.

[0048] Multi-source features are dynamically weighted and fused using a cross-modal attention mechanism, with text features accounting for 40% of the weight, image features accounting for 35% of the weight, and temporal features accounting for 25% of the weight. The final output is a unified feature matrix as a joint feature representation of the multimodal features.

[0049] S5. Based on the joint feature representation, analyze the fault evolution time sequence pattern of the wind turbine.

[0050] The sealing failure process exhibits a distinct three-stage characteristic: the incubation period is characterized by a slow temperature rise, followed by an acceleration period with a sudden increase in vibration, and finally a critical period accompanied by the spread of oil stains; this failure evolution follows a causal chain relationship of "vibration exceeding limits → bolt preload loosening → end cap deformation → sealing failure".

[0051] S6. Based on the fault evolution time sequence pattern, perform a matching search in the historical case database.

[0052] Three similar failure cases were found through historical case matching and analysis. The failures were confirmed to be caused by end cap processing errors. The diagnosis has a confidence level of 85%.

[0053] S7. Input the matching retrieval results into the dynamic knowledge graph for reasoning, and generate fault tracing results including the probability distribution of fault causes and causal reasoning paths.

[0054] Knowledge graph triplet retrieval confirmed the association between "gearbox high-speed shaft - failure mode - seal failure" and "seal failure - root cause - end cover flatness deviation". Bayesian network analysis showed that the root cause probability distribution was manufacturing process defects (65%), assembly deviation (25%), and material aging (10%). Simultaneously, a path ranking algorithm constructed a complete fault causal chain: material fatigue caused seal deformation, which in turn led to assembly stress concentration, ultimately tracing the source back to manufacturing process defects (machine tool calibration deviation).

[0055] S8. Based on the probability distribution of fault causes and causal reasoning paths in the fault tracing results, and combined with the remaining life prediction values, generate a maintenance decision scheme including a maintenance priority list.

[0056] Based on a comprehensive analysis of knowledge graphs and historical cases, the following maintenance measures are recommended: Immediately replace the end cap with the improved Type A end cap (case studies have shown it can extend service life by 30%), simultaneously recalibrate the assembly line and adjust the bolt preload to the design value of 120±5Nm, and verify the flatness of the end cap; the root cause analysis confirms that it is due to the excessive flatness of the end cap machining (machine tool calibration deviation, confidence level 85%), and it is recommended to simultaneously check the end cap condition of units in the same batch. Based on the remaining service life prediction (120 hours), the maintenance priority is listed as "urgent". If not handled in time, it is expected that a shutdown will be triggered within 48 hours due to excessive oil leakage. Therefore, a spare gearbox should be prepared in advance to avoid the risk of unplanned downtime.

[0057] The second objective of this invention is to provide a wind turbine maintenance system based on multimodal data fusion and knowledge graph, comprising: Data acquisition module: used to acquire text description data, component image data, and equipment operation sequence data of wind turbine units; Lifetime prediction module: used to predict the remaining lifetime of components based on equipment runtime sequence data and generate remaining lifetime prediction values; Feature extraction module: used to extract features from text description data, component image data and device runtime sequence data respectively, to obtain corresponding text features, image features and time sequence features; Feature fusion module: used to fuse text features, image features and temporal features to obtain joint feature representation; Fault Evolution Module: Used to analyze the fault evolution time-series patterns of wind turbines based on joint feature representation; and to perform matching and retrieval in the historical case database according to the fault evolution time-series patterns. Knowledge graph module: Used to input matching retrieval results into a dynamic knowledge graph for reasoning, generating fault tracing results including fault cause probability distribution and causal reasoning path; Maintenance Decision Module: This module is used to generate maintenance decision schemes, including a maintenance priority list, based on the probability distribution of fault causes and causal reasoning paths in the fault tracing results, combined with the remaining life prediction values.

[0058] This invention constructs a multi-source heterogeneous data entry point through a data acquisition module, providing the system with comprehensive perception capabilities; introduces a time dimension parameter through a lifespan prediction module, enabling the system to have a predictive maintenance perspective; breaks through data modality barriers through a feature extraction and fusion module, forming a joint feature representation with high information density; achieves dynamic fault tracking through time-series pattern analysis and case matching through a fault evolution module; combines expert experience with data-driven methods through a knowledge graph module, outputting interpretable causal reasoning results; and finally, integrates multi-dimensional factors such as probability distribution, causal path, and remaining lifespan to generate a priority list for quantitative risk assessment. The collaborative work of these modules not only solves the information silo problem in traditional operation and maintenance but also achieves a one-stop transformation from raw data to executable maintenance plans through the deep integration of knowledge graphs and multimodal analysis, significantly improving the accuracy, timeliness, and economy of wind turbine operation and maintenance.

[0059] In one embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or corresponding function. The processor in this embodiment of the present invention can be used for the operation of a wind turbine maintenance method based on multimodal data fusion and knowledge graphs.

[0060] This invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the wind turbine maintenance method based on multimodal data fusion and knowledge graph in the above embodiments.

[0061] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A wind turbine maintenance method based on multimodal data fusion and knowledge graph, characterized in that, Includes the following steps: Acquire text description data, component image data, and equipment runtime sequence data of the wind turbine; Based on the device's runtime sequence data, the remaining lifespan of the components is predicted, and a predicted remaining lifespan value is generated. Feature extraction is performed on the text description data, component image data, and device runtime sequence data respectively to obtain corresponding text features, image features, and time sequence features; The text features, image features, and temporal features are fused to obtain a joint feature representation; Based on the joint feature representation, the fault evolution time sequence pattern of the wind turbine is analyzed; according to the fault evolution time sequence pattern, a matching search is performed in the historical case database; The matching retrieval results are input into a dynamic knowledge graph for reasoning, generating fault tracing results that include the probability distribution of fault causes and causal reasoning paths; Based on the probability distribution of fault causes and causal reasoning paths in the fault tracing results, a maintenance decision scheme including a maintenance priority list is generated by combining the remaining life prediction value.

2. The wind turbine maintenance method based on multimodal data fusion and knowledge graph as described in claim 1, characterized in that, The text description data includes failure phenomena described in natural language; the component image data includes infrared thermal images and surface defect images of components; the equipment operation sequence data includes vibration spectra and temperature curves collected by the SCADA system and CMS system.

3. The wind turbine maintenance method based on multimodal data fusion and knowledge graph as described in claim 1, characterized in that, The feature extraction steps include: The BERT model is used to generate semantic vectors for text description data, and key fault entities are labeled using entity recognition technology; an improved ResNet network is used to extract visual features for component image data; and Fourier transform is performed on the equipment runtime sequence data to extract spectral features.

4. The wind turbine maintenance method based on multimodal data fusion and knowledge graph as described in claim 1, characterized in that, The fusion processing steps include: By using a cross-modal attention mechanism, text features, image features, and time-series features are weighted and fused to establish a feature mapping relationship, generating a unified multimodal feature matrix that serves as the joint feature representation.

5. The wind turbine maintenance method based on multimodal data fusion and knowledge graph as described in claim 1, characterized in that, The steps for analyzing the time-series patterns of wind turbine fault evolution include: A bidirectional LSTM network is used to model the temporal dependency of the joint feature representation, and the temporal evolution features of the fault state sequence are output.

6. The wind turbine maintenance method based on multimodal data fusion and knowledge graph as described in claim 1, characterized in that, The dynamic knowledge graph includes: The ontology layer constructs a triplet structure that includes the entity relationships between design parameters, process standards, and failure modes; The data layer integrates a structured database and a graph database; the structured database stores wind turbine design documents, FMEA libraries, and maintenance cases; the graph database stores the entity relationship topology. The inference layer integrates a path ranking algorithm to generate causal inference paths and uses a Bayesian network to calculate the probability distribution of fault causes.

7. The wind turbine maintenance method based on multimodal data fusion and knowledge graph as described in claim 1, characterized in that, The generated maintenance decision scheme includes: Based on the probability distribution of the causes of failures, a mapping relationship between the causes of failures and maintenance measures is established to generate a target maintenance strategy. Based on the component nodes in the causal reasoning path, recommended maintenance solutions for similar failure scenarios are retrieved from the historical case library. The target maintenance strategy and recommended maintenance solutions are fused together, and a risk assessment is performed in conjunction with the predicted remaining life of the equipment. A maintenance decision plan including a maintenance priority list is then output.

8. A wind turbine maintenance system based on multimodal data fusion and knowledge graph, characterized in that, include: Data acquisition module: used to acquire text description data, component image data, and equipment operation sequence data of wind turbine units; Lifetime prediction module: used to predict the remaining lifetime of components based on the device's runtime sequence data, and generate a remaining lifetime prediction value; Feature extraction module: used to extract features from the text description data, component image data and device runtime sequence data respectively, to obtain corresponding text features, image features and time sequence features; Feature fusion module: used to fuse the text features, image features and temporal features to obtain a joint feature representation; Fault Evolution Module: Used to analyze the fault evolution time sequence pattern of wind turbine based on the joint feature representation; and to perform matching and retrieval in the historical case database according to the fault evolution time sequence pattern; Knowledge graph module: Used to input matching retrieval results into a dynamic knowledge graph for reasoning, generating fault tracing results including fault cause probability distribution and causal reasoning path; Maintenance decision module: This module is used to generate a maintenance decision scheme, including a maintenance priority list, based on the probability distribution of fault causes and causal reasoning paths in the fault tracing results, combined with the remaining life prediction value.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images