Fan gearbox fault image recognition method and system

Abstract

The invention belongs to the technical field of fault diagnosis of wind power generation equipment, and relates to a fan gearbox fault image recognition method and system. According to the method disclosed by the invention, the multi-modal data of the fan gear box is acquired, and the multi-modal data complement each other, so that the accuracy and reliability of the data on fault characterization are improved. And carrying out deep learning analysis on the preprocessed multi-modal data to obtain a deep learning result, and carrying out efficient and accurate classification on fault types of the fan gearbox by training a deep learning model. Fault reasoning is carried out based on expert knowledge and historical data to obtain a knowledge graph reasoning result, the expert knowledge and the historical data are combined, a knowledge graph is constructed, knowledge fusion and sharing are realized, and logical reasoning is carried out by using the knowledge graph. Fault diagnosis is carried out according to a deep learning result and an identification map reasoning result, the advantages of various methods are integrated, the fault type and part can be identified more quickly, and timely guidance is provided for subsequent maintenance work.

Description

Technical Field

[0001] The invention belongs to the technical field of wind power generation equipment fault diagnosis, and relates to a wind turbine gearbox fault image recognition method and system. Background Art

[0002] As an important component of clean energy, wind power generation is of great significance to promoting energy structure transformation and reducing carbon emissions. As the core component of wind turbine generator sets, the operating status of wind turbine gearboxes is directly related to the efficiency and life of the entire power generation system, and is the key to ensuring the stable and efficient operation of wind power generation. Wind turbine gearbox fault diagnosis technology aims to timely detect potential problems of gearboxes through various sensors and analysis methods, prevent major failures, thereby reducing maintenance costs and improving the reliability and power generation efficiency of wind turbines.

[0003] At present, wind turbine gearbox fault diagnosis mainly relies on three traditional methods: vibration analysis, oil analysis and temperature monitoring. These methods have shown certain effectiveness in practical applications and provided important support for the operation and maintenance management of wind farms.

[0004] Although the above traditional methods have played a certain role in practice, they all have obvious limitations, such as: The lack of sensitivity in vibration analysis, the inability to achieve real-time monitoring of oil analysis, and the lag of temperature monitoring before the occurrence of faults have restricted the accuracy and timeliness of traditional methods in fault diagnosis. Most existing fault diagnosis systems rely on a single data source (such as vibration signals, oil samples or temperature data) and a single analysis method, which makes it difficult to fully and accurately capture the complex working state of the gearbox. This singleness leads to one-sidedness and uncertainty in fault diagnosis.

[0005] In summary, the existing fault diagnosis method for wind turbine gearbox has the problems of insufficient accuracy and untimely diagnosis. Summary of the invention

[0006] In order to overcome the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a method and system for wind turbine gearbox fault image recognition, which can timely realize the fault diagnosis of the wind turbine gearbox and the diagnosis result has good accuracy.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for identifying a fault image of a wind turbine gearbox, comprising the following steps: Collect multimodal data of wind turbine gearbox; Preprocessing the multimodal data to obtain preprocessed multimodal data; Perform deep learning analysis on the preprocessed multimodal data to obtain deep learning results; Perform fault reasoning based on expert knowledge and historical data to obtain knowledge graph reasoning results; Fault diagnosis is performed based on deep learning results and graph recognition reasoning results.

[0008] In a second aspect, the present invention provides a wind turbine gearbox fault image recognition system, comprising: Data acquisition module: used to collect multi-modal data of wind turbine gearbox; Preprocessing module: used to preprocess multimodal data to obtain preprocessed multimodal data; Deep learning module: used to perform deep learning analysis on preprocessed multimodal data to obtain deep learning results; Graph reasoning module: used to perform fault reasoning based on expert knowledge and historical data to obtain knowledge graph reasoning results; Fault diagnosis module: used to perform fault diagnosis based on deep learning results and graph recognition reasoning results.

[0009] Compared with the prior art, the present invention has the following beneficial effects: 1. The method of the present invention collects multimodal data of the wind turbine gearbox. By collecting multimodal data, the operating status of the wind turbine gearbox can be more comprehensively reflected. Real-time data collection can ensure the timely discovery of potential faults and reduce losses caused by downtime due to faults. Multimodal data complement each other, improve the accuracy and reliability of data representation of faults, and provide more targeted input for subsequent deep learning analysis. Deep learning analysis is performed on the pre-processed multimodal data to obtain deep learning results. The deep learning model can automatically learn and extract high-level feature representations from the data without the need for manual feature design. By training the deep learning model, the fault types of the wind turbine gearbox can be efficiently and accurately classified. Fault reasoning based on expert knowledge and historical data is performed to obtain knowledge graph reasoning results. Expert knowledge is combined with historical data to construct a knowledge graph to achieve knowledge fusion and sharing. Logical reasoning using the knowledge graph can discover potential associations and laws between data, providing more powerful support for fault diagnosis. Fault diagnosis is performed based on the deep learning results and knowledge graph reasoning results. Comprehensive judgment is made based on the deep learning results and knowledge graph reasoning results to improve the accuracy and reliability of fault diagnosis. By combining the advantages of multiple methods, the fault type and location can be identified more quickly, providing timely guidance for subsequent maintenance work.

[0010] 2. The system of the present invention includes: a data acquisition module, a preprocessing module, a deep learning module, a graph reasoning module and a fault diagnosis module. The data acquisition module is used to collect multimodal data of the fan gearbox; the preprocessing module is used to preprocess the multimodal data to obtain preprocessed multimodal data; the deep learning module is used to perform deep learning analysis on the preprocessed multimodal data to obtain deep learning results; the graph reasoning module is used to perform fault reasoning based on expert knowledge and historical data to obtain knowledge graph reasoning results; the fault diagnosis module is used to perform fault diagnosis based on deep learning results and knowledge graph reasoning results. The various modules cooperate with each other to realize the fault diagnosis of the fan gearbox in a timely manner, and the diagnosis results are accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a data acquisition and preprocessing module according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the core analysis and diagnosis module of an embodiment of the present invention; Figure 3 A schematic diagram of a decision support and interaction module according to an embodiment of the present invention; Figure 4 is a flow chart of the method of the present invention; Figure 5 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0013] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0014] The present invention is further described in detail below in conjunction with the accompanying drawings: See also Figure 4 The present invention discloses a method for identifying a fault image of a wind turbine gearbox, comprising the following steps: S1, collecting multimodal data of wind turbine gearbox; Preferably, the multimodal data includes image data, sound data and vibration data of the wind turbine gearbox, the image data is collected using a high-resolution spectral imaging system, the sound data is collected using an acoustic sensor array, and the vibration data is collected using a multidimensional vibration sensor network.

[0015] S2. Preprocess the multimodal data to obtain preprocessed multimodal data, as follows: Based on the local adaptive image enhancement algorithm of scene semantic segmentation, the improved wavelet packet transform denoising algorithm and the real-time image registration technology, the collected multimodal data are preliminarily cleaned, denoised and feature extracted to obtain the preprocessed multimodal data, as follows: 1. Image data processing: Adopting local adaptive image enhancement algorithm based on scene semantic segmentation: This algorithm is mainly used to process image data. It first uses a lightweight SegNet network to perform semantic segmentation on the image and identify key components such as gears and bearings. Then, it adaptively adjusts the contrast-limited adaptive histogram equalization (CLAHE) parameters for different component areas.

[0016] Real-time image registration technology: This technique is also mainly used to process image data, especially when you need to compare images taken at different times or from different angles. It can align multiple images for subsequent analysis.

[0017] 2. Sound data processing: Adopt improved wavelet packet transform denoising algorithm: This algorithm can be applied to the processing of sound data. It first uses empirical mode decomposition (EMD) to decompose the sound signal into multiple eigenmode functions, then performs wavelet packet transform on each IMF, selects the optimal basis, and implements multi-scale analysis. This helps to remove noise from sound data and extract useful acoustic features.

[0018] 3. Vibration data processing: Adopt improved wavelet packet transform denoising algorithm: Similarly, this algorithm is also applicable to the processing of vibration data. Vibration signals usually contain components of various frequencies, and wavelet packet transform can effectively decompose these components and remove noise through threshold processing.

[0019] In addition, for vibration data, some specific signal processing techniques may be used, such as: Time domain analysis: calculate statistical characteristics such as mean, variance, peak value, etc.; Frequency domain analysis: Use Fast Fourier Transform (FFT) to analyze the frequency components of vibration; Time-frequency analysis: Use short-time Fourier transform (STFT) or wavelet transform to analyze the time-frequency characteristics of vibration signals.

[0020] 4. Data Fusion: After processing various types of data, data fusion is performed to combine the data features of different modes to obtain more comprehensive fault diagnosis information.

[0021] In summary, the local adaptive image enhancement algorithm and real-time image registration technology based on scene semantic segmentation are mainly used to process image data, while the improved wavelet packet transform denoising algorithm can be applied to the processing of sound data and vibration data. Each data type has its own specific processing method, but there are also some general techniques (such as wavelet transform) that can be applied to multiple types of data. Ultimately, the purpose of these preprocessing steps is to clean the data, reduce noise, and extract useful features to provide high-quality input for subsequent deep learning analysis and fault diagnosis.

[0022] S3. Perform deep learning analysis on the preprocessed multimodal data to obtain deep learning results, as follows: Based on the improved YOLOv5 model and the improved ResNet deep convolutional neural network, deep learning analysis is performed on the preprocessed multimodal data to obtain deep learning results.

[0023] Preferably, the improved YOLOv5 model adopts CSPDarknet53 as the backbone network, introduces a cross-stage partial connection structure, and adopts a SE (Squeeze-and-Excitation) attention module; Preferably, the improved ResNet deep convolutional neural network introduces spatial and channel attention mechanisms, and the calculation process of the spatial attention mechanism includes generating feature maps, generating spatial attention maps and feature recalibration.

[0024] S4. Fault reasoning based on expert knowledge and historical data is performed to obtain the knowledge graph reasoning results, as follows: The semi-automatic ontology expansion algorithm based on text mining and the temporal fuzzy rule reasoning engine are combined with expert knowledge and historical data to perform fault reasoning and obtain the knowledge graph reasoning results, as follows: 1. Basic ontology construction: Use Protégé tool to build the basic ontology of wind turbine gearbox fault diagnosis; Define core concepts, relationships, and attributes, such as "gear", "bearing", "fault type", etc.

[0025] 2. Text mining and semi-automatic ontology expansion: a. Text data collection: Collect relevant technical documents, maintenance records, fault reports and other text materials.

[0026] b. Text preprocessing: Perform operations such as word segmentation, stop word removal, and word form restoration.

[0027] c.Term extraction: Use algorithms such as TF-IDF and TextRank to extract keywords and phrases.

[0028] d. Relationship extraction: Dependency parsing and pattern matching techniques are used to identify relationships between concepts.

[0029] e. Ontology mapping: Map the extracted terms and relations into the existing ontology structure.

[0030] f.Consistency check: Use the reasoning engine to check the consistency of the newly added knowledge with the existing ontology.

[0031] g. Expert review: The automatically extended ontology content is reviewed and validated by domain experts.

[0032] 3. Temporal fuzzy rule definition: a. Define language variables: Such as "vibration intensity", "temperature", etc., and their fuzzy sets.

[0033] b. Build fuzzy rule base: Write fuzzy rules in the form of IF-THEN based on expert knowledge; For example: "IF the vibration intensity is high AND the temperature is rising THEN the bearing may be faulty".

[0034] c. Timing operator definition: Define time series operators such as "before", "after", "continue", etc. Integrating temporal information into fuzzy rules.

[0035] 4. Historical data integration: Convert historical failure case data into instances in the knowledge graph; Establish correlations between failure symptoms, causes, and solutions.

[0036] 5. Inference engine construction: a. Fuzzification: Convert the input sensor data into fuzzy sets.

[0037] b. Rule matching: Use the Rete algorithm to efficiently match applicable fuzzy rules.

[0038] c. Temporal Reasoning: Considering the time series of symptoms, temporal operators are applied for reasoning.

[0039] d. Fuzzy reasoning: Reasoning is performed using Mamdani or Sugeno fuzzy inference systems.

[0040] e. Defuzzification: Convert fuzzy reasoning results into clear fault diagnosis results.

[0041] 6. Knowledge graph reasoning: a. Graph traversal: Based on the preliminary fault diagnosis results, graph traversal is performed in the knowledge graph.

[0042] b. Similar case search: Use graph embedding technology to retrieve similar historical failure cases.

[0043] c. Multi-path reasoning: Consider multiple possible failure paths and comprehensively evaluate the possibility of failure.

[0044] d. Confidence calculation: The confidence of each fault diagnosis is calculated by combining the fuzzy reasoning results and the graph reasoning results.

[0045] 7. Result integration and output: Summarize the results of fuzzy reasoning and graph reasoning; Generates a comprehensive report with fault type, probable cause, confidence level, and suggested solution.

[0046] 8. Continuous learning and updating: Update the knowledge graph and fuzzy rules based on actual diagnosis feedback; Use incremental learning algorithms to continuously optimize the inference model.

[0047] Through this detailed process, the system is able to combine expert knowledge, historical data and real-time sensor data to conduct comprehensive and accurate fault reasoning, and finally obtain reliable knowledge graph reasoning results. This method can not only handle known fault modes, but also discover potential new faults through the correlation of knowledge graphs, improving the adaptability and diagnostic capabilities of the system.

[0048] S5. Perform fault diagnosis based on deep learning results and graph recognition reasoning results, as follows: The improved Dempster-Shafer evidence theory is used to perform fault diagnosis based on the deep learning results and graph recognition reasoning results.

[0049] Preferably, the improved Dempster-Shafer evidence theory is as follows: The improved Dempster-Shafer evidence theory introduces an adaptive weight allocation mechanism for information entropy and mutual information, and the weight calculation takes into account the information entropy of each evidence source and the average mutual information with other evidence sources.

[0050] Preferably, after the fault diagnosis results are generated, maintenance recommendations and predictive maintenance plans are generated based on the fault diagnosis results.

[0051] See also Figure 4 In another feasible embodiment of the present invention, the following is adaptively modified according to the situation. It includes the following steps: Collecting multimodal data of the wind turbine gearbox can more comprehensively reflect the operating status of the wind turbine gearbox and provide a rich data source for subsequent analysis. Real-time data collection can ensure timely detection of potential faults and reduce losses caused by downtime. Multimodal data complement each other and improve the accuracy and reliability of data characterization of faults.

[0052] Multimodal data is preprocessed to obtain preprocessed multimodal data in order to reduce noise interference in the data and improve data quality. At the same time, by extracting key features, more targeted input is provided for subsequent deep learning analysis.

[0053] Deep learning analysis is performed on the preprocessed multimodal data to obtain deep learning results. The deep learning model can automatically learn from the data and extract high-level feature representations without the need for manual feature design. By training the deep learning model, the fault types of the wind turbine gearbox can be efficiently and accurately classified.

[0054] Fault reasoning based on expert knowledge and historical data obtains the knowledge graph reasoning result. Expert knowledge is combined with historical data to construct a knowledge graph, realize knowledge fusion and sharing, and use knowledge graph for logical reasoning to discover potential connections and patterns between data, providing more powerful support for fault diagnosis.

[0055] Fault diagnosis is performed based on the deep learning results and knowledge graph reasoning results. Comprehensive judgment is made based on the deep learning results and knowledge graph reasoning results to improve the accuracy and reliability of fault diagnosis. By combining the advantages of multiple methods, the fault type and location can be identified more quickly, providing timely guidance for subsequent maintenance work.

[0056] In summary, the present invention can timely implement fault diagnosis of the fan gearbox, and the diagnosis result has good accuracy.

[0057] Embodiment 1: See also Figure 4 This embodiment discloses a method for wind turbine gearbox fault image recognition, aiming to solve the problems existing in the prior art, such as low fault recognition accuracy, slow system response speed, and inability to effectively process multi-source heterogeneous data. The present invention adopts a distributed microservice architecture with cloud-edge collaboration, integrating advanced technologies such as multimodal data acquisition, edge computing, deep learning, and knowledge graph reasoning, to achieve high-precision recognition and predictive maintenance of wind turbine gearbox faults. It includes the following steps: The overall architecture of the platform can be divided into three main parts: data collection and preprocessing, core analysis and diagnosis, and decision support and interaction. The detailed structures of these three parts are as follows: Figure 1 , Figure 2 and Figure 3 shown.

[0058] S1. See Figure 1 , the workflow of the platform starts with the multimodal data acquisition module, which collects multi-source data such as images, sounds, and vibrations of the wind turbine gearbox. Here, a high-resolution spectral imaging system, an acoustic sensor array, and a multi-dimensional vibration sensor network are used. The high-resolution spectral imaging system uses a high-speed CMOS camera with a frame rate of 10,000 fps, combined with a liquid crystal tunable filter (LCTF) to achieve multi-spectral imaging. The acoustic sensor array consists of a spherical array of 24 MEMS microphones, covering a frequency range of 20Hz-20kHz. The multi-dimensional vibration sensor network installs a three-axis MEMS accelerometer on each key component of the gearbox, with a range of ±50g and a bandwidth of DC-5kHz. These sensors form a self-organizing wireless sensor network that uses the TSCH protocol based on the IEEE 802.15.4e standard to ensure that data transmission reliability is greater than 99.9%.

[0059] S2, see Figure 1 The collected data is then transmitted to the edge computing preprocessing module, where preliminary data cleaning, noise reduction and feature extraction are performed. This module uses a number of innovative technologies, including a local adaptive image enhancement algorithm based on scene semantic segmentation, an improved wavelet packet transform denoising algorithm and real-time image registration technology.

[0060] The adaptive image enhancement algorithm first uses a lightweight SegNet network to perform semantic segmentation on the image and identify key components such as gears and bearings. Then, it adaptively adjusts the contrast-limited adaptive histogram equalization (CLAHE) parameters for different component regions. Dynamically adjust according to the local features of the image, the calculation formula is:

[0061] in, is the contrast limit parameter of the CLAHE algorithm, represents the standard deviation of the local area, and It is an adjustable parameter.

[0062] The improved wavelet packet transform denoising algorithm first uses empirical mode decomposition (EMD) to decompose the signal into multiple intrinsic mode functions, which are referred to as IMFs. Then, each IMF is subjected to wavelet packet transform, and the optimal basis is selected to achieve multi-scale analysis. An improved soft threshold function is used for coefficient shrinkage:

[0063] in, is an adjustable parameter that controls the smoothness of the transition. is the wavelet coefficient, is the threshold, the threshold The choice of is based on Stein's unbiased risk estimate ( ):

[0064] in, is the Stein unbiased risk estimate, is the total number of coefficients, are the wavelet coefficients, is less than The number of coefficients.

[0065] S3, see Figure 2 ,The preprocessed data is sent to the cloud-based deep learning analysis module and the knowledge graph reasoning module at the same time. The cloud-based deep learning analysis module is the core of the platform, which uses the improved YOLOv5 model and ResNet deep convolutional neural network.

[0066] The improved YOLOv5 model uses CSPDarknet53 as the backbone network and introduces the cross-stage partial connection (CSP) structure. The SE (Squeeze-and-Excitation) attention module is introduced, and its calculation process is as follows: 1. Squeeze operation: ; in, is the global average of channel c (i.e. the output of the squeeze operation), is the squeeze operation function, is the two-dimensional feature map of channel c in the input feature map, is the height of the feature map, is the width of the feature map; and are the row and column indices of the feature map respectively; 2. Excitation operation:

[0067] in, For Channel The excitation weight (i.e., the output of the Excitation operation), is the Excitation operation function, is the sigmoid activation function, which is used to map the output to the (0,1) interval. is the ReLU activation function, which is used to introduce nonlinearity. is the weight matrix of the first fully connected layer, is the weight matrix of the second fully connected layer; Represents a set of learnable weight parameters; 3. Recalibration:

[0068] in, is the recalibrated feature map (output), is a scaling function that implements feature recalibration.

[0069] The loss function has also been improved, and Focal Loss has been added to solve the problem of category imbalance:

[0070] in, .

[0071] The improved ResNet deep convolutional neural network introduces spatial and channel attention mechanisms. The calculation process of the spatial attention mechanism is as follows: 1. Generate feature map: ,in ; in, is the entire feature map, containing information of all channels, is the set of real numbers, is the height of the feature map, is the width of the feature map.

[0072] 2. Generate spatial attention map: ; in, is the spatial attention map, is the sigmoid activation function, which maps the output to the (0,1) interval. is a 7×7 two-dimensional convolution operation, For the feature map Perform average pooling operation in channel dimension, For the feature map Perform the maximum pooling operation in the channel dimension 3. Feature recalibration: ; in, is the recalibrated feature map (output), is the original input feature map, The spatial attention map generated for the previous step; The channel attention mechanism uses:

[0073] in, is the channel attention weight vector, of size 1 × 1 × C (C is the number of channels in the feature map), is the sigmoid activation function, which maps the output to the (0, 1) interval. It is a multi-layer perceptron, which is usually a simple fully connected network; The knowledge graph reasoning module performs reasoning based on expert knowledge and historical data. The Protégé tool is used to build the basic ontology, a semi-automatic ontology expansion algorithm based on text mining is designed, and a consistency check mechanism is used to ensure the compatibility of the new knowledge with the existing ontology. In terms of reasoning, fuzzy timing operators are defined to describe the time evolution characteristics of fault symptoms, and a reasoning engine based on timing fuzzy rules is built.

[0074] S4. See Figure 2 , the multi-source information fusion diagnosis module comprehensively analyzes the output results of deep learning and knowledge graph reasoning to perform fault diagnosis. Here, the Dempster-Shafer evidence theory is improved, and the adaptive weight allocation mechanism of information entropy and mutual information is introduced:

[0075] The improved Dempster combination rule is:

[0076] in, is the evidence conflict measure, is the conflict adjustment coefficient, For the The weight of each source of evidence, For the The information entropy of each evidence source is For the The information entropy of each evidence source is Assume that after combination The basic probability assignment of is the normalization factor, For the The average mutual information between a source of evidence and other sources of evidence; For the The average mutual information between a source of evidence and other sources of evidence; and Assigning base probabilities to the two sources of evidence, and is a subset of the hypothesis set; S5. See Figure 3 ,The predictive maintenance decision module generates maintenance recommendations and predictive maintenance plans based on the diagnosis results.,A gearbox degradation model based on Markov process is constructed, and a multi-objective function is designed, including reliability index, maintenance cost and carbon emission.,The improved NSGA-III algorithm is used to solve the Pareto optimal solution set, and the fuzzy TOPSIS method is introduced to select the best,maintenance strategy from the Pareto solution set.

[0077] S6. See Figure 3 The human-computer interaction interface module uses WebGL technology to achieve real-time rendering and interaction of the gearbox 3D model, and uses the ECharts library to draw dynamic charts of multi-dimensional data. An augmented reality interaction system based on head-mounted AR devices has also been developed to provide on-site maintenance personnel with intuitive fault information display and support real-time guidance from remote experts.

[0078] In order to further improve the system performance, the quantum-classical hybrid computing framework and bionic learning algorithm module are introduced. The quantum-classical hybrid computing framework uses quantum Fourier transform to accelerate the image feature extraction process and implements the quantum principal component analysis (QPCA) algorithm to improve the efficiency of dimensionality reduction. The bionic learning algorithm module includes a multi-objective feature selection method based on the ant colony algorithm and a fault pattern identifier based on the artificial immune system, which improves the system's adaptability to new faults.

[0079] In terms of data security, blockchain technology is used to ensure the security and immutability of diagnostic data. The results of each fault diagnosis are recorded on the blockchain in the form of transactions, forming a traceable diagnostic history. At the same time, automated maintenance decisions are implemented based on smart contracts. When the fault diagnosis results meet the preset conditions, the corresponding maintenance process is automatically triggered.

[0080] Through practical application tests, this platform has significantly improved the accuracy and timeliness of wind turbine gearbox fault diagnosis. The average fault detection accuracy rate has increased to 98.5%, the false alarm rate has been reduced to less than 0.5%, and the missed alarm rate has been reduced to less than 0.1%. The system's real-time data processing delay has been reduced to less than 50ms, and the fault warning lead time has been increased to 72 hours before the fault occurs. In terms of economic benefits, the availability of wind turbines has increased from 95% to 98%, the average annual maintenance cost has been reduced by 20%, and the energy efficiency has increased by 3% annually.

[0081] In summary, the wind turbine gearbox fault image recognition expert platform proposed in this invention realizes high-precision identification and predictive maintenance of wind turbine gearbox faults through the organic combination of multiple innovative technologies, providing strong support for the intelligent upgrading of the wind power industry. In the future, the platform can further expand its application scope and be applicable to fault diagnosis and health management of other types of large rotating machinery and equipment.

[0082] Based on the above method, the present invention also discloses a wind turbine gearbox fault image recognition system, see Figure 5 ,include: Data acquisition module: used to collect multi-modal data of wind turbine gearbox; Preprocessing module: used to preprocess multimodal data to obtain preprocessed multimodal data; Deep learning module: used to perform deep learning analysis on preprocessed multimodal data to obtain deep learning results; Graph reasoning module: used to perform fault reasoning based on expert knowledge and historical data to obtain knowledge graph reasoning results; Fault diagnosis module: used to perform fault diagnosis based on deep learning results and graph recognition reasoning results.

[0083] The cooperation of various modules can timely realize the fault diagnosis of the fan gearbox, and the diagnosis result is accurate.

[0084] Embodiment 2: See also Figures 1 to 3 , this embodiment discloses a wind turbine gearbox fault image recognition system, comprising: Multimodal data acquisition module, used to collect image data, sound data and vibration data of the wind turbine gearbox; Wherein, the multimodal data acquisition module includes a high-resolution spectral imaging system, an acoustic sensor array and a multi-dimensional vibration sensor network; Preferably, the high-resolution spectral imaging system comprises a high-speed CMOS camera with a frame rate of 10,000 fps and a liquid crystal tunable filter, with a wavelength range of 400-1000 nm and a bandwidth adjustable between 10-20 nm.

[0085] Preferably, the acoustic sensing array is composed of 24 MEMS microphones forming a spherical array, covering a frequency range of 20 Hz-20 kHz, using a delay-sum beamforming algorithm, and a spatial resolution of 5°.

[0086] Preferably, the multi-dimensional vibration sensing network installs a three-axis MEMS acceleration sensor on each key component of the gearbox, with a range of ±50g and a bandwidth of DC-5kHz, and uses the TSCH protocol based on the IEEE 802.15.4e standard to build a wireless sensor network.

[0087] Edge computing preprocessing module, used for preliminary data cleaning, noise reduction and feature extraction of collected multimodal data; The edge computing preprocessing module adopts a local adaptive image enhancement algorithm based on scene semantic segmentation, an improved wavelet packet transform denoising algorithm and a real-time image registration technology; Preferably, the contrast limiting parameters of the CLAHE algorithm in the local adaptive image enhancement algorithm are dynamically adjusted according to local features of the image, wherein the local features include the standard deviation of the local area.

[0088] Preferably, a quantum-classical hybrid computing framework is also included to accelerate the image feature extraction and dimensionality reduction process, and the framework includes quantum Fourier transform and quantum principal component analysis algorithms.

[0089] Cloud-based deep learning analysis module, used to perform deep learning analysis on preprocessed data; The cloud-based deep learning analysis module includes an improved YOLOv5 model and an improved ResNet deep convolutional neural network; Preferably, the improved YOLOv5 model adopts CSPDarknet53 as the backbone network, introduces a cross-stage partial connection structure, and adopts a SE (Squeeze-and-Excitation) attention module.

[0090] Preferably, the improved ResNet deep convolutional neural network introduces spatial and channel attention mechanisms, and the calculation process of the spatial attention mechanism includes generating feature maps, generating spatial attention maps and feature recalibration.

[0091] Knowledge graph reasoning module, used for fault reasoning based on expert knowledge and historical data; The knowledge graph reasoning module adopts a semi-automatic ontology expansion algorithm based on text mining and a temporal fuzzy rule reasoning engine; Multi-source information fusion diagnosis module, which is used to comprehensively analyze the output results of deep learning and knowledge graph reasoning to perform fault diagnosis; The multi-source information fusion diagnosis module adopts the improved Dempster-Shafer evidence theory.

[0092] Preferably, the improved Dempster-Shafer evidence theory introduces an adaptive weight allocation mechanism of information entropy and mutual information, and the weight calculation takes into account the information entropy of each evidence source and the average mutual information with other evidence sources.

[0093] Predictive maintenance decision module, used to generate maintenance recommendations and predictive maintenance plans based on fault diagnosis results; Human-computer interaction interface module, used to display diagnostic results and maintenance suggestions, and provide user interaction functions; Preferably, the human-computer interaction interface module uses WebGL technology to achieve real-time rendering and interaction of the gearbox 3D model, uses the ECharts library to draw dynamic charts of multi-dimensional data, and includes an augmented reality interaction system based on a head-mounted AR device to support real-time guidance functions by remote experts.

[0094] In summary, the present invention significantly improves the comprehensiveness and accuracy of fault diagnosis through multimodal data acquisition and fusion technology. The edge computing preprocessing module greatly improves the data processing efficiency and reduces the data transmission burden. The combination of cloud-based deep learning analysis and knowledge graph reasoning enhances the system's ability to identify complex fault modes. The multi-source information fusion diagnosis module further improves the reliability of the diagnosis results. The predictive maintenance decision-making function effectively reduces maintenance costs and extends equipment life. The optimization of the human-computer interaction interface improves the system's operability and decision-making support capabilities.

[0095] Overall, this platform has realized the intelligent and precise diagnosis of wind turbine gearbox faults, greatly improved the operational efficiency and economic benefits of wind farms, and provided strong support for the sustainable development of the wind power industry.

[0096] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for identifying fault images of a wind turbine gearbox, characterized in that: The following steps are involved: Collect multimodal data of wind turbine gearbox; Preprocessing the multimodal data to obtain preprocessed multimodal data; Perform deep learning analysis on the preprocessed multimodal data to obtain deep learning results; Perform fault reasoning based on expert knowledge and historical data to obtain knowledge graph reasoning results; Fault diagnosis is performed based on deep learning results and graph recognition reasoning results.

2. The wind turbine gearbox fault image recognition method according to claim 1 is characterized in that: The multimodal data includes image data, sound data and vibration data of the wind turbine gearbox. The image data is collected using a high-resolution spectral imaging system, the sound data is collected using an acoustic sensor array, and the vibration data is collected using a multidimensional vibration sensor network.

3. The wind turbine gearbox fault image recognition method according to claim 1, characterized in that: The steps of preprocessing the multimodal data to obtain preprocessed multimodal data are specifically as follows: A local adaptive image enhancement algorithm based on scene semantic segmentation, an improved wavelet packet transform denoising algorithm and real-time image registration technology are used to perform preliminary data cleaning, denoising and feature extraction on the collected multimodal data to obtain preprocessed multimodal data.

4. The wind turbine gearbox fault image recognition method according to claim 1, characterized in that: The steps of performing deep learning analysis on the pre-processed multimodal data to obtain deep learning results are specifically as follows: Based on the improved YOLOv5 model and the improved ResNet deep convolutional neural network, deep learning analysis is performed on the preprocessed multimodal data to obtain deep learning results.

5. The wind turbine gearbox fault image recognition method according to claim 4 is characterized in that: The improved YOLOv5 model and the improved ResNet deep convolutional neural network are specifically as follows: The improved YOLOv5 model adopts CSPDarknet53 as the backbone network, introduces a cross-stage partial connection structure, and adopts the SE (Squeeze-and-Excitation) attention module; The improved ResNet deep convolutional neural network introduces spatial and channel attention mechanisms. The calculation process of the spatial attention mechanism includes generating feature maps, generating spatial attention maps and feature recalibration.

6. The wind turbine gearbox fault image recognition method according to claim 1, characterized in that: The steps of performing fault reasoning based on expert knowledge and historical data to obtain knowledge graph reasoning results are as follows: A semi-automatic ontology expansion algorithm based on text mining and a temporal fuzzy rule reasoning engine are combined with expert knowledge and historical data to perform fault reasoning and obtain knowledge graph reasoning results.

7. The wind turbine gearbox fault image recognition method according to claim 1, characterized in that: The steps of performing fault diagnosis based on the deep learning results and the graph recognition and reasoning results are as follows: The improved Dempster-Shafer evidence theory is used to perform fault diagnosis based on the deep learning results and graph recognition reasoning results.

8. The wind turbine gearbox fault image recognition method according to claim 7, characterized in that: The improved Dempster-Shafer evidence theory is as follows: The improved Dempster-Shafer evidence theory introduces an adaptive weight allocation mechanism for information entropy and mutual information, and the weight calculation takes into account the information entropy of each evidence source and the average mutual information with other evidence sources.

9. The wind turbine gearbox fault image recognition method according to claim 1, characterized in that: The step of performing fault diagnosis based on deep learning results and graph recognition reasoning results generates a maintenance suggestion and a predictive maintenance plan based on the fault diagnosis result.

10. A wind turbine gearbox fault image recognition system, characterized in that: include: Data acquisition module: used to collect multi-modal data of wind turbine gearbox; Preprocessing module: used to preprocess multimodal data to obtain preprocessed multimodal data; Deep learning module: used to perform deep learning analysis on preprocessed multimodal data to obtain deep learning results; Graph reasoning module: used to perform fault reasoning based on expert knowledge and historical data to obtain knowledge graph reasoning results; Fault diagnosis module: used to perform fault diagnosis based on deep learning results and graph recognition reasoning results.

Images