Fault diagnosis method of wind driven generator based on multi-mode large language model

Abstract

The invention relates to the field of mechanical detection, in particular to a fault diagnosis method for a wind driven generator based on a multi-modal large language model. According to the research, a GPT-3-based data set construction scheme is innovatively provided, and PDF-format mechanical fault related literatures are effectively converted into a structured high-quality question and answer pair data set. Then, fine adjustment is carried out on the base large language model by utilizing a data set, and the performance of the model in a professional task of mechanical fault diagnosis is successfully improved through fine adjustment parameter adjustment and exploration of a fine adjustment method. Besides, according to an actual use scene, a signal-to-text module is expanded, so that the signal-to-text module can receive multi-modal data of mechanical faults in real time, feature information contained in digital signals is changed into a text form and transmitted to a large language model module, and finally a diagnosis result is output by a large language model.

Description

Technical Field

[0001] The invention belongs to the field of machinery, and in particular relates to a fault diagnosis method for a wind turbine generator based on a multi-modal large language model. Background Art

[0002] As the proportion of wind turbine power generation increases, the demand for wind turbine fault diagnosis is also growing. Since wind turbines are often located in environments that are not convenient for manual diagnosis, many wind turbines have long-term faults, resulting in reduced power generation efficiency, which poses a challenge to the rational development of clean energy. Existing wind power fault detection relies on maintenance personnel entering the generator head for manual diagnosis, which has certain risks.

[0003] Large language models have attracted a lot of research and use by scholars due to their powerful language understanding and reasoning capabilities, low dependence on fine-tuning training data, and strong scalability. Through deep learning and massive data training, large language models can generate content that is highly in line with user needs. The LLama large language model is an open source large language model based on the Transformer architecture developed by Meta. It has demonstrated strong performance in a variety of NLP tasks, including text generation, question answering, and summarization. In addition, the LLama model is also widely used in multimodal tasks, providing a solid foundation for achieving powerful language understanding and generation capabilities. Alibaba's Tongyi Qianwen has occupied a place in the field of Chinese large language models with its open source strategy and high performance, promoting low-cost, easy-to-deploy commercial solutions. Google's PaLM series of models have reached new heights in both parameter quantity and performance. Among them, PaLM-E integrates multimodal processing capabilities, opening up new possibilities for the application of LLM.

[0004] With the enhancement of the functions and performance of large language models, fine-tuned versions of large language models (LLMs) have been widely developed and applied to meet the specific needs of specific industries. In the financial field, there are models such as bloombergGPT, FinGPT, Xuanyuan 2.0, AntFinGLM, etc. The fine-tuning models in the financial field are aimed at improving the model's capabilities in financial data analysis, risk assessment, investment decision-making, etc. In the medical field, Med-PaLM, developed by Google Research and Deep Mind, is an expert-level medical question-answering large language model based on the PaLM base. It is the first model to exceed the "pass" score of the US Medical Licensing Examination sample questions. Medical dialogue models such as DoctorGLM and ClinicalGPT assist experts in the medical field. In the model evaluation, it was evaluated by experts with a medical background, showing that it has high security and knowledge availability. The Chinese Materia Medica Large Language Model was developed by the relevant team using the Chinese medical knowledge graph of CMeKG and sampling knowledge instances from the knowledge graph. With the rapid development of intelligent manufacturing, the complexity and precision of wind turbines are increasing, and higher requirements are also placed on the accuracy and timeliness of fault diagnosis. Based on in-depth research on the extensive applications and achievements of big language models in the financial and medical fields, we combine the advantages of big language models to meet the challenges in the field of wind turbine fault diagnosis.

[0005] In the field of wind turbine fault diagnosis, deep learning models, especially convolutional neural networks (CNN), recurrent neural networks (RNN) and their variants, are widely used to extract the operating characteristics of equipment and identify potential fault modes, so as to achieve fast and accurate fault diagnosis. For example, a fault diagnosis method combining CNN and long short-term memory network (LSTM) is used. CNN is used to extract spatial features from vibration signals, and LSTM network captures the temporal dynamic characteristics in time series data. RNN is also used to build an autoencoder to learn the essential characteristics of bearing vibration signals and diagnose faults by comparing the difference between the reconstructed signal and the original signal. And the advanced feature extraction capability of the Transformer algorithm is used to effectively identify known fault types and diagnose previously unseen fault types. In order to improve the generalization ability of intelligent fault diagnosis, deep transfer learning consisting of transfer learning and deep learning components has been developed accordingly.

[0006] Deep learning has made some progress in the field of wind turbine fault diagnosis, but it still faces many challenges in practical applications. First, the data imbalance problem seriously restricts the performance of deep learning models, making it easy for the model to favor normal data and ignore fault data during training. Secondly, the available data is in a single form, and it is impossible to analyze and diagnose multimodal data. In addition, the deep learning model has poor interpretability, and its complex internal nonlinear mapping relationship makes it difficult for people to understand how the model makes decisions. Therefore, when the large language model of the prior art is applied to the fault diagnosis of wind turbines, there is a problem of low fault detection accuracy. Summary of the invention

[0007] The purpose of the present invention is to solve the problem of low fault detection accuracy when the large language model of the prior art is applied to the fault diagnosis of wind turbines. A fault diagnosis method for wind turbines based on a multimodal large language model is proposed, comprising:

[0008] 1. Construct a large language model for wind turbine fault diagnosis; obtain a trained large language model for wind turbine fault diagnosis;

[0009] 2. Collect the data of the wind turbine to be diagnosed, input the trained large language model for wind turbine fault diagnosis, and obtain the fault diagnosis results and fault solutions;

[0010] The specific process of constructing a large language model for wind turbine fault diagnosis in the first step and obtaining a trained large language model for wind turbine fault diagnosis is as follows:

[0011] S1: Get the training set;

[0012] S2: Construct a large language model for wind turbine fault diagnosis; train the large language model for wind turbine fault diagnosis according to the training set to obtain a trained large language model for wind turbine fault diagnosis.

[0013] The specific process of obtaining the training set in S1 is:

[0014] S1.1: Obtain public vibration analysis text data, and build a fault text dataset based on the public vibration analysis text data;

[0015] S1.2: Build a vibration analysis test bench and obtain a fault simulation signal data set based on the vibration analysis test bench;

[0016] S1.3: Combine the fault text dataset and the fault simulation signal dataset into a training set.

[0017] The beneficial effects of the present invention are:

[0018] This study constructs a customized multimodal large language model (MLLM) for the field of wind turbine fault diagnosis, aiming to provide innovative solutions and theoretical support for the field of mechanical fault diagnosis and intelligent operation and maintenance by leveraging MLLM's outstanding capabilities in text understanding and learning. This study innovatively adopts an efficient dataset construction strategy to batch convert mechanical fault diagnosis-related literature in PDF format into structured fault diagnosis question-answer pair datasets. The dataset generation method proposed in this paper not only significantly improves the efficiency and scale of dataset generation, but also provides a paradigm for the construction of cross-domain fine-tuning datasets. The experimental results show that the mechanical fault diagnosis large language model proposed in this study shows excellent performance in the third-level professional test set of vibration analysis, successfully passes the evaluation criteria of the third-level vibration analyst, and also shows accuracy and rigor in signal data processing that exceeds other mainstream large language models.

[0019] The present invention proposes a question-answer pair-dataset automatic conversion process, which can quickly obtain a professional high-quality fault diagnosis fine-tuning data set; it also proposes a signal-to-text module, which can actively collect multimodal digital signals and convert signal features into descriptive text, thereby expanding the available data forms; the present invention utilizes the language and reasoning capabilities of a large language model to provide a rigorous logical reasoning basis, as well as final results and suggestions, making the results highly interpretable, thereby solving the problem of low fault detection accuracy when the large language model of the prior art is applied to the fault diagnosis of wind turbines. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the overall structure of the model of the present invention;

[0021] Figure 2 is a signal processing schematic diagram of the present invention;

[0022] Figure 3 is a schematic diagram of a signal image of the present invention;

[0023] Figure 4 It is a schematic diagram of the structure of llama3-8b of the present invention;

[0024] Figure 5 It is a schematic diagram of the structure of the fault diagnosis language model of the method;

[0025] Figure 6 It is a schematic diagram of the experimental platform of the present invention

[0026] Figure 7 It is a schematic diagram of comparing the accuracy of the vibration theory basis of the present invention;

[0027] Figure 8It is a comparative schematic diagram of the specific performance of the present invention;

[0028] Fig. 9 It is a schematic diagram comparing the signal data analysis of the present invention and the mainstream large language model;

[0029] Fig.10 It is a schematic diagram comparing the five major capabilities of the model of the present invention. DETAILED DESCRIPTION

[0030] Specific implementation method 1: Combination Figure 1 The invention is described, comprising:

[0031] 1. Construct a large language model for wind turbine fault diagnosis; obtain a trained large language model for wind turbine fault diagnosis;

[0032] 2. Collect the data of the wind turbine to be diagnosed, input the trained large language model for wind turbine fault diagnosis, and obtain the fault diagnosis results and fault solutions.

[0033] The following is a brief introduction to the working principle of wind turbines: After the blades of wind turbines capture wind energy, they transfer the energy to the gearbox through the rotor axis of the main shaft, and the gearbox transfers the rotational energy to the generator to complete the power generation. Wind turbines are responsible for the conversion of huge energy. For example, when there is a fault in the gearbox, it will directly affect the overall power generation efficiency of the wind turbine.

[0034] The present invention can timely discover faults existing in wind turbines. The trained wind turbine fault diagnosis large language model of the present invention can acquire signal data in real time and then convert it into descriptive text. The large language model then analyzes and outputs the results, and fault detection can be performed without stopping the generator.

[0035] In addition, the present invention can also reduce the number of times maintenance personnel go up to the head of the wind turbine for manual diagnosis, thereby reducing the risks for maintenance personnel.

[0036] Furthermore, the large language model for wind turbine fault diagnosis trained by the present invention can also provide reasonable fault solution methods according to real-time fault signals, helping engineers to adopt reasonable solutions during maintenance, reducing losses caused by improper solutions and improving the economy and safety of wind turbines.

[0037] Specific implementation method 2: The difference between this implementation method and specific implementation method 1 is that:

[0038] The specific process of constructing a large language model for wind turbine fault diagnosis in the first step and obtaining a trained large language model for wind turbine fault diagnosis is as follows:

[0039] S1: Get the training set;

[0040] S2: Construct a large language model for wind turbine fault diagnosis; train the large language model for wind turbine fault diagnosis based on the training set to obtain a trained large language model for wind turbine fault diagnosis

[0041] The other steps and parameters are the same as those in the first specific implementation.

[0042] Specific implementation method three: This implementation method is different from the specific implementation method one in that:

[0043] The specific process of obtaining the training set in S1 is:

[0044] S1.1: Obtain public vibration analysis text data, and build a fault text dataset based on the public vibration analysis text data;

[0045] S1.2: Build a vibration analysis test bench and obtain a fault simulation signal data set based on the vibration analysis test bench;

[0046] S1.3: Combine the fault text dataset and the fault simulation signal dataset into a training set.

[0047] The other steps and parameters are the same as those in the first to second embodiments.

[0048] Specific implementation method 4: This implementation method is different from specific implementation methods 1 to 4 in that:

[0049] The vibration analysis text data disclosed in S1.1 includes: authoritative training materials for vibration analysts, internationally recognized standardized documents and training materials;

[0050] The fault text data set is constructed based on the public vibration analysis text data; the specific process is:

[0051] Convert the public vibration analysis text data into a fault text dataset in the form of Question-Answer (question-answer pair) suitable for model fine-tuning;

[0052] The information of the wind turbine equipment usually includes the following aspects:

[0053] 1. Basic parameters include:

[0054] Rated power: The output power of the generator at rated wind speed.

[0055] Starting wind speed: The minimum wind speed required for a wind turbine to start generating electricity.

[0056] Rated wind speed: The wind speed at which the generator reaches rated power.

[0057] Cut-out wind speed: When the wind speed reaches this value, the generator automatically stops working to protect the equipment.

[0058] 2. The structure includes:

[0059] Blades: Usually made of composite materials and designed to optimize aerodynamic performance.

[0060] Nacelle: Contains the generator, transmission and control systems.

[0061] Tower: The structure, usually made of steel or concrete, that supports the nacelle and blades.

[0062] Base: used to fix the tower and ensure the stability of the equipment.

[0063] 3. Materials include:

[0064] Blade materials: such as glass fiber, carbon fiber composite materials, etc.

[0065] Nacelle material: usually steel or aluminum alloy.

[0066] Coating: Anti-corrosion and UV protection coating.

[0067] 4. The control system includes:

[0068] Monitoring system: real-time monitoring of wind speed, power generation, temperature and other parameters.

[0069] Adjustment System: Adjusts blade angle to optimize power output and protect equipment.

[0070] 5. Installation and maintenance include:

[0071] Installation method: such as ground installation, offshore installation, etc.

[0072] Maintenance Requirements: Regular inspection and maintenance procedures including lubrication, cleaning and replacement of parts.

[0073] 6. Environmental adaptability includes:

[0074] Wind resistance: Designed to handle strong winds and extreme weather.

[0075] Noise Level: Factors that affect the surrounding environment.

[0076] 7. Technical standards include:

[0077] Industry standards: Comply with relevant national and international standards (such as IEC 61400 series).

[0078] Certifications: Quality and safety certifications obtained, such as ISO certification, etc.

[0079] The specific process of converting the public vibration analysis text data into a fault text data set in the form of Question-Answer (question-answer pair) suitable for model fine-tuning is: writing a python automatic conversion program to convert the PDF data into a question-answer pair data set

[0080] The vibration analyst Mobius Institute, as an authoritative organization that has won the dual certification of ISO / IEC 17024 and ISO 18436-1, has been deeply engaged in the field of vibration analysis for many years and is committed to providing a vibration analyst certification system covering four progressive levels from CAT-I to CAT-IV. This certification system is not only a comprehensive examination of the depth and breadth of students' theoretical knowledge, but also emphasizes the accumulation of their practical experience and the mastery of professional skills. By participating in and successfully passing the certification exam carefully designed by the organization, students will obtain the internationally recognized vibration analyst qualification certificate, which is a high recognition of their professional ability. As a leader in the field of vibration analysis education, Mobius Institute has carefully compiled a series of professional training materials. These materials are complete in system and detailed in content, covering many key areas such as basic theory of vibration analysis, high-precision data acquisition technology, advanced signal processing algorithms, real-time condition monitoring strategies, and precise fault diagnosis methods. These materials purchased through legal channels, such as "CAT II: BASIC MACHINERY VIBRATIONS: SI UNITS TRAINING MANUAL", provide a solid knowledge foundation for the large language model.

[0081] In the field of technology, international standards play a vital role in ensuring the consistency and reliability of technical practices. This study focuses on the field of mechanical vibration analysis and fault diagnosis, and systematically collects widely recognized international standards and specification systems in this field, especially ISO 10816 (Evaluation standard for mechanical vibration) and ISO 13373 (Standardized criteria for vibration diagnosis). These authoritative standard documents are obtained through official certification channels, such as the official websites of the International Organization for Standardization (ISO) and the American National Standards Institute (ANSI), ensuring the authority and timeliness of the information. In addition, for the collected standard information, we strictly abide by the corresponding copyright regulations and authorization procedures to ensure the legality and compliance during use. In the data conversion process, this study carefully designed two key stages to efficiently convert the original PDF data into a question-answer pair JSON dataset suitable for fine-tuning the large language model. The conversion of relevant PDF data to the question-answer pair fine-tuning dataset provides a solid foundation for building a high-precision and highly professional large language model.

[0082] The method converts the public vibration analysis text data into a fault text data set in the form of Question-Answer (question-answer pair) suitable for model fine-tuning;

[0083] The specific process is:

[0084] PDF materials include image-based PDF and text-based PDF.

[0085] Use Python's PDF operation library to open the PDF. If it is a picture-type PDF, use Python's OCR library to convert it to text type and save it to a txt file.

[0086] If it is a text-based PDF, extract the text directly into a txt file.

[0087] Then we use the 500 characters of the txt file to be read, passed into GPT3.5, and converted into question-answer pairs with the prompt words. Our program receives the returned question-answer data and saves it to another txt file again.

[0088] The question-answer pair format is:

[0089] {'question':xxxx? ,'answer':'xxxx'}

[0090] The vibration analysis test bench is constructed in S1.2, and the specific process is: constructing a vibration analysis test bench of a corresponding type according to the type of the wind turbine to be diagnosed;

[0091] For example, when the wind turbine to be diagnosed is a bearing-type wind turbine, the structure of the bearing vibration analysis test bench constructed includes: a drive motor, a coupling device, a radial loading seat, an aluminum disc bracket, a bearing fault seat and an acceleration sensor;

[0092] The drive motor is used to provide power to rotate the system.

[0093] The coupling device is used to connect the drive motor and other parts of the system to transmit power.

[0094] The radial load seat is used to apply radial loads to simulate the load conditions under actual operating conditions.

[0095] The aluminum disk bracket is used to fix the aluminum disk, which rotates in the system to generate friction and other mechanical effects.

[0096] The bearing fault seat is used to simulate bearing faults such as wear, cracks, etc. to generate specific signals.

[0097] Accelerometers are used to measure the vibration and acceleration of the system, thereby capturing signals related to bearing faults;

[0098] The fault simulation signal data set includes I fault simulation signal data, where I is a positive integer

[0099] The other steps and parameters are the same as those in the first to third embodiments.

[0100] Specific implementation method 5: This implementation method is different from specific implementation methods 1 to 4 in that:

[0101] The wind turbine fault diagnosis large language model in S2 includes: a signal conversion text module and a large language model module; the wind turbine fault diagnosis large language model is trained according to the training set to obtain the trained wind turbine fault diagnosis large language model. The specific process is:

[0102] S2.1: Input the signal conversion text module into the fault simulation signal data set in the training set to obtain the feature description text set;

[0103] S2.2: Input the feature description text set and the fault text data set of the training set into the large language model module to obtain the fault diagnosis results and fault solutions;

[0104] S2.3: Train the large language model module to obtain a trained large language model for wind turbine fault diagnosis

[0105] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0106] Specific implementation method 6: This implementation method is different from specific implementation methods 1 to 5 in that:

[0107] The fundamental frequency A is the lowest frequency component in the vibration signal and usually corresponds to the natural frequency or main vibration mode of the system.

[0108] The fundamental frequency amplitude B reflects the strength of the fundamental frequency component in the vibration signal.

[0109] The fundamental frequency phase information C describes the offset of the vibration signal relative to a certain reference point, and plays a key role in understanding the waveform shape and timing relationship of the vibration signal.

[0110] Harmonics D and E are integer multiple frequency components of the fundamental frequency. They often appear in vibration signals and may provide additional information about the nonlinear behavior, damping characteristics or failure modes of the system. The following are the numerical calculation processes:

[0111] The fundamental frequency A is the frequency corresponding to the maximum value in the amplitude spectrum found by fast Fourier transform (FFT).

[0112] In S2.1, the signal conversion text module is input into the fault simulation signal data set in the training set to obtain the feature description text set. The specific process is:

[0113] S2.1.1: performing baseband extraction processing on each fault simulation signal in the fault simulation signal data set to obtain the baseband of each fault simulation signal;

[0114] S2.1.2: According to the fundamental frequency of each fault simulation signal obtained in S2.1.1, extract the fundamental frequency amplitude and fundamental frequency phase corresponding to the fundamental frequency of each fault simulation signal;

[0115] S2.1.3: Extract the harmonic frequency of each fault simulation signal according to the fundamental frequency of each fault simulation signal obtained in S2.1.1;

[0116] S2.1.4: extracting the multiple harmonic amplitude of the multiple harmonic frequency of each fault simulation signal according to the multiple harmonic frequency of each fault simulation signal;

[0117] S2.1.5: Construct a feature description text set based on the fundamental frequency, fundamental frequency amplitude, fundamental frequency phase, multiple harmonic frequency, and multiple harmonic amplitude of each fault simulation signal obtained from S2.1.1 to S2.1.4;

[0118] For example, the feature description text of the i-th fault simulation signal is expressed as:

[0119] "Information about the fundamental frequency and harmonics of the signal:

[0120] Base Frequency: Approximately 57.86Hz

[0121] Base Amplitude: about 13941.49

[0122] Base Phase: Approximately 1.53 radians

[0123] The following is the information of 2nd to 5th harmonics (after retaining two decimal places):

[0124] 2nd Harmonic:

[0125] Frequency: 115.72Hz

[0126] Amplitude:31.78

[0127] Phase: -2.31 radians

[0128] 3rd Harmonic:

[0129] Frequency: 173.57Hz

[0130] Amplitude:11.81

[0131] Phase: 1.40 radians

[0132] 4th Harmonic:

[0133] Frequency: 231.43Hz

[0134] Amplitude:1.14

[0135] Phase: -0.64 radians

[0136] 5th Harmonic:

[0137] Frequency: 289.29Hz

[0138] Amplitude:1.32

[0139] Phase: 3.01 radians"

[0140] ; Other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0141] Specific implementation method 7: This implementation method is different from specific implementation methods 1 to 6 in that:

[0142] In S2.1.1, the ith fault simulation signal f i The specific process of performing base frequency extraction processing to obtain the base frequency Ai of the i-th fault simulation signal is as follows:

[0143] S2.1.1.1: Perform fast Fourier transform on the ith fault simulation signal to obtain an amplitude spectrum of the ith fault simulation signal;

[0144] S2.1.1.2: Extract the frequency corresponding to the maximum value in the amplitude spectrum of the i-th fault simulation signal as the fundamental frequency A i ; i = 1, 2, ..., I; expressed as:

[0145] A i = argmax f |Y(f i )| (1)

[0146] In the formula, Y(f i ) represents the simulated signal f for the i-th fault i The result after fast Fourier transform, |Y(f i )|is Y(f i )’s amplitude spectrum, argmax f Indicates the frequency corresponding to the maximum value;

[0147] The baseband A in S2.1.2 i The fundamental frequency amplitude is denoted as B i ∈B; fundamental frequency A i The fundamental frequency phase is expressed as Ci ∈C; expressed by the formula:

[0148]

[0149] C i =∠Y(A i ) (3)

[0150] N represents the number of sampling points of the fault simulation signal when performing fast Fourier transform; ∠ is the sign of the phase angle, indicating the phase of the complex number;

[0151] Where, the fundamental frequency amplitude is the FFT amplitude value corresponding to the fundamental frequency. The fundamental frequency amplitude calculation needs to be divided by the total number of FFT points N and multiplied by an appropriate normalization factor (such as For a two-sided spectrum, or For one-sided spectra and considering symmetry);

[0152] The total number of points N is the number of sampling points of the input signal when performing the fast Fourier transform FFT. This value can be any positive integer, but in practical applications, it is usually chosen to be a power of 2 (such as 256, 512, 1024, 2048, etc.) because this can make the FFT calculation more efficient.

[0153] A fund is the fundamental amplitude, which refers to the amplitude at the fundamental frequency f fund For the convenience of reference, the present invention uses B to represent the fundamental frequency amplitude and uses Bx to represent the fundamental frequency amplitude of the xth fundamental frequency.

[0154] is the normalization factor.

[0155] In the case of a double-sided spectrum, the FFT result contains both positive and negative frequency components, so the total energy is distributed in N points. However, this study needs to consider the total energy of the entire frequency range, so after dividing by N, it must be multiplied by 2. Here, N is the number of FFT points.

[0156] The two-sided spectrum is a representation of the signal spectrum, which includes the spectrum information of the signal in the positive frequency and negative frequency ranges, and is commonly used in the case of using fast Fourier transform (FFT) for signal analysis;

[0157] f fund It is the representation of fundamental frequency, which refers to the frequency component with the largest amplitude in the signal spectrum and is usually directly related to the periodicity of the signal. fund ) is the signal at the fundamental frequency f fundThe FFT result at is a complex number that contains amplitude and phase information. fund )| is at the fundamental frequency f fund The absolute value or modulus of the FFT result at a certain frequency represents the amplitude of the signal at that frequency.

[0158] The fundamental frequency phase is the FFT phase value corresponding to the fundamental frequency:

[0159] In the formula, φ fund It is the fundamental phase, which refers to the fundamental frequency f fund The phase angle of the FFT result at . ∠ is the sign of the phase angle, indicating the phase of the complex number. In mathematics and signal processing, this symbol is used to represent the phase angle of a complex number. fund ) is the signal at the fundamental frequency f fund The FFT result at is a complex number, which includes amplitude and phase information. In order to simplify the representation, the fundamental frequency phase is represented by C in the present invention, and the fundamental frequency phase of the xth fundamental frequency is represented by Cx.

[0160] The n-fold harmonic frequency of the ith fault simulation signal in S2.1.3 is represented by D ni , n represents the harmonic multiplication factor, n = 2, 3, 4, ...;

[0161] The formula is:

[0162] D ni =n×A i (4)

[0163] In S2.1.4, the n-fold harmonic amplitude of the ith fault simulation signal is represented by E ni ,

[0164] The formula is:

[0165]

[0166] The harmonic amplitudes usually need to be estimated by interpolation, because the frequency resolution of the FFT may not be sufficient to locate the f directly. n n is an integer representing the number of harmonics expressed as multiples of the fundamental frequency. However, in most cases, the calculation requires the use of an interpolation method (such as linear interpolation) to estimate E ni :

[0167] E ni ≈Interpolator(D ni ) (6)

[0168] In the formula, Interpolator is a ni)| and fi are constructed. In addition, in actual analysis, it can be expanded according to frequency domain analysis and time domain analysis.

[0169] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0170] Specific implementation eight: This implementation differs from specific implementations one to seven in that:

[0171] The large language model module in S2.2 is the Llama3-8B large language model;

[0172] The model is built on a highly optimized autoregressive Transformer architecture. From the perspective of model parameter configuration, Llama3-8B demonstrates its large scale and sophisticated design, which is manifested in: it has a vocabulary of up to 128K, covering a wide range of language phenomena; the total number of model parameters reaches 8 billion (8B), ensuring a strong model capacity; through 32 layers of Transformer stacking, deep feature extraction and conversion are achieved; the hidden layer dimension is set to 4096, providing the model with rich internal representation capabilities; equipped with 32 attention heads, it effectively enhances the model's parallel processing capabilities for input information; at the same time, it supports the model to process 128,256 tokens, providing a solid foundation for processing long texts and complex dialogue scenarios. The specific structure is as follows: Figure 4 As shown:

[0173] This study further fine-tuned the Llama3-8B model to make it more focused on specific problems in the field of mechanical fault diagnosis, and to achieve more accurate and efficient fault diagnosis support by optimizing model parameters and training strategies. In terms of hardware architecture selection, this study uses NVIDIA A10 series graphics cards. In terms of memory configuration, this study configured 30GB of random access memory (RAM) and used the Ubuntu system as the underlying platform. The specific hardware configuration is shown in Table 1:

[0174] Table 1 Hardware conditions for fine-tuning large language models

[0175]

[0176] The other steps and parameters are the same as those in the first to seventh embodiments.

[0177] Specific implementation method 9: This implementation method is different from specific implementation methods 1 to 8 in that:

[0178] The algorithm for training the large language model module is the LoRA algorithm; the learning rate scheduling method for the large language model module is the Cosine learning rate scheduling method.

[0179] The large language model is fine-tuned using the fault diagnosis question-answer data set mentioned in Section 3, which has been converted to more than 5,000 items. Among the many fine-tuning techniques, LoRA (Low-Rank Adaptation) has attracted much attention due to its unique advantages. The core idea of ​​LoRA is to adapt to specific tasks or data sets by introducing low-rank matrices into the model without significantly increasing the number of parameters. This method allows the model to quickly adapt to new tasks or fields while maintaining the original pre-training knowledge.

[0180] Specifically, LoRA injects a trainable low-rank decomposition matrix into each layer of the Transformer architecture of the pre-trained model. LoRA is finally inserted into the Q, K, V, O matrices of the Self-Attention layer, and the rest of the positions such as MLP are not added. These low-rank matrices are used to capture knowledge for specific downstream tasks. During training, only the parameters of these low-rank matrices are updated, while the weights of the pre-trained model remain unchanged. This approach significantly reduces the number of parameters that need to be trained and reduces computational and storage costs.

[0181] The reasoning process of LoRA is to merge the trained low-rank matrix with the weights of the pre-trained model to obtain a fine-tuned model for a specific task. The original pre-trained parameters of the network are denoted as W 0 , after fine-tuning on the new downstream task, the parameters become W∈R d×k , it can be found that the parameter change ΔW can be expressed as

[0182] ΔW=WW 0 (7)

[0183] LoRA decomposes the parameter change matrix ΔW into two low-rank matrices multiplied by each other, as follows:

[0184] ΔW=BA (8)

[0185] Where B∈R d×r ,A∈R r×k .

[0186] Adding the LoRA parameters back to the original parameters can merge the LoRA and the original parameters of the model:

[0187] W 0 x+ΔWx=(W 0 +ΔW)x=Wx (9)

[0188] Among them, W 0 x is the original pre-trained parameters W 0The matrix multiplication of the input data x gives the output of the original model. Wx is the matrix multiplication of the combined parameter W and the input data x. ΔWx is the matrix multiplication of the parameter change ΔW and the input data x to obtain the output change caused by the LoRA adjustment. (W 0 +ΔW)x is the matrix multiplication of the merged new parameter matrix and the input data x, and the output of the fine-tuned model is obtained.

[0189] In addition, in terms of learning rate scheduling methods, this study chooses the Cosine learning rate scheduling method to optimize the model training process. The core idea of ​​the Cosine learning rate scheduling method is to simulate the change process of the learning rate as a periodic change of a cosine function. Specifically, during the training process, the learning rate starts from the initial value, gradually decreases to the minimum value according to the shape of the cosine function, and then gradually increases to near the initial value. This periodic adjustment strategy helps the model to update weights with different step sizes at different stages of training, thereby achieving a balance between exploring the loss function space and using the current optimal solution. The mathematical expression of Cosine learning rate scheduling is as follows:

[0190]

[0191] Where lr represents the current learning rate. η min and η max Respectively represent the minimum and maximum values ​​of the learning rate. cur Indicates the current number of training steps (or iterations). max Represents the number of steps in a cosine cycle, that is, the total number of steps required for the learning rate to decrease from the maximum value to the minimum value and then increase back to the maximum value.

[0192] In order to improve the training efficiency and final performance of the model, the training parameters selected in this study are shown in Table 2:

[0193] Table 2 Training parameter settings

[0194]

[0195]

[0196] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0197] Specific implementation method 10: This implementation method is different from specific implementation methods 1 to 9 in that:

[0198] The wind turbine generator data to be diagnosed in the second step is the vibration signal data of the bearing of the wind turbine generator to be diagnosed;

[0199] We use the vibration sensor installed on the wind turbine data to collect the vibration signal data of the wind turbine bearing to be diagnosed in real time, and record the acceleration, load, and speed of the wind turbine during its working process as the wind turbine data to be diagnosed for fault identification;

[0200] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0201] Combining the simulation analysis of specific implementation methods one to ten

[0202] Based on comprehensive considerations of industry recognition, skill level measurement, practical application ability testing, technological progress promotion, and user interest protection, this study selected the International Vibration Analyst Level 3 test. Level 3 test questions usually cover in-depth knowledge and advanced skills in the field of vibration analysis, including but not limited to vibration monitoring, fault diagnosis, data analysis, etc. In the assessment of international vibration analysts, the final total score is calculated on a percentage basis, and the reference personnel are required to score more than 70 points to pass the general level assessment. This study also uses such assessment standards to test the final model of this study. Requiring the model to achieve 70% accuracy means that the model needs to be able to accurately understand and process complex vibration data and identify potential problems in equipment or systems, which directly reflects the knowledge level and skill depth of the model.

[0203] This experiment summarizes the assessment objectives of the third-level vibration analyst into these five points: vibration theory foundation, fault diagnosis reasoning ability, vibration monitoring technology, data analysis method, equipment maintenance and management. The vibration theory foundation and fault diagnosis reasoning mainly rely on the knowledge reserve and reasoning ability of the large language model module, while the vibration monitoring technology, data analysis method and equipment maintenance and management mainly rely on the collaboration of the large language model and the signal conversion text module. The evaluation annotation of the comparative experiment conducted in this study is the accuracy of each assessment objective of the third-level vibration analyst test questions.

[0204] 4.2 Experimental Procedure

[0205] Under the evaluation index of 4.1, the experiment better demonstrates the advantages of the model in this study through comparative analysis. The experiment is divided into large language model comparison and multimodal data analysis task result comparison. The former mainly compares the vibration theory basis and fault diagnosis reasoning ability of the model; the latter compares the vibration monitoring technology, data analysis method, and equipment maintenance and management capabilities of the model. This study selects the current mainstream multimodal large language model and large language model including GPT-4o, GPT-4, GPT3.5, ChatGLM4, ChatGLM3 and Qwen2.5.

[0206] 4.3 Experimental Results

[0207] In terms of the mechanical fault diagnosis capability test of the large language model module, this study showed excellent performance on the highly specialized and technology-intensive test set of the Level 3 Vibration Analyst Examination. In particular, the accuracy of the model in this study exceeded the passing line of 70% in the Level 3 Vibration Analyst Examination, highlighting the unique advantages of this study in dealing with complex and professional problems.

[0208] In terms of multimodal data analysis, the multimodal large language model of this study, with its powerful real-time data processing capabilities, has achieved instant capture and analysis of multi-dimensional, multimodal data streams, ensuring rapid identification of abnormal data points and abnormal behavior patterns. Once signs of a fault are detected, the model can further use its comprehensive analysis capabilities to analyze the abnormal data and estimate the fault location. This process greatly speeds up the speed of fault location, effectively shortens the troubleshooting cycle, and improves operation and maintenance efficiency.

[0209] 4.3.1 Comparison of Large Language Models

[0210] This comparative test was conducted by testing the accuracy of the International Vibration Analyst Level 3 test questions on various major language models. The vibration theory foundation part of the vibration analyst level 3 test questions and the accuracy of various major language models were compared. Figure 8 :

[0211] Figure 8 Comparison of the accuracy of vibration theory foundation

[0212] The experimental results show that although the GPT3.5, ChatGLM4, ChatGLM3 and Qwen2.5 models perform well in general fields, their accuracy rates fail to reach the level of the model in this study when faced with the Level 3 Vibration Analyst Examination, which requires deep professional knowledge and precise analytical capabilities. Although the GPT-4 and GPT-4o models have also demonstrated problem-solving capabilities to varying degrees, they have also failed to cross the 70% accuracy threshold, showing their limitations in application in specific professional fields.

[0213] The results of the comparative experiment of the large language model on the third-level test questions prove the advantages of the model in this study in the following aspects: First, more professional knowledge bases and training data in the field of vibration analysis are incorporated into the model construction process, so as to achieve accurate understanding and answers to professional terms, theories and practical problems; secondly, as mentioned in 3.2, the llama3-8B model used in this study has a more advanced algorithm architecture and optimization strategy, which makes the model more efficient and accurate in processing complex logic and reasoning tasks; finally, the high accuracy of the model in this study in answering the vibration theory foundation and fault diagnosis reasoning test questions of the third-level vibration analyst strongly proves the feasibility and effectiveness of the model in practical applications.

[0214] 4.3.2 Comparison of multi-modal large language model fault signal data analysis

[0215] In the MLLM fault signal data analysis test, this study built a test bench to simulate the real fault signals in the assessment based on the three-level test questions on vibration monitoring technology, data analysis methods, and equipment maintenance and management, providing data for the test of the large language model.

[0216] The signal data files will be collected regularly and uploaded to the multimodal large language model, which will then generate the diagnosis results. In the comparative experiment, the specific performance of each multimodal large language model is shown in the following figure:

[0217] Fig. 9 Specific performance comparison

[0218] Through systematic comparative experimental analysis, the large language model constructed in this study has shown significant advantages in the field of signal processing, which is specifically reflected in the accurate characterization of signal data features, the high-quality generation of waveform images, and the significant improvement of the final fault diagnosis accuracy. For the fault diagnosis task of vibration signal data, the currently used multimodal large language models are generally limited to providing short text analysis based on data, drawing inaccurate waveform images under prompt word emphasis, and predicting possible fault types. However, these models fail to go deep into the data and lack the ability to conduct rigorous data analysis and verification from multiple dimensions and levels. In the model test experiment, the performance in the three aspects of vibration monitoring technology, data analysis methods, and equipment maintenance and management is the accuracy of the signal features converted into descriptive text expressions, the accuracy of the signal visualization image, and the accuracy of the equipment fault diagnosis results. Fig. 9 This is an analysis and comparison of the accuracy of mainstream MLLM in three aspects: the accuracy of signal features converted into descriptive text expressions, the accuracy of signal visualization images, and the accuracy of equipment fault diagnosis results:

[0219] Fig. 9 Signal data analysis comparison of mainstream large language models

[0220] In contrast, the model proposed and verified in this study has achieved significant progress in the processing flow. This model can not only fully and accurately describe the intrinsic characteristics of signal data, but also effectively transform these data into intuitive waveform images, and then use these images and feature data to carry out detailed deduction and accurate diagnosis of fault information.

[0221] This study compares the accuracy of each model in vibration analyst test questions in five aspects: vibration theory basis, fault diagnosis reasoning ability, vibration monitoring technology, data analysis method, equipment maintenance and management. Fig.10 As shown:

[0222] Fig.10 Comparison chart of the five major capabilities of the model

[0223] According to the comparison chart, we can see that the MLLM in this study is superior to other models in all aspects except equipment knowledge. This is mainly because this study has specially expanded the signal conversion text module according to actual tasks to improve the model's data analysis and active fault monitoring capabilities. The large language model module after fine-tuning and specialization has stronger professional knowledge and more rigorous reasoning logic compared to other models, which also improves the accuracy of model fault diagnosis.

[0224] The multimodal large language model of this study has the professional capabilities equivalent to those of a third-level vibration analyst, indicating its broad application prospects in the fields of auxiliary fault diagnosis and intelligent operation and maintenance. By deeply integrating actual cases in the field of fault diagnosis with the professional knowledge of vibration analysts, and carrying out refined field adaptation and functional expansion of the basic large language model, this study successfully achieved the professional transformation of the model in the field of mechanical fault diagnosis. This innovation not only enhances the model's ability to deeply analyze complex fault modes, but also promotes the construction of real-time automated diagnosis processes, providing strong technical support for improving the maintenance efficiency and reliability of industrial systems.

[0225] The above only describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above specific implementation methods. Although the present invention has been disclosed as above in the preferred embodiments, it is not used to limit the present invention. Any technician familiar with the profession can make some changes or modify the technical contents disclosed above into equivalent embodiments with equivalent changes without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A fault diagnosis method for a wind turbine based on a multimodal large language model, characterized in that: The following steps are involved: Step 1: Build a large language model for wind turbine fault diagnosis; obtain a trained large language model for wind turbine fault diagnosis; Step 2: Collect the data of the wind turbine to be diagnosed, input the trained wind turbine fault diagnosis large language model, and obtain the fault diagnosis results and fault solutions.

2. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 1, characterized in that: In the step 1, a large language model for wind turbine fault diagnosis is constructed; and a trained large language model for wind turbine fault diagnosis is obtained. The specific process is as follows: S1: Get the training set; S2: Construct a large language model for wind turbine fault diagnosis; train the large language model for wind turbine fault diagnosis according to the training set to obtain a trained large language model for wind turbine fault diagnosis.

3. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 2, characterized in that: The specific process of obtaining the training set in S1 is: S1.1: Obtain vibration analysis text data, and construct a fault text dataset based on the vibration analysis text data; S1.2: Build a vibration analysis test bench and obtain a fault simulation signal data set based on the vibration analysis test bench; S1.3: Combine the fault text dataset and the fault simulation signal dataset into a training set.

4. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 3, characterized in that: The vibration analysis text data in S1.1 includes: wind turbine equipment data, training materials for vibration analysts, international standardization documents and training materials; The specific process of constructing a fault text data set based on vibration analysis text data is as follows: Convert vibration analysis text data into fault text dataset in the form of question-answer pairs; The specific process of constructing a vibration analysis test bench in S1.2 is as follows: constructing a vibration analysis test bench of a corresponding type according to the specific components of the wind turbine to be diagnosed; The fault simulation signal data set includes I fault simulation signal data, where I is a positive integer.

5. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 4, characterized in that: The wind turbine fault diagnosis large language model in S2 includes: a signal conversion text module and a large language model module; The specific process of training the wind turbine fault diagnosis large language model according to the training set to obtain the trained wind turbine fault diagnosis large language model is as follows: S2.1: Input the fault simulation signal data set in the training set into the signal conversion text module to obtain the feature description text set; S2.2: Input the feature description text set and the fault text data set of the training set into the large language model module to obtain the fault diagnosis results and fault solutions; S2.3: Train the large language model module to obtain a trained large language model for wind turbine fault diagnosis.

6. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 5, characterized in that: In S2.1, the signal conversion text module is input into the fault simulation signal data set in the training set to obtain the feature description text set. The specific process is: S2.1.1: Perform fundamental frequency extraction processing on all fault simulation signals in the fault simulation signal data set to obtain fundamental frequencies A of all fault simulation signals; S2.1.2: According to the fundamental frequencies of all fault simulation signals obtained in S2.1.1, extract the fundamental frequency amplitude B and fundamental frequency phase C corresponding to the fundamental frequencies of all fault simulation signals; S2.1.3: Extract the harmonic frequency D of all fault simulation signals according to the fundamental frequency of all fault simulation signals obtained in S2.1.1; S2.1.4: Extract the harmonic amplitude E of the harmonic frequency of all fault simulation signals according to the harmonic frequency of all fault simulation signals; S2.1.5: According to S2.1.1 to S2.1.4, the fundamental frequency A, fundamental frequency amplitude B, fundamental frequency phase C, multiple harmonic frequency D, and multiple harmonic amplitude E of all fault simulation signals are obtained; construct a feature description text set.

7. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 6, characterized in that: In S2.1.1, the ith fault simulation signal f i The specific process of performing base frequency extraction processing to obtain the base frequency Ai of the i-th fault simulation signal is as follows: S2.1.1.1: Perform fast Fourier transform on the ith fault simulation signal to obtain an amplitude spectrum of the ith fault simulation signal; S2.1.1.2: Extract the frequency corresponding to the maximum value in the amplitude spectrum of the i-th fault simulation signal as the fundamental frequency A i ; i = 1, 2, ..., I; expressed as: A i =argmax f |Y(f i )| (1) In the formula, Y(f i ) represents the simulated signal f for the i-th fault i The result after fast Fourier transform, |Y(f i )|is Y(f i )’s amplitude spectrum, argmax f Indicates the frequency corresponding to the maximum value; The baseband A in S2.1.2 i The fundamental frequency amplitude is denoted as B i ∈B; fundamental frequency A i The fundamental frequency phase is expressed as C i ∈C; expressed by the formula: C i =∠Y(A i (3) Y() indicates fast Fourier transform, N indicates the number of sampling points of the fault simulation signal when fast Fourier transform is performed; ∠ is the sign of the phase angle, indicating the phase of the complex number; The n-fold harmonic frequency of the ith fault simulation signal in S2.1.3 is represented by D ni , n represents the harmonic multiplication factor, n = 2, 3, 4, ...; The formula is: D ni =n×A i (4) In S2.1.4, the n-fold harmonic amplitude of the ith fault simulation signal is represented by E ni , The formula is:

8. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 7, characterized in that: The large language model module in S2.2 is the Llama3-8B large language model.

9. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 8, characterized in that: The algorithm used to train the large language model module in S2.3 is the LoRA algorithm; the learning rate scheduling method in the training process is the Cosine learning rate scheduling method.

10. A fault diagnosis method for a wind turbine generator based on a multimodal large language model according to claim 9, characterized in that: The wind turbine generator data to be diagnosed collected in step 2 is vibration signal data of the bearing of the wind turbine generator to be diagnosed.

Images