Large language model rotating machinery fault diagnosis method, system, medium and equipment

Abstract

A rotating machinery fault diagnosis method, system, medium and equipment based on a large language model, in the method, the vibration signal of the rotating machinery is collected and self-adaptive Z-Score normalization preprocessing is performed; a dual-mode input is constructed, multi-dimensional physical features in the time domain, frequency domain and envelope domain are extracted from the preprocessed vibration signal, a structured physical anchor prompt text is generated based on a preset natural language template, the vibration signal is divided into multiple fixed-length fine-grained signal patches through a sliding window, and the fine-grained signal patches are mapped to the embedding space of the large language model through a trainable linear projection layer to obtain a signal embedding sequence; the text embedding sequence obtained after the physical anchor prompt text is processed by the tokenizer and embedding layer of the large language model is spliced with the signal embedding sequence along the time dimension to form a heterogeneous input sequence; the heterogeneous input sequence is input into the large language model which is fine-tuned through quantization and low-rank adaptation, and a structured fault diagnosis result is output.

Description

Technical Field

[0001] This invention relates to the field of rotating machinery technology, and in particular to a method, system, medium, and equipment for diagnosing rotating machinery faults based on a large language model. Background Technology

[0002] Rotating machinery, as core equipment in modern industry, is widely used in key fields such as power, chemical, and manufacturing. Its operating status directly affects the continuity and safety of industrial production. Sudden failures such as bearing failure, rotor bending, bar breakage, misalignment, and imbalance can lead to huge economic losses and serious safety hazards. Therefore, high-precision and high-reliability fault diagnosis systems are a core requirement for industrial health monitoring.

[0003] In real-world industrial scenarios, rotating machinery often faces non-stationary operating conditions, leading to data distribution shifts and increased differences across operating conditions. Furthermore, early fault signals are weak and strongly coupled with operating condition information, further complicating fault diagnosis. In addition, due to increased equipment reliability, real fault samples are scarce, making the small sample size problem a key bottleneck restricting the generalization ability of diagnostic models. While traditional signal processing-based methods and early data-driven methods are effective in specific scenarios, they struggle to simultaneously address the three core challenges of cross-condition adaptability, small sample size diagnosis, and model interpretability.

[0004] In recent years, large language models (LLMs) have made breakthroughs in natural language processing and time series analysis due to their powerful pattern recognition and reasoning capabilities, providing a new technical path for fault diagnosis of rotating machinery. However, there is a cross-modal gap between the text corpus pre-trained by LLMs and the time series sensing data of rotating machinery, and direct application faces problems such as semantic sparsity and insufficient adaptation of temporal features. Existing research attempts to solve this problem through two paradigms, but both have significant shortcomings and are difficult to meet the stringent requirements of industrial-grade fault diagnosis. Existing technologies struggle to balance information integrity and interpretability: the input paradigm of existing large language models in rotating machinery fault diagnosis presents a binary contradiction, making it difficult to balance information integrity and interpretability. While feature-language methods using text-to-feature conversion offer interpretability, they suffer from severe information loss in weak fault scenarios; data-language methods that directly use raw signals retain high fidelity but lack semantic foundation, easily becoming "black box" models. The problem of lost weak fault information: Traditional feature engineering has limitations in early fault diagnosis. Initial fault features are often averaged during statistical index calculation, making it difficult for the model to capture weak fault signals and affecting the accuracy of weak fault identification. Insufficient generalization ability across operating conditions: Existing methods are limited by the low confidence of pseudo-labels generated from unlabeled target domains, causing the model to heavily rely on transferable knowledge under a single operating condition and fail to generalize to unknown operating conditions, resulting in poor adaptability to unknown operating conditions. Prominent diagnostic challenges in small sample and zero sample scenarios: In practical industrial applications, system reliability is high, and samples of rotating machinery faults are scarce. Existing methods rely excessively on data volume, and diagnostic performance drops sharply in zero-sample cold start or small sample scenarios. Insufficient model interpretability: Model interpretability is disconnected from physical reality. The "black box" nature of existing intelligent diagnostic models means that their decision-making process lacks physical semantic support, and post-exposure interpretation techniques can only achieve feature ranking, failing to reflect the physical mechanisms behind the faults, affecting the credibility of industrial deployment. Existing methods still struggle to systematically integrate physical prior knowledge with high-fidelity signal details, and fail to fully utilize the semantic reasoning and contextual learning capabilities inherent in large language models to achieve robust, interpretable, and highly generalized diagnosis under varying operating conditions, weak faults, and small sample scenarios.

[0005] The information disclosed in the background section is only for enhancing the understanding of the background of this invention, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To address the shortcomings, this invention provides a method, system, medium, and device for rotating machinery fault diagnosis based on a large language model. Through the synergy of physical anchoring prompts and high-fidelity signal segments, it achieves more robust, reliable, and easily generalizable intelligent diagnosis of rotating machinery. Leveraging the powerful contextual learning capabilities of the large language model, it can quickly adapt to different mechanical systems and operating environments under zero-sample or few-sample conditions, significantly reducing reliance on labeled data. This provides a unified, reliable, and interpretable solution for intelligent health management, predictive maintenance, and safety decision-making for critical rotating equipment such as wind turbine gearboxes, aero-engines, and large pump sets.

[0007] A method for diagnosing rotating machinery faults based on a large language model includes:

[0008] Vibration signals from rotating machinery are collected and preprocessed using adaptive Z-Score normalization.

[0009] A dual-modal input is constructed, and physical features in three dimensions—time domain, frequency domain, and envelope domain—are extracted from the preprocessed vibration signal. The physical features generate structured physical anchoring prompt text based on a preset natural language template. The vibration signal is divided into multiple fixed-length fine-grained signal patches through a sliding window, and the fine-grained signal patches are mapped to the embedding space of a large language model through a trainable linear projection layer to obtain a signal embedding sequence.

[0010] The text embedding sequence obtained by processing the physical anchoring prompt text through the word segmenter and embedding layer of the large language model is concatenated with the signal embedding sequence along the time dimension to form a heterogeneous input sequence.

[0011] The heterogeneous input sequence is input into a large language model that has been fine-tuned by quantization and low-rank adaptation. A multi-head self-attention mechanism is used to guide the attention of physical semantics to signal details, complete cross-modal collaborative reasoning, and output structured fault diagnosis results, including fault type, fault location and confidence level.

[0012] In the aforementioned method for diagnosing rotating machinery faults based on a large language model, the multi-dimensional physical features include:

[0013] Used to quantify signal energy, statistical distribution and impact characteristics, and to identify the time-domain characteristics of impact-type anomalies, including mean, standard deviation, root mean square, peak value, kurtosis, skewness, peak factor and shape factor;

[0014] Frequency domain characteristics used to diagnose imbalance and misalignment faults include fundamental frequency amplitude, 2nd-5th harmonic amplitude, total harmonic distortion, spectral centroid, and root mean square frequency.

[0015] Envelope domain features used for demodulating high-frequency signals and detecting periodic impact fault characteristics of rolling bearings and gears include envelope mean, envelope standard deviation, envelope root mean square, and envelope kurtosis.

[0016] In the aforementioned method for fault diagnosis of rotating machinery based on a large language model, segmentation into fine-grained signal patches includes fine-grained segmentation and embedding mapping of the vibration signal. Given a vibration signal sequence X of length L, a sliding window strategy is used to segment it into N signal patches of fixed length. The number of segmented signal patches N can be calculated using a preset window size and step size. A trainable linear projection layer W is then used. p The original numerical signal patch is mapped to the high-dimensional embedding space of the LLM to generate a signal embedding sequence, and a linear projection layer W is applied. p As a semantic aligner, it enables cross-modal alignment between the continuous time series feature space and the discrete text embedding space, allowing LLM to process two modalities of data in parallel.

[0017] In the rotating machinery fault diagnosis method based on a large language model, the length of the fine-grained signal patch is determined according to the rotational speed and sampling frequency of the rotating machinery.

[0018] In the rotating machinery fault diagnosis method based on a large language model, the large language model is LLaMA-3-8B-Instruct, and it is loaded with 4-bit quantization. The parameter fine-tuning is achieved by updating the parameters only through a low-rank adaptation layer.

[0019] In the aforementioned method for diagnosing rotating machinery faults based on a large language model, the large language model processes heterogeneous input sequences through a multi-head self-attention mechanism. The text embedding sequence serves as the query condition, guiding the attention head of the large model to focus on the segment in the signal embedding sequence that matches the physical feature description for contextual reasoning.

[0020] In the rotating machinery fault diagnosis method based on a large language model, structured physical anchoring prompt text is generated by an embedded rotating machinery fault diagnosis domain knowledge base. The rotating machinery fault diagnosis domain knowledge base supports feature-semantic mapping rules for bearing, gear, rotor imbalance, misalignment, and broken bar fault modes.

[0021] A system for performing the method includes:

[0022] The signal acquisition and preprocessing module is used to acquire and normalize vibration signals;

[0023] The bimodal construction module includes a physical anchoring cue generation unit for generating structured physical anchoring cue text and a fine-grained signal patch embedding unit for generating fine-grained signal patches;

[0024] The heterogeneous interactive reasoning module integrates a large language model fine-tuned by QLoRA, which is used to receive concatenated heterogeneous input and perform cross-modal reasoning.

[0025] The diagnostic output module generates structured results that include fault type, location, and confidence level.

[0026] A computer storage medium including computer instructions that, when run on a computer, cause the computer to perform the method.

[0027] An electronic device, the electronic device comprising:

[0028] Memory, processor, and computer programs stored in memory and executable on the processor, wherein,

[0029] The processor implements the method when executing the program.

[0030] Compared with existing technologies, this invention has the following advantages: This invention constructs a heterogeneous interaction paradigm that balances information integrity and interpretability, resolving the binary contradiction between feature-language methods and data-language methods, and achieving the synergistic fusion of physical semantic guidance and high-fidelity signal details. Through this heterogeneous interaction paradigm, physical anchoring prompts provide semantic constraints and interpretability, while signal patches preserve high-fidelity details, resolving the binary contradiction of existing methods that either "lose information or become a black box." Experimental verification shows that on the KAIST, HUST, and SQV datasets, the diagnostic accuracy of this invention reaches 99.89%, 98.49%, and 97.01%, respectively, significantly outperforming the feature-language method's 79.59%, 91.38%, and 92.91% and the data-language method's 0%.

[0031] This invention enhances weak fault identification capabilities by mutually reinforcing fine-grained signal patches and physical anchoring cues to capture subtle fault signatures masked by statistical features, thereby improving diagnostic performance in weak fault scenarios. Fine-grained signal patches capture the waveform details of weak faults, and the semantic guidance provided by physical anchoring cues effectively avoids fault signal loss caused by statistical feature averaging. Experimental results demonstrate that in weak fault scenarios on the KAIST dataset, this invention achieves an accuracy of 99.89%, while the feature-language method only achieves 79.59%, fully validating its advantages in weak fault identification.

[0032] Enhanced robustness across operating conditions enables the model to withstand statistical feature drift caused by changes in operating conditions such as varying load and speed, maintaining high diagnostic accuracy in zero-sample cross-operating condition scenarios. Physical anchoring cues guide the model to learn shape-invariant fault signatures, mitigating the impact of statistical feature drift. Experimental results demonstrate that in zero-sample cross-operating condition experiments on the KAIST dataset with varying load, the diagnostic accuracy of this invention for target operating conditions of 0Nm, 2Nm, and 4Nm reaches 84.52%, 97.16%, and 99.66%, respectively, while the feature-language method only achieves 31.44%, 56.19%, and 34.82%, exhibiting superior cross-operating condition adaptability.

[0033] To improve adaptability to small and zero-sample scenarios, this invention leverages the contextual reasoning capabilities and heterogeneous interaction mechanisms of LLMs to achieve accurate diagnosis even when fault samples are scarce. In low-sample scenarios, near-perfect performance is achieved with only 10% of the target domain data. Experimental results demonstrate that in a low-sample experiment on the KAIST dataset with a 0Nm condition, the accuracy of this invention increased from 84.52% with zero samples to 99.80% after adding 10% of the samples, significantly reducing the reliance on labeled data in industrial scenarios.

[0034] Enhancing cross-domain generalization capabilities enables the model to adapt to rotating machinery systems of different types and structures, achieving cross-domain reuse of diagnostic knowledge and reducing deployment costs in new scenarios. The model learns general fault physical semantics through physical anchoring cues, enabling it to adapt to rotating machinery systems of different types and structures. Experimental results demonstrate that in cross-domain experiments on three heterogeneous datasets—KAIST, HUST, and SQV—the accuracy reaches a maximum of 92.28% in zero-shot scenarios and a maximum of 96.71% in few-shot scenarios, achieving cross-domain reuse of diagnostic knowledge.

[0035] To achieve physical semantic interpretability, physical anchoring prompts are used to anchor the model's reasoning process to the physical mechanisms of the industrial domain, ensuring the physical authenticity of diagnostic results and the interpretability of the decision-making process, thereby improving the credibility of industrial deployments. Physical anchoring prompts anchor the model's reasoning process to industrial physical mechanisms, and the diagnostic results not only provide the fault type but also associate physical features, solving the interpretability problem of existing "black box" models and improving the credibility and security of industrial deployments. QLoRA technology is used to achieve efficient parameter fine-tuning, and the basic model is loaded with 4-bit precision, allowing it to run on consumer-grade hardware without the need for expensive dedicated computing equipment; it also supports zero-shot and few-shot deployments, reducing the annotation cost for new scenarios and adapting to actual industrial needs. Attached Figure Description

[0036] Various other advantages and benefits of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0037] In the attached diagram:

[0038] Figure 1 This is a schematic diagram of the Hi-LLM framework for heterogeneous large language models of the present invention;

[0039] Figure 2 This is a schematic diagram of the process of the present invention.

[0040] The present invention will be further explained below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0041] The following will refer to the appendix. Figures 1 to 2 Specific embodiments of the invention will be described in more detail below. While specific embodiments of the invention are shown in the accompanying drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0042] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that different terms may be used to refer to the same component. This specification and claims do not distinguish components based on differences in terminology, but rather on differences in function. The terms "comprising" or "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising but not limited to." The following descriptions are preferred embodiments for carrying out the invention; however, these descriptions are for the purpose of understanding the general principles of the specification and are not intended to limit the scope of the invention. The scope of protection of this invention is determined by the appended claims.

[0043] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. The accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0044] In one embodiment, the method includes,

[0045] Signal Acquisition and Preprocessing: Vibration signals generated during the operation of rotating machinery are first acquired, and then preprocessed using an adaptive Z-Score normalization method. This preprocessing operation can eliminate signal amplitude interference caused by factors such as changes in equipment operating conditions and differences in sensor sensitivity, making the signal distribution more uniform and laying the foundation for subsequent analysis.

[0046] Bimodal Input Construction: The following two sub-steps are executed in parallel to construct the input, which includes feature extraction and cue generation, and signal patch segmentation and embedding:

[0047] Feature extraction and prompt generation: Physical features in three dimensions—time domain, frequency domain, and envelope domain—are extracted from the preprocessed vibration signal. Based on a preset natural language template, these extracted physical features are then transformed into structured physical anchoring prompt text, which provides physical semantic reference for subsequent diagnosis.

[0048] Signal Patch Segmentation and Embedding: A sliding window strategy is adopted to segment the preprocessed vibration signal into multiple fine-grained signal patches of fixed length, thereby preserving the high-fidelity waveform details of the signal; then, through a trainable linear projection layer, these fine-grained signal patches are mapped to the embedding space of the large language model, and finally the signal embedding sequence is obtained.

[0049] Heterogeneous input sequence construction: The physical anchoring prompt text is input into the large language model. After processing by the model's tokenizer and word embedding and transformation by the embedding layer, a text embedding sequence is obtained. Then, the text embedding sequence is concatenated with the previously obtained signal embedding sequence along the time dimension to form a heterogeneous input sequence, thereby realizing the fusion of physical semantic information and signal detail information.

[0050] Model reasoning and diagnosis: The heterogeneous input sequence is input into a large language model fine-tuned by quantization low-rank adaptation technique; the multi-head self-attention mechanism of the model is used to guide the model to focus on key segments related to the signal embedding sequence by the physical semantic information in the text embedding sequence, so as to realize cross-modal collaborative reasoning; finally, the large language model outputs a structured fault diagnosis result, which includes at least the fault type, fault location and its diagnostic confidence.

[0051] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model,

[0052] The segmentation and embedding mapping process for fine-grained signal patches is as follows:

[0053] Given a vibration signal sequence X of length L, a sliding window strategy is first used to uniformly divide the sequence into N fixed-length signal patches. The number of signal patches N after segmentation can be calculated using a preset window size and step size. Then, a trainable linear projection layer Wp maps each original numerical signal patch to the high-dimensional embedding space of the large language model, generating the corresponding signal embedding sequence. The linear projection layer Wp acts as a semantic aligner, enabling cross-modal alignment between the continuous time-series feature space and the discrete text embedding space, ensuring that the large language model can process both text and signal data in parallel.

[0054] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model,

[0055] The length of the fine-grained signal patch needs to be determined comprehensively based on the actual operating speed of the rotating machinery and the sampling frequency during signal acquisition. For example, for high-speed equipment, the patch length can be appropriately shortened to capture rapidly changing signal details; for low-speed equipment, the patch length can be appropriately increased to ensure that the complete fault characteristic cycle is included.

[0056] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model,

[0057] The process by which large language models handle heterogeneous input sequences is as follows:

[0058] The model employs a multi-head self-attention mechanism to perform feature fusion and inference on the concatenated heterogeneous input sequences. The text embedding sequence serves as a query condition, guiding the model's attention head to actively focus on key segments in the signal embedding sequence that match the physical feature description. For example, when the text prompt mentions "high kurtosis," the attention will focus on segments in the signal exhibiting impact characteristics. Contextual inference is then performed based on these key segments, ultimately achieving a precise correspondence between physical semantics and signal details, thus improving the accuracy of diagnostic results.

[0059] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model, the generation of the structured physical anchoring prompt text relies on a built-in knowledge base for rotating machinery fault diagnosis. This knowledge base stores feature-semantic mapping rules corresponding to common fault modes such as bearings, gears, rotor imbalance, misalignment, and broken bars. By calling the rules in this knowledge base, the extracted physical features can be transformed into logically clear and semantically explicit natural language prompt text.

[0060] A system for performing the method includes:

[0061] Signal acquisition and preprocessing module: Its main function is to acquire vibration signals of rotating machinery through sensors and other devices, perform adaptive Z-Score normalization preprocessing on the acquired raw signals, and output the normalized signals.

[0062] The bimodal construction module comprises two sub-units: a physical anchoring cue generation unit and a fine-grained signal patch embedding unit. The physical anchoring cue generation unit is responsible for extracting multi-dimensional physical features and generating structured cue text; the fine-grained signal patch embedding unit is responsible for segmenting signal patches and mapping them to generate signal embedding sequences.

[0063] Heterogeneous Interactive Reasoning Module: Integrates a large language model fine-tuned with QLoRA technology, capable of receiving concatenated heterogeneous input sequences, completing cross-modal collaborative reasoning through a multi-head self-attention mechanism, and outputting preliminary diagnostic results.

[0064] Diagnostic output module: Organizes the preliminary results obtained from model inference and generates a structured diagnostic report containing fault type, fault location and confidence level, presenting it to the user in an intuitive and clear form.

[0065] like Figure 2 As shown, the rotating machinery fault diagnosis method based on a large language model includes the following steps:

[0066] Vibration signals of rotating machinery are collected, and the collected signals are preprocessed using adaptive Z-Score normalization.

[0067] An example of the specific process of adaptive Z-Score normalization preprocessing: Specific process: Signal segment division: The acquired continuous vibration signal is segmented according to the "fine-grained signal patch length," with each segment being a signal unit to be processed; Calculation of local statistics for each segment: The mean μ and standard deviation σ are calculated for each signal segment X; Adaptive normalization calculation: The Z-Score formula with a numerical stability term is used to calculate the mean μ and standard deviation σ for each data point x within the segment. i Normalize: Output normalized signal: The normalized results of each segment are concatenated in their original order to obtain a complete preprocessed signal, which is used for subsequent physical feature extraction and fine-grained patch segmentation. In the data preprocessing stage, adaptive Z-Score normalization is applied to the acquired raw vibration signals. This method calculates the mean and standard deviation independently for each data segment, converting the raw signal into a standard distribution with zero mean and unit variance, effectively eliminating amplitude interference introduced by factors such as changes in operating conditions and sensor differences. After this processing, vibration signals from different sources and under different conditions are unified to a comparable scale, allowing subsequent models to focus more on learning the essential shape and frequency domain pattern features of the signal waveform related to the fault, laying the foundation for improving the robustness and cross-domain generalization ability of the diagnostic model. This is a commonly used and generalized method for processing normalized data.

[0068] A bimodal input is constructed, extracting multi-dimensional physical features in the time, frequency, and envelope domains from the preprocessed vibration signal. Structured physical anchoring prompts are generated based on a preset natural language template. The vibration signal is segmented into multiple fixed-length fine-grained signal patches using a sliding window, and these patches are mapped to the embedding space of a large language model using a trainable linear projection layer, resulting in a signal embedding sequence. When generating the structured physical anchoring prompts based on the preset natural language template, multi-dimensional physical features in the time, frequency, and envelope domains are first extracted from the preprocessed vibration signal. These numerical features are then substituted into a fixed template containing "instruction + input features + output format" to generate structured natural language prompts understandable by the large language model—i.e., feature textification. Segmenting the fine-grained signal patches requires determining the patch length based on the rotational speed and sampling frequency of the rotating machinery. A sliding window strategy is used to segment the vibration signal into multiple fixed-length segments. These numerical signal patches are then mapped to the high-dimensional embedding space of the large language model using a trainable linear projection layer, ultimately obtaining the signal embedding sequence. This achieves cross-modal alignment between continuous time series and discrete text embedding spaces. The specific process is the construction process of bimodal input.

[0069] The text embedding sequence obtained by processing the physical anchoring prompt text through the word segmenter and embedding layer of the large language model is concatenated with the signal embedding sequence along the time dimension to form a heterogeneous input sequence;

[0070] The heterogeneous input sequence is input into a large language model that has been fine-tuned by quantization and low-rank adaptation. A multi-head self-attention mechanism is used to guide the attention of physical semantics to signal details, complete cross-modal collaborative reasoning, and output structured fault diagnosis results, including fault type, fault location and confidence level.

[0071] The specific process of quantizing low-rank adaptation fine-tuning of large language models:

[0072] 1. Quantization Model Loading: Base Model Selection: LLaMA-3-8B-Instruct is used as the pre-trained large language model, which has strong contextual reasoning capabilities and is suitable for cross-modal tasks; 4-bit quantization processing: This can compress the model weights from multi-bit floating-point numbers to 4-bit integers while preserving the core reasoning capabilities of the model. After quantization, the model only occupies limited GPU memory, solving the hardware limitation problem of large model deployment.

[0073] 2. Insertion of low-rank adaptation layer: Insertion location: In the Transformer encoder layer of the large model, insert a low-rank adaptation layer for the key weight matrix of the multi-head self-attention mechanism; Parameter freezing and training: Freeze the pre-trained weights of the large model and train only the low-rank matrix parameters of the LoRA layer.

[0074] 3. Training: The dataset input is a heterogeneous input of "physical anchoring prompt text + signal embedding sequence", and the labels are structured results of fault type, location and confidence. The cross-entropy loss function is used to enable the model to learn the mapping relationship of "physical semantics - signal features - fault label".

[0075] The specific steps for cross-modal collaborative reasoning are as follows: Utilizing a multi-head self-attention mechanism to guide the attention of physical semantics to signal details.

[0076] The model first transforms the physical semantics of the text description (such as "impact features") into a set of query vectors as guiding signals. Then, these query vectors are similar to key vectors representing signal segments to generate an attention weight distribution, thereby dynamically filtering and focusing on feature segments in the signal that are highly related to the physical semantics (such as periodic impact waveforms). Finally, the weights are used to perform weighted fusion of signal value vectors containing rich information to construct a joint feature representation with semantic injection. The association is further strengthened and decoded through a feedforward network, ultimately achieving accurate mapping and structured output from physical description to specific fault evidence.

[0077] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model, the multi-dimensional physical features include:

[0078] Used to quantify signal energy, statistical distribution and impact characteristics, and to identify the time-domain characteristics of impact-type anomalies, including mean, standard deviation, root mean square, peak value, kurtosis, skewness, peak factor and shape factor;

[0079] Frequency domain characteristics used to diagnose imbalance and misalignment faults include fundamental frequency amplitude, 2nd-5th harmonic amplitude, total harmonic distortion, spectral centroid, and root mean square frequency.

[0080] Envelope domain features used for demodulating high-frequency signals and detecting periodic impact fault characteristics of rolling bearings and gears include envelope mean, envelope standard deviation, envelope root mean square, and envelope kurtosis.

[0081] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model, segmentation into fine-grained signal patches includes fine-grained segmentation and embedding mapping of the vibration signal. Given a vibration signal sequence X of length L, a sliding window strategy is used to segment it into N signal patches of fixed length, and then a trainable linear projection layer W is used. p The original numerical signal patch is mapped to the high-dimensional embedding space of the LLM to generate a signal embedding sequence, and the projection layer W is used. p As a semantic aligner, this layer achieves cross-modal alignment between the continuous time-series feature space and the discrete text embedding space, enabling LLM to process two modalities of data in parallel. Since the LLM embedding space is pre-trained for processing discrete text tokens and cannot directly understand continuous vibration signal numerical sequences, this linear projection layer is needed as a "translator" or "semantic aligner." It linearly maps fixed-length, fine-grained patches of the original signal (continuous numerical vectors) obtained through sliding window segmentation to a high-dimensional semantic space with the same dimension as the LLM text embeddings. This process transforms the signal from a "time-amplitude" physical feature space to a "semantic concept" space that LLM can understand. This allows the signal embedding sequence and the text embedding sequence of the physically anchored cue text to be directly concatenated along the time dimension, forming a unified heterogeneous input sequence for subsequent LLM (fine-tuned by QLoRA) to perform cross-modal collaborative inference through its attention mechanism.

[0082] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model, the length of the fine-grained signal patch is determined according to the rotational speed and sampling frequency of the rotating machinery.

[0083] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model, the large language model is LLaMA-3-8B-Instruct, and is loaded with 4-bit quantization, with parameter fine-tuning achieved only by updating parameters through a low-rank adaptation layer.

[0084] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model, the large language model processes heterogeneous input sequences through a multi-head self-attention mechanism, wherein the text embedding sequence serves as a query condition, guiding the attention head to focus on the segment in the signal embedding sequence that matches the physical feature description for contextual reasoning.

[0085] In a preferred embodiment of the rotating machinery fault diagnosis method based on a large language model, structured physical anchoring prompt text is generated by an embedded rotating machinery fault diagnosis domain knowledge base. The rotating machinery fault diagnosis domain knowledge base supports feature-semantic mapping rules for bearing, gear, rotor imbalance, misalignment, and broken bar fault modes.

[0086] A system for performing the method includes:

[0087] The signal acquisition and preprocessing module is used to acquire and normalize vibration signals;

[0088] The bimodal construction module includes a physical anchoring cue generation unit for generating structured physical anchoring cue text and a fine-grained signal patch embedding unit for generating fine-grained signal patches;

[0089] The heterogeneous interactive reasoning module integrates a large language model fine-tuned by QLoRA, which is used to receive concatenated heterogeneous input and perform cross-modal reasoning.

[0090] The diagnostic output module generates structured results that include fault type, location, and confidence level.

[0091] A computer storage medium including computer instructions that, when run on a computer, cause the computer to perform the method.

[0092] An electronic device, the electronic device comprising:

[0093] Memory, processor, and computer programs stored in memory and executable on the processor, wherein,

[0094] The processor implements the method when executing the program.

[0095] In one embodiment, by constructing a parallel heterogeneous interaction between physical anchoring cues and high-fidelity signal patches, accurate, robust, and interpretable diagnosis of rotating machinery faults is achieved, applicable to complex industrial scenarios such as variable operating conditions, small sample sizes, and cross-domain applications. The overall architecture of this invention comprises three core stages: dual-modal input construction, heterogeneous interaction and LLM fine-tuning, and diagnostic output. These stages work collaboratively to form a complete closed loop of "physical semantic guidance - signal detail support - model inference diagnosis," with the specific structure and functions as follows:

[0096] The core idea of ​​Hi-LLM is physical information-guided collaborative representation. This framework does not force the model to choose between physical anchoring cues and data-language methods; instead, it designs a dual-stream input system, enabling LLM to handle both modalities simultaneously, similar to the diagnostic process of human experts. The overall framework architecture, as shown in Figure 1, includes three core modules: physical anchoring cue generation, fine-grained signal patching, and heterogeneous interaction and LLM processing. Efficient fine-tuning is achieved through QLoRA (Quantization Low-Rank Adaptation) technology, ultimately outputting fault diagnosis results.

[0097] Bimodal input construction is the basic data preprocessing step of the framework. It constructs physical anchoring cues (text modality) and fine-grained signal patches (signal modality) through two parallel streams, providing high-quality input for subsequent heterogeneous interactions.

[0098] 1) Physical anchoring prompt generation:

[0099] The core function of this module is to extract key physical features from the vibration signals of rotating machinery and synthesize them into structured natural language prompts, providing physical semantic constraints and reasoning basis for LLM.

[0100] a) Physical Feature Extraction: Based on physical knowledge in the field of rotating machinery fault diagnosis, three major categories of representative physical features are extracted from the original vibration signal, covering the time domain, frequency domain, and envelope domain. Time domain features include: mean, standard deviation, root mean square (RMS), peak value, kurtosis, skewness, peak factor, and shape factor, used to quantify signal energy, statistical distribution, and impact characteristics, and to identify impact-related anomalies. Frequency domain features include: fundamental frequency amplitude, 2nd-5th harmonic amplitude, total harmonic distortion, spectral centroid, and RMS frequency, used to diagnose shaft-related faults such as imbalance and misalignment. Envelope domain features include: envelope mean, envelope standard deviation, envelope RMS, and envelope kurtosis, used to demodulate high-frequency signals and detect periodic impact fault characteristics in components such as rolling bearings and gears.

[0101] b) Structured Hint Generation: Extracted numerical physical features are converted into natural language hints using predefined templates. For example: "Instruction: You are a rotating machinery fault diagnosis expert. You need to analyze vibration signals based on the following physical features and provide a diagnostic result. Input: Time domain features: Root mean square value: XXX; Peak value: XXX; Kurtosis: XXX; Frequency domain features: Fundamental frequency amplitude: XXX; 2nd-5th harmonic total harmonic distortion: XXX; Envelope domain features: Envelope kurtosis: XXX; Envelope peak amplitude: XXX. Output: The diagnostic result is XXX fault." This hint provides an interpretable high-level physical context for LLM, anchoring the model inference process to industrial physical mechanisms.

[0102] 2) Fine-grained signal patch (signal mode):

[0103] The core function of this module is to perform fine-grained segmentation and embedding mapping on the original vibration signal, retain high-fidelity waveform details, avoid information loss caused by feature extraction, and reduce the computational complexity of LLM processing long sequence signals.

[0104] a) Signal Patch Segmentation: Given an original vibration signal sequence X of length L, a sliding window strategy is used to segment it into N signal patches of fixed length. This segmentation method can capture local semantic patterns, avoid the loss of details caused by global statistical pooling, and compress the sequence length from L, reducing computational complexity.

[0105] b) Signal embedding mapping: through a trainable linear projection layer W p The original numerical signal patch is mapped to the high-dimensional embedding space of the LLM to generate a signal embedding sequence. This projection layer acts as a semantic aligner, realizing cross-modal alignment between the continuous time series feature space and the discrete text embedding space, enabling the LLM to process two modalities of data in parallel.

[0106] Heterogeneous interaction and LLM processing are the core reasoning components of the framework. Through modal splicing and LLM fine-tuning, heterogeneous interaction between physical anchoring prompts and signal patches is achieved. By leveraging the contextual reasoning capabilities and physical semantic constraints of LLMs, fault diagnosis is completed.

[0107] 1) Construction of heterogeneous inputs:

[0108] The text embedding sequence E obtained after processing the physical anchoring cues by an LLM pre-trained tokenizer and an embedding layer text , with signal embedding sequence E signal By splicing along the time dimension, a heterogeneous input sequence E is formed. input This splicing method achieves a deep fusion of physical semantic information and high-fidelity signal details, providing LLM with a dual-dimensional input of "physical knowledge + signal evidence".

[0109] 2) LLM fine-tuning and inference: The LLaMA-3-8B-Instruct is selected as the base model, and the QLoRA technique is used for efficient parameter fine-tuning: the base model is loaded with 4-bit precision, and a small number of trainable parameters are introduced through a low-rank adaptation layer. While ensuring the model generation and inference capabilities, the consumption of computing resources is reduced, and it is suitable for consumer-grade hardware deployment.

[0110] a) Inference mechanism: LLM uses a multi-head self-attention mechanism to handle heterogeneous input E input Processing is performed, in which text is embedded in E text As a query condition, it guides attention to the signal embedded in E. signalThe model uses fragments that match the physical feature descriptions to achieve collaborative reasoning of "physical semantic guidance - signal detail verification". For example, when the text prompt describes "high kurtosis value", the model will focus on the corresponding pulse waveform fragment in the signal patch to verify whether there are typical impact characteristics of bearing failure.

[0111] b) Output generation: After fine-tuning, the model directly outputs structured diagnostic results, such as specific fault types, fault locations, and fault confidence levels, including inner ring faults, outer ring faults, and imbalance problems.

[0112] Data preprocessing and optimization are performed before constructing the bimodal input model to improve its adaptability to operating conditions and numerical stability.

[0113] Adaptive Z-Score normalization: Normalizes each signal segment to eliminate amplitude differences caused by different operating conditions and sensor sensitivities, allowing the model to focus on waveform patterns rather than absolute energy levels, thus avoiding numerical stability issues where the denominator is zero.

[0114] The fault diagnosis process of this invention follows the action logic of "data preprocessing → dual-modal input construction → heterogeneous interactive reasoning → diagnostic output", and the action relationships of each step are as follows:

[0115] 1) After the vibration signal of rotating machinery in physical space is collected by the sensor, it is first subjected to adaptive Z-Score normalization preprocessing to eliminate the influence of amplitude difference.

[0116] 2) The preprocessed signal is processed in two parallel paths: one path extracts physical features in the time domain, frequency domain, and envelope domain, synthesizes physical anchoring cues, and converts them into text embeddings. text The other path is split into fine-grained signal patches, which are then linearly projected and converted into signal embeddings (E). signal ;

[0117] 3) Text Embedding E text With signal embedding E signal The heterogeneous input Einput is formed by splicing the components, and then input into the LLM that has been fine-tuned by QLoRA.

[0118] 4) LLM achieves heterogeneous interaction between physical anchoring cues and signal patches through a multi-head self-attention mechanism. Guided by physical semantics, it focuses on key fault segments in the signal and performs contextual reasoning.

[0119] 5) LLM outputs structured diagnostic results to complete fault identification;

[0120] 6) Diagnostic results can be fed back to the model optimization stage, and the diagnostic performance can be further improved in specific scenarios such as small sample sizes by fine-tuning with a small amount of labeled data.

[0121] This invention proposes a heterogeneous interaction paradigm that balances information integrity and interpretability. By employing parallel heterogeneous interaction between physically anchored cues and high-fidelity signal patches, it resolves the binary contradiction inherent in existing methods, achieving a synergistic fusion of physical semantic guidance and signal detail support. Based on physical knowledge in the field of rotating machinery fault diagnosis, multi-dimensional physical features in the time, frequency, and envelope domains are extracted and synthesized into structured natural language cues. This anchors the model's reasoning process to industrial physical mechanisms, ensuring the physical authenticity and interpretability of the diagnostic results. Fine-grained patches are obtained by segmenting the original signal using a sliding window, and then mapped to the LLM embedding space via a trainable linear projection layer, preserving high-fidelity waveform details while reducing computational complexity. A low-rank adaptation layer enables efficient parameter fine-tuning, ensuring model performance while reducing hardware resource consumption, making it suitable for consumer-level deployment scenarios. Physically anchored cues guide the model to learn shape-invariant fault signatures, rather than relying on statistical features, enhancing the model's resilience to changing operating conditions and its cross-domain generalization ability. By mutually reinforcing physical anchoring cues and signal patches, weak fault signatures masked by statistical features are captured, improving diagnostic performance in weak fault scenarios. Leveraging the contextual reasoning capabilities and heterogeneous interaction paradigms of LLMs, accurate diagnosis is achieved even in situations where fault samples are scarce, such as zero-sample and few-sample scenarios, reducing the reliance on labeled data in industrial deployments. A pioneering physical information-guided collaborative representation method combines domain knowledge semantics with the reasoning capabilities of large models by constructing a bimodal input of structured physical cues and high-fidelity signal patches. This preserves the complete dynamic details of the original signal to capture weak fault features while providing an interpretable semantic basis for the diagnostic process through physical cues, effectively solving the "black box" problem and information loss dilemma in traditional methods. Parallel heterogeneous interaction and strong generalization mechanism: The framework designs a unique parallel heterogeneous interaction mechanism, enabling large language models to simultaneously process text semantics and signal waveforms, and achieving cross-modal attention guidance and feature fusion. This mechanism endows the model with powerful shape-invariant feature learning capabilities, enabling it to demonstrate excellent generalization performance and rapid adaptation in cross-condition and cross-domain diagnostic tasks with zero or few samples, providing a general solution to the challenges of data scarcity and variable operating conditions in industrial practice.

[0122] By simultaneously utilizing physical semantics and high-fidelity signals, this model is particularly suitable for complex industrial scenarios where variable speeds and loads lead to non-stationary signals and drifting feature distributions, such as wind turbine gearboxes, aero engines, and electric drive systems for new energy vehicles. In these scenarios, the model not only maintains high-precision diagnosis, but its decision-making process is also linked to physical fault modes, providing a reliable basis for maintenance decisions and reducing unplanned downtime and safety risks caused by misjudgments or missed detections. Through physical guidance and contextual learning, the Hi-LLM framework can achieve effective diagnosis under zero-sample or few-sample conditions, greatly reducing the model's dependence on the scale of historical fault data. Furthermore, the fault physical semantics it learns have good cross-device and cross-structure transfer potential, allowing diagnostic capabilities to be quickly adapted to different types of rotating machinery systems, significantly shortening the customized development cycle and cost for new scenarios.

[0123] Three publicly available benchmark datasets were used, all of which involved data collection from real bearing systems. System experiments and ablation analyses were then conducted based on these datasets. These datasets are: the Xi'an Jiaotong University SQV variable speed dataset, constructed on a real bearing test bench (used to evaluate basic diagnostic performance under variable speed conditions); the Korea Advanced Institute of Science and Technology (KAIST) variable load dataset, collected on a real rotating machinery platform (used to evaluate the model's cross-condition zero-shot and few-shot generalization ability under real variable load conditions); and the HUST dataset, collected from real motor system bearings and published in BMC Research Notes (used to evaluate the model's cross-domain adaptability across different real mechanical systems). Experimental results show that the proposed method achieves comprehensive diagnostic accuracy of 99.89%, 98.49%, and 97.01% on the SQV, KAIST, and HUST datasets, respectively, significantly outperforming the control method. In particular, in the real variable load zero-sample cross-condition test on the KAIST dataset, the diagnostic accuracy for three unknown target conditions of 0Nm, 2Nm, and 4Nm reaches 84.52%, 97.16%, and 99.66%, respectively, fully demonstrating its excellent cross-condition robustness, weak fault identification capability, and potential for rapid adaptation to small samples in real industrial scenarios.

[0124] Furthermore, the present invention leverages the guiding role of physical semantics in signal analysis: physical anchoring cues encode prior knowledge in the field of rotating machinery fault diagnosis (such as high kurtosis indicating impact faults, and amplitude increases in the 2nd–5th harmonics reflecting asymmetry) in structured natural language, serving as high-level semantic constraints during LLM inference. These cues not only provide interpretable decision-making basis for the model but also act as a "query vector" in the attention mechanism, actively guiding the model to focus on local waveforms (such as high-frequency pulses and periodic modulations) in the original signal patch that match the physical features. This "top-down" semantic guidance mechanism enables the model to accurately locate weak fault signatures from massive amounts of noise, significantly improving the detection capability of weak early faults.

[0125] The role of high-fidelity signal patching in ensuring information integrity: Traditional methods rely on manually or automatically extracted statistical features, which are prone to losing key details such as transient impacts and non-stationary modulations during pooling or dimensionality reduction. This invention uses a sliding window to perform fine-grained segmentation of the original vibration signal, preserving the complete time-domain waveform structure, and maps the continuous numerical sequence to the discrete embedding space of LLM through a trainable linear projection layer. This design avoids information loss caused by feature engineering and enables LLM to directly "read" the original signal segments, thereby capturing weak fault dynamics masked by statistical averaging and providing high-fidelity evidence support for weak fault identification.

[0126] The enhancement of model generalization ability through heterogeneous interactions: Under varying operating conditions (such as load and speed changes), traditional statistical feature distributions are prone to drift, leading to model failure. This invention, however, emphasizes the shape-invariant physical semantics of faults through physical anchoring cues (e.g., "periodic impact exists" rather than "kurtosis = 5.2"), enabling the model to learn the essential fault pattern rather than the numerical representation under specific operating conditions. Simultaneously, signal patches provide actual waveform evidence, and the two are dynamically aligned and verified within the LLM through a self-attention mechanism. The 'semantic-evidence' closed-loop mechanism, achieved through text-guided attention weights and the forward and backward fine-tuning processes of signal features, endows the model with powerful cross-operating condition and cross-device generalization capabilities, enabling reasonable inferences based on physical consistency even under zero-sample conditions.

[0127] The Supporting Role of QLoRA Fine-Tuning in Deployment Feasibility: Addressing the challenge of large language models having numerous parameters and being difficult to deploy on industrial edge devices, this invention employs a highly efficient fine-tuning strategy combining 4-bit quantization loading with low-rank adaptation (LoRA). By updating only a small number of low-rank matrix parameters, adaptation to heterogeneous inputs can be achieved, significantly reducing memory usage and computational overhead. Real-world testing shows that this solution can run in real-time on consumer-grade GPUs (such as the RTX 4090), meeting the industrial demand for lightweight, low-cost deployment and bridging the last mile from the laboratory to the production line for large models.

[0128] The structured output enhances interpretability and credibility: the model's final output not only includes the fault category but also associates it with specific physical characteristics (e.g., "diagnosed as bearing outer race fault, based on: significantly increased envelope kurtosis and periodic impacts in the signal patch"). This output format, which anchors the decision-making process to physical mechanisms, surpasses the ex-post feature ranking interpretation of traditional "black box" models. It achieves physical truth interpretability that is traceable during the process and predictable beforehand, greatly enhancing the trust of maintenance personnel in the diagnostic results and providing a reliable basis for predictive maintenance decisions.

[0129] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A method for fault diagnosis of rotating machinery based on a large language model, characterized in that, Includes the following steps: Vibration signals from rotating machinery are collected and preprocessed using adaptive Z-Score normalization. A dual-modal input is constructed, and physical features in three dimensions—time domain, frequency domain, and envelope domain—are extracted from the preprocessed vibration signal. The physical features generate structured physical anchoring prompt text based on a preset natural language template. The vibration signal is divided into multiple fixed-length fine-grained signal patches through a sliding window, and the fine-grained signal patches are mapped to the embedding space of a large language model through a trainable linear projection layer to obtain a signal embedding sequence. The text embedding sequence obtained by processing the physical anchoring prompt text through the word segmenter and embedding layer of the large language model is concatenated with the signal embedding sequence along the time dimension to form a heterogeneous input sequence. The heterogeneous input sequence is input into a large language model that has been fine-tuned by quantization and low-rank adaptation. A multi-head self-attention mechanism is used to guide the attention of physical semantics to signal details, complete cross-modal collaborative reasoning, and output structured fault diagnosis results, including fault type, fault location and confidence level.

2. The method for diagnosing rotating machinery faults based on a large language model according to claim 1, characterized in that, Preferably, the multi-dimensional physical features include: Used to quantify signal energy, statistical distribution and impact characteristics, and to identify the time-domain characteristics of impact-type anomalies, including mean, standard deviation, root mean square, peak value, kurtosis, skewness, peak factor and shape factor; Frequency domain characteristics used to diagnose imbalance and misalignment faults include fundamental frequency amplitude, 2nd-5th harmonic amplitude, total harmonic distortion, spectral centroid, and root mean square frequency. Envelope domain features used for demodulating high-frequency signals and detecting periodic impact fault characteristics of rolling bearings and gears include envelope mean, envelope standard deviation, envelope root mean square, and envelope kurtosis.

3. The method for diagnosing rotating machinery faults based on a large language model according to claim 1, characterized in that, Segmentation into fine-grained signal patches involves fine-grained segmentation and embedding mapping of the vibration signal. Given a vibration signal sequence X of length L, a sliding window strategy is used to segment it into N fixed-length signal patches. The number of segments, N, can be calculated using a preset window size and stride. A trainable linear projection layer W is then used. p The original numerical signal patch is mapped to the high-dimensional embedding space of the LLM to generate a signal embedding sequence, and a linear projection layer W is applied. p As a semantic aligner, it enables cross-modal alignment between the continuous time series feature space and the discrete text embedding space, allowing LLM to process two modalities of data in parallel.

4. The method for diagnosing rotating machinery faults based on a large language model according to claim 1, characterized in that, The length of the fine-grained signal patch is determined based on the rotational speed of the rotating machinery and the sampling frequency.

5. The method for diagnosing rotating machinery faults based on a large language model according to claim 1, characterized in that, The large language model is LLaMA-3-8B-Instruct, and it is loaded with 4-bit quantization. Parameter fine-tuning is achieved by updating parameters only through a low-rank adaptation layer.

6. The method for diagnosing rotating machinery faults based on a large language model according to claim 1, characterized in that, The large language model processes heterogeneous input sequences through a multi-head self-attention mechanism, where the text embedding sequence serves as the query condition, guiding the large model's attention head to focus on the segments in the signal embedding sequence that match the physical feature description for contextual reasoning.

7. The method for diagnosing rotating machinery faults based on a large language model according to claim 1, characterized in that, Structured physical anchoring prompt text is generated by the built-in knowledge base for rotating machinery fault diagnosis. The knowledge base supports feature-semantic mapping rules for bearing, gear, rotor imbalance, misalignment, and broken bar fault modes.

8. A system for performing the method as described in any one of claims 1 to 7, characterized in that, It includes: The signal acquisition and preprocessing module is used to acquire and normalize vibration signals; The bimodal construction module includes a physical anchoring cue generation unit for generating structured physical anchoring cue text and a fine-grained signal patch embedding unit for generating fine-grained signal patches; The heterogeneous interactive reasoning module integrates a large language model fine-tuned by QLoRA, which is used to receive concatenated heterogeneous input and perform cross-modal reasoning. The diagnostic output module generates structured results that include fault type, location, and confidence level.

9. A computer storage medium, characterized in that, The storage medium includes computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-7.

10. An electronic device, characterized in that, The electronic device includes: Memory, processor, and computer programs stored in memory and executable on the processor, wherein, When the processor executes the program, it implements the method as described in any one of claims 1-7.

Images