A Physically Guided Rotation Speed ​​Adaptive Multimodal Large Model Bearing Fault Diagnosis Method

By using a physically guided speed-adaptive multimodal large model, combined with structured text description and speed-adaptive embedding module, the problem of insufficient robustness and interpretability in rolling bearing fault diagnosis under variable speed conditions is solved, and accurate fault identification is achieved under non-stationary signal and speed drift conditions.

CN123111697APending Publication Date: 2026-11-17TAIYUAN UNIVERSITY OF SCIENCE AND TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611331944.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-31
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Under variable speed conditions, existing rolling bearing fault diagnosis methods suffer from insufficient robustness, generalization, and interpretability, especially in cases of non-stationary signals and speed drift, making accurate fault identification difficult.

Method used

A physics-guided, speed-adaptive, multimodal large-scale model is employed. Through joint representation of structured text descriptions and time-frequency graphs, combined with a speed-adaptive embedding module and an entropy-driven fusion prediction module, bearing fault diagnosis is achieved. This method mitigates the dynamic misalignment of time-frequency observations by injecting physical knowledge, adaptively adjusts the speed matching sharpness, and utilizes information entropy to quantify visual discrimination uncertainty, thereby enhancing global-local decision consistency.

Benefits of technology

It improves the diagnostic generalization ability under variable speed conditions, enhances the robustness and interpretability of the model, reduces the dependence on labeled data, and ensures the accuracy and stability of fault identification under variable speed conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123111697A_ABST
    Figure CN123111697A_ABST
Patent Text Reader

Abstract

This invention relates to the field of intelligent fault diagnosis technology for rotating machinery, and discloses a physical-guided, speed-adaptive, multimodal, large-model bearing fault diagnosis method. This method first constructs a structured text prompt based on physical information, aligning it with the short-time Fourier transform time-frequency spectrum to form a unified visual-semantic representation. Then, a speed-adaptive embedding module is used to dynamically adjust the clarity of multimodal feature matching according to the rotational speed, reducing feature drift. Finally, an entropy-driven fusion prediction module is designed to quantify the uncertainty of visual judgment through information entropy, adaptively fusing local time-frequency features with global semantic reasoning to achieve bearing fault diagnosis under variable speed conditions. This invention improves the accuracy and robustness of fault diagnosis results and is suitable for non-stationary industrial environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent fault diagnosis technology for rotating machinery, and in particular to a physical-guided speed-adaptive multimodal large-model bearing fault diagnosis method. Background Technology

[0002] Rolling bearings, as core load-bearing components of rotating machinery, are widely used in industrial rotating machinery. Their health directly determines the operational safety and production efficiency of the machinery. In actual operating conditions, bearings often operate at variable speeds, resulting in non-stationary vibration signals: the fault impact period drifts with the rotational frequency, and the fault characteristics become blurred. This distortion can easily mask early faults, and failure to identify them in time can lead to significant economic losses and safety accidents. Therefore, conducting accurate diagnosis for variable speed operating conditions has important theoretical and engineering significance.

[0003] Currently, rolling bearing fault diagnosis methods are mainly divided into three categories: physical model-based, signal processing-based, and data-driven. Physical model-based methods rely on dynamic laws and perform well in deterministic systems, but system parameters change over time under varying speeds, making it difficult to establish an accurate global model. Signal processing-based methods can handle non-stationary signals, but they are highly dependent on expert experience, and features are prone to drift when the speed changes drastically, resulting in insufficient diagnostic stability. In recent years, deep learning methods, such as convolutional neural networks and variational autoencoders, have improved diagnostic accuracy by mining the implicit features of vibration signals; however, they are sensitive to distribution shifts under varying speeds and generally lack interpretability. In summary, all three types of methods are constrained by insufficient robustness, generalization, or interpretability under varying speed conditions, necessitating the introduction of intelligent diagnostic methods with stronger semantic understanding and knowledge representation capabilities.

[0004] In recent years, large-scale model technology has been explored and applied in the field of industrial monitoring due to its powerful semantic understanding and multimodal reasoning capabilities. In bearing fault diagnosis, some studies combine vibration signals with expert knowledge text prompts, using multimodal large-scale models to achieve fault diagnosis. However, existing methods have two key limitations: first, they rely on static expert knowledge injection rather than guiding the model to autonomously perceive the dynamic coupling between rotational speed and fault; second, most studies focus on constant rotational speed conditions, failing to incorporate rotational speed as a dynamic adjustment factor into visual-semantic alignment, thus failing to address the drift of time-frequency features with rotational speed. The essence of this lies in the failure to explicitly model the dynamic misalignment between the physical period and time-frequency observations, leading to fluctuations in local discrimination confidence with rotational speed and disrupting global-local alignment consistency.

[0005] In summary, in the current problem of fault diagnosis under variable speed operating conditions, it is necessary to study methods that can construct structured text descriptions by injecting physical knowledge to alleviate the dynamic misalignment between physical cycles and time-frequency observations. At the same time, it is necessary to achieve speed-adaptive visual-semantic matching sharpness adjustment and entropy-driven multi-source discrimination fusion, so as to carry out stable and reliable rolling bearing fault diagnosis in non-stationary industrial environments and provide a solid guarantee for the safe operation of rotating machinery. Summary of the Invention

[0006] This invention provides a physical-guided, speed-adaptive, multimodal, large-model bearing fault diagnosis method. The beneficial effects of this invention are: 1. Alleviate the dynamic misalignment between the physical period and time-frequency observation under varying speeds, and improve the generalization ability of cross-speed diagnostics: Under variable speed conditions, the signal-to-noise ratio varies greatly, the data distribution is non-stationary, and the fault impact period drifts with the rotational frequency. This invention forms a visual-semantic joint representation by aligning structured text descriptions injected with physical knowledge with STFT time-frequency graphs. The rotational speed is explicitly used as a dynamic adjustment factor for matching sharpness through a speed-adaptive embedding module—increasing temperature smoothing weights to suppress noise at low speeds and restoring the base temperature to maintain separability at high speeds—adaptively compensating for time-frequency feature drift, thus achieving collaborative learning of physical mechanisms and data features.

[0007] 2. Address the insufficient confidence level of visual discrimination at low speeds and enhance global-local decision consistency: At low speeds, fault impact energy is weak, signal-to-noise ratio is low, and time-frequency features are ambiguous, leading to easy misdiagnosis. This invention uses an entropy-driven fusion prediction module to quantify the uncertainty of visual discrimination with information entropy: when entropy is high, visual weights are reduced and reliance on global semantic reasoning of a large model is increased; when entropy is low, visual weights are increased and the advantages of local time-frequency discrimination are leveraged. This dynamic balancing mechanism effectively mitigates the fluctuation of local confidence with speed and maintains global-local alignment consistency.

[0008] 3. Enhance model robustness and reduce dependence on labeled data: Industrial field variable speed data annotation is costly and samples are scarce. This invention uses LoRA parameter fine-tuning, freezes pre-trained parameters and sets low-rank matrix updates, adapting to variable speed conditions while retaining general cross-modal knowledge, and avoiding overfitting and catastrophic forgetting in full parameter fine-tuning.

[0009] In a first aspect, the present invention provides a bearing fault diagnosis method based on a physically guided speed adaptive multimodal large model, applied to electronic devices. The electronic devices are equipped with a bearing fault diagnosis model, comprising: acquiring the vibration signal and speed signal of the bearing under test; generating a corresponding time-frequency diagram based on the vibration signal; generating a corresponding speed change pattern based on the speed signal; inputting the time-frequency diagram and speed change pattern into the bearing fault diagnosis model; and obtaining the fault diagnosis result of the bearing under test output by the bearing fault diagnosis model. The bearing fault diagnosis model is obtained through the following steps: Step 1, under variable speed conditions, collecting different test... The process involves three steps: 1) Obtaining vibration and rotational speed signals from the bearing; 2) Determining the fault state of the corresponding test bearing based on the preset fault category labeling information corresponding to the vibration signal; 3) Determining the rotational speed change pattern of the corresponding test bearing based on the rotational speed change pattern and the corresponding fault category label based on the fault state and rotational speed change pattern of different test bearings; 4) Converting the vibration signal into a time-frequency graph; 5) Generating a structured text description of the corresponding test bearing based on the rotational speed change pattern and fault category label; 6) Generating training and testing sets based on the time-frequency graphs and structured text descriptions corresponding to different test bearings; 7) Initializing the pre-trained multimodal large-scale model and configuring an efficient fine-tuning strategy for the parameters of the multimodal large-scale model. Step 4: Fine-tune the multimodal large model using the training set for domain adaptation. This includes freezing the original parameters of the multimodal large model and simulating parameter updates by introducing a low-rank matrix to adapt the multimodal large model to the variable-speed bearing fault diagnosis task. Step 5: Using the speed adaptive embedding module in the multimodal large model, determine the visual embedding based on the time-frequency graph, determine the fault category semantic embedding based on the structured text description, determine the similarity between the visual embedding and the fault category semantic embedding, determine the speed adaptive temperature coefficient based on the speed change pattern, and dynamically adjust the similarity using the speed adaptive temperature coefficient to generate fuzzy mappings corresponding to different fault categories. Step 6: Using the entropy-driven fusion prediction module in the multimodal large model, calculate the corresponding normalized entropy based on the fuzzy weight vector, and determine the dynamic fusion coefficient based on the normalized entropy; Based on the classification probabilities of different fault categories output by the multimodal large model and the classification probabilities corresponding to the fuzzy weight vector, fuse them according to the dynamic fusion coefficient to obtain the fusion probabilities corresponding to different fault categories; Traverse the fusion probabilities corresponding to different fault categories, and determine the fault category corresponding to the fusion probability with the highest probability value as the final bearing diagnosis result; Step 7: Repeat steps 4 to 6 until the maximum number of iterations is reached, and stop to obtain the bearing fault diagnosis model.

[0010] In one possible implementation of the first aspect above, converting the vibration signal into a time-frequency graph includes: using a Blackman window as a transform window function to determine a preset fixed window length and a preset window overlap rate; dividing different vibration signals using a corresponding length sliding window with the fixed window length and preset window overlap rate, and then performing STFT to generate time-frequency graphs corresponding to different vibration signals, wherein the time-frequency graphs of different vibration signals are standardized time-frequency graphs.

[0011] In one possible implementation of the first aspect mentioned above, the structured text description includes at least the following three dimensions: equipment attribute dimension, including at least the bearing model, pitch circle diameter, rolling element diameter, number of rolling elements, and bearing installation position parameters of different test bearings; dynamic operating condition dimension, including at least the starting and ending speed values, speed change mode, and operating condition duration of different test bearings, wherein the speed change mode includes at least monotonous acceleration, monotonous deceleration, acceleration followed by deceleration, and deceleration followed by acceleration; and fault characteristic dimension, including at least different fault types and damage levels, wherein the fault types include at least normal, inner ring fault, outer ring fault, rolling element fault, and compound fault types, and both inner ring fault and outer ring fault include the degrees of slight damage, moderate damage, and severe damage.

[0012] In one possible implementation of the first aspect above, training sets and test sets are generated based on the time-frequency diagrams and structured text descriptions corresponding to different test bearings, including: using the time-frequency diagrams and structured text descriptions corresponding to different test bearings as full samples; dividing the preprocessed full samples into training sets and test sets according to a preset ratio, wherein the training sets and test sets include complete data and restricted data, the complete data uses all segmented sub-samples, and the restricted data directly uses the original continuous vibration signal without sample segmentation.

[0013] In one possible implementation of the first aspect mentioned above, the pre-trained multimodal large model adopts Qwen2.5-VL. The multimodal large model has a built-in Vision Transformer visual encoder and a large model, which can accept joint input of time-frequency graphs and structured text descriptions. The multimodal large model has formed a stable cross-modal alignment capability during the pre-training stage.

[0014] In one possible implementation of the first aspect above, the parameter-efficient fine-tuning strategy employs LoRA as the fine-tuning strategy, wherein the fine-tuning strategy satisfies the following: the pre-trained original weights W of the multimodal large model remain frozen, and the parameter update amount ΔW is composed of two low-rank matrices. , The product approximation is used, where d is the feature dimension of the original weights of the pre-trained multimodal large model, and r is the rank of the low-rank decomposition of LoRA.

[0015] In one possible implementation of the first aspect above, determining the adaptive temperature coefficient of rotational speed based on the rotational speed variation pattern includes: resolving the initial rotational speed value X and the final rotational speed value Y from the rotational speed variation pattern; and calculating the average initial rotational speed. At maximum operating speed S max Perform normalization: , , where s is the input speed adaptive embedded module used to determine the speed adaptive temperature coefficient.

[0016] In one possible implementation of the first aspect above, the processing procedure of the rotational speed adaptive embedding module includes: Step 5.1, encoding the time-frequency map using the built-in Vision Transformer visual encoder of the multimodal large model Qwen2.5-VL, and outputting different visual embeddings at the image level. Step 5.2: Perform word embedding and masked average pooling on the structured text description to output semantic embeddings for different fault categories. Step 5.3, use the Pearson coefficient to measure the coefficient using the formula. Computer vision embedding semantic embedding with fault category The similarity, among which, For visual embedding The mean, semantic embedding for fault categories The mean; Step 5.4, using the formula Construct an adaptive temperature coefficient for rotational speed, where τ0 is the base temperature, β is the adjustment range, and k is the steepness factor; Step 5.5, combine the adaptive temperature with Softmax to generate fuzzy weight vectors corresponding to different fault categories: Step 5.6: Weighted fusion of all fault category semantic embeddings to generate fuzzy semantic embeddings: Where C represents the total number of fault categories.

[0017] In one possible implementation of the first aspect above, the processing procedure of the entropy-driven fusion prediction module includes: Step 6.1, extracting the terminal embedding features h from the output of the last layer of the multimodal large model. last Determine the embedding feature h last semantic embedding with different fault categories The similarity, classified by the multimodal large model, is determined by the temperature coefficient τ. llm After scaling, the classification probabilities of different fault categories are obtained using softmax. Step 6.2: Calculate the information entropy based on the fuzzy weight vector α. Where ε is a minimal constant; Step 6.3, normalize the information entropy. The dynamic fusion coefficient is determined based on the normalized entropy. Step 6.4: Weave the fuzzy weight vector with the fusion coefficients and the classification probabilities of different fault categories, and the fusion probabilities corresponding to different fault categories. .

[0018] In a second aspect, embodiments of the present invention provide a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to implement any of the physically guided speed adaptive multimodal large model bearing fault diagnosis methods provided in the first aspect and various possible implementations of the first aspect.

[0019] Thirdly, embodiments of the present invention provide an electronic device comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, one of the processors of the electronic device, for executing the instructions stored in the memory to implement any of the physically guided speed adaptive multimodal large model bearing fault diagnosis methods provided by the first aspect and various possible implementations of the first aspect.

[0020] Fourthly, embodiments of the present invention provide a program product that includes instructions that, when executed by an electronic device, enable the electronic device to implement any of the physical-guided speed-adaptive multimodal large-model bearing fault diagnosis methods provided in the first aspect and various possible implementations of the first aspect. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a physical-guided rotational speed adaptive multimodal large-model bearing fault diagnosis method. Figure 2 A strategy for efficient fine-tuning of LoRA parameters; Figure 3 A schematic diagram of the construction of a structured text description paradigm for injecting physical knowledge, including the hierarchical organization of equipment attribute dimensions, dynamic operating condition dimensions, and fault characteristic dimensions; Figure 4 This is a schematic diagram of the adaptive speed embedding module, demonstrating the complete process of speed parsing and normalization, Pearson similarity calculation, adaptive temperature adjustment, and fuzzy semantic embedding generation. Figure 5 This is a schematic diagram of the entropy-driven fusion prediction module, illustrating the decision-making process of information entropy calculation, normalized entropy, dynamic fusion coefficient generation, and final fusion probability output. Figure 6 This is a horizontal bar chart comparing the present invention with a CNN network on the complete OU dataset; Figure 7This is a visualization diagram of the t-SNE comparison with a CNN network on the complete OU dataset of this invention; Figure 8 This is a horizontal bar chart comparing the present invention with a CNN network on the complete SQV dataset; Figure 9 This is a visualization diagram of the t-SNE comparing the present invention with a CNN network on the complete SQV dataset; Figure 10 This is a bar chart comparing the invention with a CNN network on the limited data of the OU dataset; Figure 11 This is a bar chart comparing the invention with a CNN network on the limited data of the SQV dataset; Figure 12 This is the ROC curve of the present invention compared with the existing untuned large model on the complete OU dataset; Figure 13 This is a t-SNE visualization diagram comparing the present invention with an existing untuned large model on the complete OU dataset; Figure 14 This is the ROC curve of the present invention compared with the existing untuned large model on the complete SQV dataset; Figure 15 This is a visualization diagram of the t-SNE of the present invention compared with the existing un-fine-tuned large model on the complete SQV dataset; Figure 16 This invention presents a parametric sensitivity analysis and accuracy cloud map on the complete OU and SQV datasets. Figure 17 This invention provides a parameter sensitivity analysis of the complete OU and SQV datasets, specifically an F1 score cloud-rain map. Figure 18 This is a wall diagram comparing the complete SQV dataset used in this invention to generalize from OU to the existing model with existing models. Figure 19 This is a wall diagram comparing the complete data of the SQV generalization to the OU dataset with existing models in this invention; Figure 20 This is a wall diagram comparing the generalization of the present invention from OU to SQV dataset with limited data and existing models; Figure 21 This is a wall diagram comparing the generalization of SQV to the OU dataset with existing models using limited data. Figure 22 This is a schematic diagram of a module of an electronic device according to the present invention. Detailed Implementation

[0022] The illustrative embodiments of the present invention include, but are not limited to, a physical-guided speed-adaptive multimodal large-model bearing fault diagnosis method.

[0023] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0024] The following is in conjunction with the appendix Figures 1 to 22 The technical solution of the present invention will be described.

[0025] Figure 1 According to some embodiments of the present invention, a schematic flowchart of a physical-guided rotational speed adaptive multimodal large-model bearing fault diagnosis method is shown. It can be understood that... Figure 1 The processes shown are all executed by electronic devices. For the sake of simplicity, the following description will be further elaborated upon. Figure 1 The execution entity will not be described again in the illustrated process. For example, as shown... Figure 1 As shown, the method includes the following steps: Step 1: Vibration signal acquisition and data construction.

[0026] In some embodiments, vibration and rotational speed signals of different test bearings are collected under variable speed conditions. For example, vibration signals of the target test bearing under variable speed conditions are collected in real time using an accelerometer, and the shaft rotational speed signal is simultaneously measured using a speed encoder to construct a multi-condition, multi-modal variable speed vibration dataset.

[0027] For example, in the vibration dataset, the bearing fault status is divided into multiple states such as normal, inner ring fault, outer ring fault, rolling element fault, and compound fault according to the vibration signal. Among them, the inner ring fault and outer ring fault are not distinguished in terms of damage degree, while the latter are divided into three levels: mild, moderate and severe.

[0028] In some embodiments, the fault state of the corresponding test bearing can be determined based on the preset fault category labeling information corresponding to the vibration signal. For example, when the fault state is normal, the preset fault category labeling information can be the uniformity of the energy distribution of the vibration signal, the smoothness of the spectral transition, and the absence of a dominant frequency component; when the fault state is an inner ring fault and the damage is minor, the preset fault category labeling information can be the presence of weak harmonic components in the vibration signal; when the fault state is an inner ring fault and the damage is moderate, the preset fault category labeling information can be the presence of clear harmonic lines in the vibration signal accompanied by obvious amplitude modulation; when the fault state is an inner ring fault and the damage is severe, the preset fault category labeling information can be the presence of significant harmonic components and obvious modulation sidebands in the vibration signal; when the fault state is an outer ring fault and the damage is minor, the preset fault category labeling information can be the presence of weak but continuous frequency components in the vibration signal; when the fault state is an outer ring fault and the damage is moderate, the preset fault category labeling information can be the presence of clear and significant energy frequency components and harmonics in the vibration signal; when the fault state is an outer ring fault and the damage is severe, the preset fault category labeling information can be the presence of a wide-bandwidth, high-energy frequency band with significant sideband activity in the vibration signal.

[0029] For example, in the vibration dataset, the corresponding speed change pattern of the test bearing is determined based on the speed signal. For example, the speed change pattern includes a continuous speed change pattern (where the speed signal is monotonically accelerating (e.g., from 14.1 Hz to 23.8 Hz), monotonically decelerating (e.g., from 28.9 Hz to 13.7 Hz), increasing first and then decreasing (e.g., from 14.7 Hz to 25.3 Hz, then decreasing to 21.0 Hz), decreasing first and then increasing (e.g., from 24.2 Hz to 14.8 Hz, then increasing to 20.6 Hz)) and a complete acceleration-deceleration cycle pattern (stationary-acceleration-steady-deceleration-stationary (e.g., gradually accelerating from a stationary state to 3000 rpm, then maintaining a steady speed, and finally gradually decelerating to 0)).

[0030] Furthermore, the corresponding fault category labels are determined based on the fault state and speed change pattern of different test bearings. For example, when the fault state type is normal and the speed change pattern is first decreasing and then increasing, the fault category labels are normal and first decreasing and then increasing; when the fault state type is inner ring fault, the damage level is minor, and the speed change pattern is continuous speed change mode, the fault category labels are inner ring fault, minor damage, and continuous speed change mode.

[0031] Step 2, constructing multimodal inputs.

[0032] In some embodiments, the vibration signal data obtained in step 1 is preprocessed to obtain visual modality data. For example, the vibration signal is converted into a time-frequency graph; for instance, a two-dimensional time-frequency graph is generated from the vibration signal using a STFT as visual input. For example, a Blackman window is used as the transform window function to determine a preset fixed window length and a preset window overlap rate; after segmenting different vibration signals using a corresponding length sliding window with the fixed window length and preset window overlap rate, an STFT is executed to generate time-frequency graphs corresponding to different vibration signals, wherein the time-frequency graphs of different vibration signals are standardized time-frequency graphs. Simultaneously, a structured text description of the test bearing is generated based on the speed change pattern and fault category label; for example, a structured text description based on physical knowledge is constructed to obtain text modality data. For instance, as shown... Figure 2 As shown, a structured text description includes at least: Equipment attribute dimensions: at least include the bearing model (e.g., ER16K, NSK6203), geometric parameters (e.g., pitch circle diameter, rolling element diameter, number of rolling elements), and bearing installation position parameters for different test bearings.

[0033] Dynamic operating condition dimension: This includes at least the different test bearing speed ranges (such as the start and end values ​​of the speed), speed change modes (such as monotonous acceleration (or monotonous speed increase), monotonous deceleration (or monotonous speed decrease), compound speed change (such as speed increase followed by speed decrease, speed decrease followed by speed increase), complete speed cycle), and operating condition duration, etc.

[0034] Fault characteristic dimensions: including at least the fault state layer (such as normal, inner ring fault, outer ring fault, rolling element fault and compound fault), the damage degree layer (such as inner ring fault and outer ring fault including the degree of mild damage, moderate damage and severe damage), and the observation layer (such as the characteristic clues to be observed in the time-frequency graph).

[0035] In some embodiments, information from the three dimensions of equipment attributes, dynamic operating conditions, and fault characteristics is fused to generate a physically consistent text description. For example, a prompt for a normal bearing with monotonic acceleration: "Analyze the ER16K ball bearing, with specifications of: pitch circle diameter 38.52mm, rolling element diameter 7.94mm, containing 9 rolling elements. This bearing is installed on the SpectraQuest mechanical fault simulator. Current operating status: speed increases from 14.1Hz to 29.0Hz within 10 seconds. Analyze the time-frequency spectrum to ensure smooth energy distribution without isolated peaks. Analyze the vibration time-frequency graph to diagnose the bearing's health condition."

[0036] The time-frequency graph and structured text description are organized into a multimodal input: each sample is organized in the three-message format of system / user / assistant, and then organized into a standardized multimodal input by the Qwen2.5-VL processor, generating a dictionary containing input_ids, attention_mask, pixel_values, image_grid_thw and corresponding labels.

[0037] Furthermore, training and test sets are generated based on the time-frequency diagrams and structured text descriptions corresponding to different test bearings. For example, the time-frequency diagrams and structured text descriptions corresponding to different test bearings are used as the full sample; the preprocessed full sample is divided into training and test sets according to a preset ratio. The training and test sets include complete data and restricted data. The complete data uses all segmented subsamples, while the restricted data directly uses the original continuous vibration signal without sample segmentation.

[0038] Step 3: Backbone model initialization and fine-tuning configuration.

[0039] In some embodiments, a pre-trained multimodal large model is initialized as a backbone diagnostic model (or backbone model). For example, the pre-trained multimodal large model uses Qwen2.5-VL, and incorporates a VisionTransformer visual encoder and a large language model. It can accept multimodal input formats of "time-frequency graph + text prompts" (e.g., it can accept joint input of time-frequency graphs and structured text descriptions). During the pre-training phase, the multimodal large model has developed stable cross-modal alignment capabilities.

[0040] In some embodiments, an efficient fine-tuning strategy for configuring parameters of a multimodal large model is employed. Exemplarily, this efficient fine-tuning strategy (i.e., fine-tuning the configuration) uses LoRA as the fine-tuning strategy, wherein the fine-tuning strategy satisfies the following conditions: the pre-trained original weights W of the multimodal large model remain frozen, and the parameter update amount ΔW is determined by two low-rank matrices. , The product approximation is used, where d is the feature dimension of the original weights of the pre-trained multimodal large model, and r is the rank of the LoRA low-rank decomposition. For example, fine-tuning measures include: rank r=64, scaling factor lora_alpha=128, target module (containing 7 linear layers such as q_proj and v_proj), and dropout rate lora_dropout=0.1; gradient checkpointing and mixed-precision training (bfloat16) are enabled. The initial learning rate is set to 4×10⁻⁴, the batch size is 4, and the gradient accumulation steps are 4.

[0041] Step 4: Domain adaptation fine-tuning.

[0042] In some embodiments, a multimodal large model is fine-tuned for domain adaptation using a training set. This domain adaptation fine-tuning includes: freezing the original parameters of the multimodal large model and simulating parameter updates by introducing a low-rank matrix to adapt the multimodal large model to the variable-speed bearing fault diagnosis task. For example, as... Figure 3 As shown, the backbone diagnostic model is fine-tuned for domain adaptation using the training set. For example, see the following formula (1): (1) As shown in formula (1), for pre-trained weights Update its parameters Reconstructed into two low-dimensional parameter matrices and The product form ( Thus, a low-rank approximation is achieved.

[0043] Furthermore, combining formula (1), given the input Output after adopting LoRA structure For example, see the following formula (2) as follows: (2) In some embodiments, the original weights W are kept frozen during training (e.g., the original parameters are frozen) to preserve the general fault knowledge that the model learns from massive amounts of data. With the help of low-rank updates, the model retains general cross-modal knowledge while adjusting the model to a semantic-visual aligned subspace that is more suitable for the current operating conditions. This allows the model to learn the speed-fault coupling mode specific to variable speed conditions—how the period, energy distribution, harmonic structure, and sideband morphology of the fault impact change synchronously under different speed conditions.

[0044] In some embodiments, the training process employs a standard autoregressive language modeling loss. For example, see the following formula (3): (3) As shown in formula (3), N is the length of the input sequence containing the dynamic history of rotational speed, and y i X is the i-th semantic unit (token) in the fault diagnosis conclusion. input This is an enhanced multimodal input that integrates visual features from time-frequency diagrams, rotational speed descriptions, and fault physics knowledge.

[0045] Step 5: Speed ​​adaptive embedding module.

[0046] In some embodiments, a speed-adaptive embedding module is provided in the multimodal large model. Through this module, the speed is explicitly used as a dynamic adjustment factor for visual-semantic matching. For example, the module determines the visual embedding based on a time-frequency graph, determines the fault category semantic embedding based on a structured text description, determines the similarity between the visual embedding and the fault category semantic embedding, determines a speed-adaptive temperature coefficient based on the speed change pattern, and dynamically adjusts the similarity using this coefficient to generate fuzzy weight vectors and fuzzy semantic embeddings corresponding to different fault categories.

[0047] For example, such as Figure 4 As shown, the input to the rotational speed adaptive embedding module is the image embedding (or visual embedding) output by the visual encoder. For example, the time-frequency map is encoded using the built-in Vision Transformer visual encoder in the multimodal large model Qwen2.5-VL, outputting different visual embeddings at the image level. Furthermore, the normalized rotational speed s∈[0,1] extracted from the system message by the rotational speed adaptive embedding module is output as a fuzzy semantic embedding. and its corresponding fuzzy weight vector , where q represents the embedding used as a query vector in subsequent calculations, and is matched with the predefined fault category semantic embeddings for similarity. Here, D is the embedding vector dimension, corresponding to the hidden layer dimension of the large model, and C is the number of fault categories.

[0048] Speed ​​analysis and normalization (e.g., speed analysis): For the "from X Hz to Y Hz" format, extract the starting speed value X and the ending speed value Y from the speed change pattern, and calculate the average original speed. (Interval Analysis); For the complete acceleration / deceleration process, the peak speed is used as a reference. All original speed values ​​are based on the maximum operating speed S. max Normalization is performed, as shown in the following formula (4): (4) As shown in formula (4), s max = xxx Hz ( / xxx rpm). The normalized rotational speed s serves as a key parameter for subsequent temperature regulation. For example, the rotational speed s is used to determine the parameters of the speed-adaptive temperature coefficient, which are then fed into the speed-adaptive embedded module. It is understandable that when the rotational speed is high, s tends to 1, and the time-frequency graph of the corresponding vibration signal shows strong impact energy, high signal-to-noise ratio, and significant features; when the rotational speed is low, s tends to 0, and the time-frequency graph of the corresponding vibration signal shows weak impact energy, low signal-to-noise ratio, and blurred features.

[0049] Category semantic embedding construction: Word embedding and masked average pooling are performed on the structured text description to output different fault category semantic embeddings. For example, construct a text description desc for each fault category i. i The token embeddings are obtained from the input embedding layer of a large model, and then semantic embeddings for different fault categories are obtained through masked average pooling. For example, as shown in the following formula (5): (5) As shown in formula (5), m ij E is the attention mask for the j-th token. token This is a word embedding matrix. All category embeddings. Pre-compute and cache before training begins.

[0050] Similarity measurement: The Pearson correlation coefficient is used as the similarity measure to calculate the visual embedding. semantic embedding with fault category The similarity is used to eliminate the influence of rotational speed on the feature mean. For example, the following formula (6) shows: (6) As shown in formula (6), For visual embedding The mean, semantic embedding for fault categories The mean.

[0051] Speed-adaptive temperature regulation: A speed-adaptive temperature coefficient τ(s) is designed to dynamically adjust the matching sharpness. For example, as shown in the following formula (7): (7) As shown in Equation (7), the speed adaptive temperature coefficient is constructed using Equation (7), where τ0 is the base temperature, β is the adjustment range, and k is the steepness factor. When the speed s approaches 0, the temperature is increased to τ0(1+β) to smooth the weight distribution and suppress noise; when the speed s approaches 1, the temperature is restored to the base value τ0 to maintain the separability of features.

[0052] Fuzzy weight calculation: After multiplying the Pearson correlation coefficient by the adaptive temperature, the fuzzy weight vector corresponding to different fault categories is obtained by the Softmax function. For example, the following formula (8) shows: (8) Fuzzy semantic embedding generation: A weighted fusion of all fault category semantic embeddings is used to generate a fuzzy semantic embedding, for example, by weighted fusion of the semantic embeddings of all fault categories. For example, as shown in the following formula (9): (9) As shown in formula (9), C is the total number of fault categories.

[0053] Step 6, Entropy-driven fusion prediction module.

[0054] In some embodiments, the multimodal large model includes an entropy-driven fusion prediction module. This module calculates the information entropy of the fuzzy weight distribution to quantify the uncertainty of visual discrimination criteria. Based on this, fusion coefficients are dynamically generated, adaptively integrating the fuzzy weights with the classification probabilities of the multimodal large model to obtain the final fusion probability. The fault category corresponding to the maximum value is taken as the final fault diagnosis result for the bearing under test. For example, the entropy-driven fusion prediction module calculates the corresponding normalized entropy based on the fuzzy weight vector and determines the dynamic fusion coefficient based on the normalized entropy. It then fuses the different fault category classification probabilities output by the multimodal large model and the classification probabilities corresponding to the fuzzy weight vectors according to the dynamic fusion coefficients to obtain the fusion probabilities corresponding to different fault categories. Finally, it iterates through the fusion probabilities corresponding to different fault categories and determines the fault category corresponding to the fusion probability with the highest probability value as the final bearing diagnosis result. For example, such as Figure 5 As shown, the input to the entropy-driven fusion prediction module includes at least two parts: one is the fuzzy weight vector generated by the rotational speed adaptive embedding module. Secondly, it is based on the classification probability output by the large model. The output is the final probability after fusion. Used for fault category determination.

[0055] Large model classification probability acquisition: Extracting terminal embedding features from the output of the last layer of a multimodal large model. This serves as a semantic summary of the entire input sequence. The terminal embedding feature h is calculated. last Semantic embeddings with different fault categories The similarity, classified by the multimodal large model, is determined by the temperature coefficient τ. llm After scaling, the classification probabilities of different fault categories are obtained through softmax. For example, as shown in the following formula (10): (10) Information entropy calculation: Calculate the information entropy of the fuzzy weight distribution, for example, by calculating the information entropy based on the fuzzy weight vector α. For example, see the following formula (11): (11) As shown in formula (11), ε is a very small constant to avoid taking the logarithm of zero.

[0056] Entropy normalization: Normalizing the entropy to the interval [0,1] yields the normalized information entropy. For example, as shown in the following formula (12): (12) As shown in formula (12), logC is the maximum possible entropy when the weight distribution is uniform.

[0057] Dynamic fusion coefficient generation: The dynamic fusion coefficient is determined based on the normalized entropy. For example, as shown in the following formula (13): (13) Probabilistic Fusion and Decision (Entropy-Driven Fusion Prediction Module): This module weights and fuses the fuzzy weight vector with different fault category classification probabilities using fusion coefficients to obtain the fusion probabilities corresponding to different fault categories. For example, it combines the fuzzy weight vector α with the large model classification probability p. llm The final fusion probability is obtained by weighting the combinations according to the fusion coefficient λ. For example, the following formula (14) shows: (14) As shown in formula (14), the category corresponding to the maximum value in the fusion probability is taken as the final diagnostic result.

[0058] Step 7: Iterative training convergence and model evaluation.

[0059] In some embodiments, steps 4 to 6 are repeated until the maximum number of iterations is reached, resulting in the final bearing fault diagnosis model. After training, the low-rank matrix product (BA) is merged back into the original weights, enabling the model to dynamically adapt to changes in rotational speed during inference without additional computational delay. The model performance is evaluated using a test set, with evaluation metrics including accuracy, F1-score, precision, recall, ROC curve, and t-SNE scatter plot.

[0060] In some embodiments, after obtaining the bearing fault diagnosis model, the model can be deployed inside an electronic device or in the cloud. For example, when fault diagnosis of a bearing under test is required, the electronic device acquires the vibration signal and rotational speed signal of the bearing under test, generates a corresponding time-frequency diagram based on the vibration signal, and generates a corresponding rotational speed change pattern based on the rotational speed signal. The time-frequency diagram and rotational speed change pattern are then input into the bearing fault diagnosis model to obtain the corresponding fault diagnosis result for the bearing under test output by the bearing fault diagnosis model.

[0061] In some embodiments, the training datasets used in this invention include the OU variable speed bearing vibration dataset (University of Ottawa dataset) and the SQV variable speed bearing vibration dataset (SQ-Variable-speed, Xi'an Jiaotong University variable speed Spectra-Quest dataset).

[0062] In other embodiments, the present invention may also use other or fewer training datasets, without limitation.

[0063] As can be understood, in the OU variable speed bearing vibration dataset (hereinafter referred to as OU), the experiment was conducted on the SpectraQuest mechanical fault simulator. Vibration signals were acquired using an accelerometer, and the shaft speed was simultaneously measured using an incremental encoder. All data were recorded via an NI USB-6212 BNC acquisition card at a sampling frequency of 200 kHz for a duration of 10 seconds. The dataset contains five bearing states: normal, inner ring fault, outer ring fault, rolling element fault, and combined fault. The speed conditions were designed with four speed variation modes: monotonic acceleration, monotonic deceleration, acceleration followed by deceleration, and deceleration followed by acceleration. Each combination of bearing state and speed condition was repeated three times, resulting in a total of 60 data files. The bearing model used in the experiment was ER16K, as shown in Table II. Its structural parameters include a pitch circle diameter of 38.52 mm, a rolling element diameter of 7.94 mm, and 9 rolling elements. Based on this parameter, the ratio of fault characteristic frequency to shaft rotation frequency can be calculated: the fault characteristic frequency of the inner ring is 5.43 times the rotation frequency, the fault characteristic frequency of the outer ring is 3.57 times the rotation frequency, and the fault characteristic frequency of the ball is 2.32 times the rotation frequency. Each file in the dataset contains two channels: channel 1 is the vibration acceleration signal measured by the accelerometer, and channel 2 is the shaft rotation speed signal measured by the encoder. Understandably, in the SQV variable speed bearing vibration dataset (hereinafter referred to as SQV), the experiment utilized SpectraQuest's comprehensive mechanical fault simulation test bench to simulate outer and inner ring faults in motor bearings. The test bench consists of three main parts: a motor, a rotor, and a load. A piezoelectric accelerometer was used to collect motor bearing signals, with a CoCo80 data acquisition instrument and a sampling frequency of 25.6 kHz. The motor bearing model was NSK6203, with the faulty bearing located at the motor drive end. The accelerometer was deployed above the motor drive end cover via a magnetic base. Six faults were simulated, including inner and outer ring faults under three different damage levels. Vibration signals from the six faulty bearings and the bearing in normal condition were collected. Each experiment lasted 15 seconds, including a complete acceleration and deceleration process from a stationary state to gradually accelerate to 3000 rpm, then stabilize, and finally gradually decelerate to 0. The severity of fault damage was quantified by artificially creating single-point defects with different damage diameters and depths through machining. A minor fault had a damage area of ​​4 mm² and a damage depth of 0.5 mm; a moderate fault had a damage area of ​​8 mm² and a damage depth of 1 mm; and a severe fault had a damage area of ​​12 mm² and a damage depth of 2 mm. Minor, moderate, and severe faults were represented by 1, 2, and 3, respectively. Two TXT files were obtained from each data acquisition experiment. The channel 2 output file, RECXXXX_ch2.txt, corresponds to the vibration signal and contains two columns of data: column X corresponds to the time scale in seconds; column Y represents the vibration amplitude at the corresponding time scale, in units of gravitational acceleration. The channel 3 output file, RECXXXX_ch3.txt, corresponds to the velocity pulse signal and contains two columns of data: column X corresponds to the time scale in seconds; and column Y corresponds to the pulse voltage signal at the time scale, in units of V.

[0064] To more clearly explain the beneficial effects of this invention, the following examples illustrate the analysis of model training and testing results.

[0065] In some embodiments, evaluation was performed over 10 epochs when compared to a CNN. When compared to a large model, each baseline model underwent 5 training epochs. For fine-tuning the large model, the LoRA configuration included rank r = 64, scaling factor lora_alpha = 128, target module (containing 7 linear layers including q_proj and v_proj), and dropout rate lora_dropout = 0.1; gradient checkpointing and mixed-precision training (bfloat16) were enabled. The initial learning rate was set to 2 × 10⁻⁴, batch size to 4, and gradient accumulation steps to 4. Evaluation metrics included Accuracy, F1-score, Precision, Recall, ROC curves, and t-SNE scatter plots.

[0066] For example, ablation experiments of the rotational speed adaptive embedding module and the entropy-driven fusion prediction module are shown in Tables 1 and 2 below: Table 1 Ablation experimental results on the OU dataset

[0067] As shown in Table 1, the baseline accuracy on OU is only 0.7310, indicating some feature drift under variable speed conditions. Adding the speed adaptive embedding module or the entropy-driven fusion prediction module separately improves the accuracy to 0.9520 and 0.9400, respectively, indicating that each module contributes to the performance improvement from different perspectives. Combining these two modules further improves the accuracy to 0.9680, confirming their complementary effect.

[0068] Table 2 Ablation experimental results on the SQV dataset As shown in Table 2, the baseline accuracy on SQV is 0.7647, and the F1 score is 0.6310. Adding either the speed-adaptive embedding module or the entropy-driven fusion prediction module individually improves the accuracy to 0.9200 and 0.8600, respectively, indicating that each module contributes to the performance improvement from different perspectives. Combining these two modules further improves the accuracy to 0.9500, demonstrating the effectiveness of the proposed module under more challenging variable speed conditions.

[0069] For example, the comparative quantization results of this invention with various untuned existing multimodal large models (Qwen2.5-VL-3B, Qwen2.5-VL-7B, Gemma3n-E4B, InternVL3_5-4B) on the OU and SQV variable speed bearing datasets are shown in Tables 3 and 4: Table 3. Comparison results with the unadjusted baseline on the OU dataset.

[0070] As shown in Table 3, on the OU dataset, the accuracy of this invention reached 0.9490, which is 29.82% higher than the baseline Qwen2.5-VL-3B (0.7310) in the comparison model. The F1-score, precision, and recall also improved simultaneously.

[0071] Table 4. Comparison results with the unadjusted baseline on the SQV dataset.

[0072] As shown in Table 4, on the SQV dataset, the accuracy of this invention reached 0.9600, which is 19.82% higher than the baseline model Qwen2.5-VL-7B (0.8012). The fine-tuning benefits are still significant in stronger variable speed scenarios, indicating that LoRA can adapt to different degrees of nonstationarity.

[0073] For example, the parameter sensitivity analysis quantification results of the present invention on the OU and SQV datasets are shown in Tables 5 and 6: Table 5 τ0, τ in the OU dataset llm Parameter sensitivity analysis

[0074] As shown in Table 5, the best performance on the OU dataset is at τ0 = 0.6 and τ llm This is achieved when the value is 0.1.

[0075] Table 6 τ0, τ in SQV dataset llm Parameter sensitivity analysis

[0076] As shown in Table 6, on the SQV dataset, τ0 is reduced to 0.1 to achieve better accuracy and F1 score. llm Increase it to 1.0.

[0077] For example, the cross-domain generalization capabilities of the present invention under different data distributions are compared in Tables 7-10: Table 7 Performance Comparison of OU Generalization to SQV Dataset (Full Data)

[0078] Table 8 Performance Comparison of SQV Generalization to OU Dataset (Complete Data)

[0079] Table 9 Performance Comparison of OU Generalization to SQV Dataset with Constrained Data

[0080] Table 10 Performance Comparison of SQV Generalization to OU Datasets with Constrained Data As shown in Tables 7 to 10, the bidirectional transfer experiments were conducted in two directions: OU→SQV (as shown in Tables 7 and 9) and SQV→OU (as shown in Tables 8 and 10). With complete data settings, the present invention achieved an accuracy of 0.7414 in the A→B direction (Table 7) and 0.5563 in the B→A direction (Table 8), outperforming CNN and the baseline LLM. With limited data settings, the present invention achieved an accuracy of 0.9857 in the A→B direction (Table 9) and 0.5000 in the B→A direction (Table 10). These results demonstrate that the present invention possesses robust cross-domain generalization ability under non-stationary drift conditions.

[0081] Figure 6 According to some embodiments of the present invention, a horizontal bar chart comparing the network with a CNN on the complete OU dataset is shown. The present invention achieves an accuracy, F1 score, precision, and recall of 0.9990, which is 5.0 percentage points higher than the second-best OverLock in terms of accuracy, and 5.4 and 8.0 percentage points higher than LSNet and EfficientNetV2, respectively.

[0082] Figure 7 According to some embodiments of the present invention, a t-SNE visualization diagram comparing it with a CNN network on the complete OU dataset is shown. The five fault clusters of the present invention are compactly separated and have high intra-class aggregation, while the three CNNs all exhibit inter-class overlap and intra-class diffusion.

[0083] Figure 8 According to some embodiments of the present invention, a horizontal bar chart comparing the network with a CNN network on the complete SQV dataset is shown. The present invention achieves an accuracy of 0.9930, which is approximately 10.1 percentage points higher than EfficientNetV2 (0.8918), with balanced improvements in both precision and recall.

[0084] Figure 9 According to some embodiments of the present invention, a t-SNE visualization diagram comparing it with a CNN network on the complete SQV dataset is shown. The seven classes of samples in the present invention form compact, independent clusters with clear intra-class aggregation and inter-class separation, while the cluster boundaries of the CNN network are blurred and the samples overlap significantly.

[0085] Figure 10 According to some embodiments of the present invention, a bar chart comparing the present invention with a CNN network on the restricted data of the OU dataset is shown. The present invention achieves a precision and recall of 0.9580 on OU, which is an improvement of 0.8 and 5.8 percentage points, respectively, over the second-best OverLoCK.

[0086] Figure 11According to some embodiments of the present invention, a bar chart comparing the performance of the present invention with that of a CNN network on the limited SQV dataset is shown. The present invention achieves precision and recall of 0.9100 and 0.9060 on SQV, respectively, representing improvements of approximately 0.5 and 4.9 percentage points over OverLoCK.

[0087] Combination Figures 6 to 11 As shown, this paper comprehensively demonstrates the comparison of the present invention with three CNN networks (EfficientNetV2, OverLoCK, and LSNet) on two variable speed bearing datasets, OU and SQV, under both complete and limited data conditions, in terms of quantitative indicators such as diagnostic accuracy and feature clustering quality. This fully verifies that the present invention is superior to CNN methods, especially in scenarios with limited samples.

[0088] Figure 12 According to some embodiments of the present invention, an ROC curve is shown comparing the present invention with an existing untuned large model on the complete OU dataset. The ROC curve shows that the present invention achieves an AUC close to 1.0 in all categories, with a micro-average AUC of 0.9880, representing a 9.90% improvement over the suboptimal large model Gemma3n-E4B (0.8990).

[0089] Figure 13 According to some embodiments of the present invention, a t-SNE visualization diagram comparing the complete OU dataset with existing unadjusted large models is shown. The present invention forms compact, independent clusters of five classes of samples, while the unadjusted model suffers from loose clustering, blurred boundaries, or severe confounding.

[0090] Figure 14 According to some embodiments of the present invention, an ROC curve is shown comparing the model with an existing untuned large model on the complete SQV dataset. The present invention achieves an AUC of 1.0 in all categories, with Qwen2.5-VL-3B achieving an AUC of 0.9110 in category 2 and 0.9390 in category 5.

[0091] Figure 15 According to some embodiments of the present invention, a t-SNE visualization diagram comparing the complete SQV dataset with an existing untuned large model is shown. The present invention shows that the seven classes of samples form compact, independent clusters with high inter-class separation, while InternVL3_5-4B has higher inter-class overlap and more misclassification points.

[0092] Combination Figures 12 to 15As shown, this paper comprehensively demonstrates the comparison between the classification confidence under ROC curves and the t-SNE feature clustering quality of the present invention on two variable speed bearing datasets, OU and SQV, and several existing untuned multimodal large models (Qwen2.5-VL-3B, Qwen2.5-VL-7B, Gemma3n-E4B, InternVL3_5-4B). This fully verifies that after LoRA domain fine-tuning, the AUC of each category of the present invention is close to 1.0 and the feature clusters are compact and separated, which is better than the untuned baseline model. This proves the key role of domain adaptation fine-tuning in improving the performance of variable speed fault diagnosis.

[0093] Figure 16 According to some embodiments of the present invention, a parameter sensitivity analysis-accuracy rain map is shown on complete OU and SQV datasets. For OU, the accuracy distribution points corresponding to each value of the classification probability temperature coefficient are densely packed at low values ​​and scattered at high values, indicating a concentrated base temperature. For SQV, the accuracy distribution points corresponding to each value of the classification probability temperature coefficient are more scattered, indicating a more concentrated base temperature.

[0094] Figure 17 According to some embodiments of the present invention, a parameter sensitivity analysis-F1 score cloud-rain plot is shown on complete data of OU and SQV datasets. In OU, the F1 score distribution points corresponding to each value of the classification probability temperature coefficient are densely packed with low values ​​and scattered with high values, indicating a concentrated base temperature; in SQV, the F1 score distribution points corresponding to each value of the classification probability temperature coefficient are dispersed, indicating a relatively concentrated base temperature.

[0095] Combination Figure 16 and Figure 17 As shown, this invention comprehensively demonstrates its effectiveness on two variable speed bearing datasets, OU and SQV, for both the speed-adaptive temperature coefficient τ0 and the LLM classification probability temperature coefficient τ. llm Accuracy and F1 score cloud-rain plot analysis of the two key hyperparameters in the range of [0.1, 1.0] fully verify the differences and adjustability of the performance distribution of the present invention under different hyperparameter values, and maintain an acceptable performance range over a wide parameter range, proving that the method has good parameter robustness and engineering deployment stability.

[0096] Figure 18 According to some embodiments of the present invention, a wall diagram comparing the complete data from OU generalization to the SQV dataset with existing models is shown. The present invention achieves high accuracy across 5 epochs, followed by Qwen2.5-VL-7B and Qwen2.5-VL-3B, while traditional CNN networks show lower accuracy. The accuracy of each model tends to stabilize with each epoch.

[0097] Figure 19According to some embodiments of the present invention, a wall diagram comparing the complete data of the OU dataset generalized from SQV with existing models is shown. While the cross-domain difficulty increases and the overall accuracy decreases, the accuracy of the present invention (approximately 0.6) remains high, demonstrating strong stability.

[0098] Figure 20 According to some embodiments of the present invention, a wall diagram comparing generalization from OU to the SQV dataset with existing models on limited data is shown. The present invention achieves high accuracy across 5 epochs, while the accuracy of EfficientNetV2 and Qwen2.5-VL-7B fluctuates with epoch changes.

[0099] Figure 21 According to some embodiments of the present invention, a wall diagram comparing SQV generalization to OU dataset with limited data and existing models is shown. The accuracy of each model fluctuates with epochs, but the present invention still achieves high accuracy, verifying its robustness under data-limited conditions.

[0100] Combination Figures 18 to 21 As shown, this paper comprehensively demonstrates the generalization performance of the present invention on a bidirectional cross-domain transfer task between two variable speed bearing datasets, OU and SQV (including both complete and restricted data conditions). A comprehensive comparison with existing CNN networks and the baseline of a large model without fine-tuning is conducted, fully verifying that the present invention consistently outperforms the baseline method under the cross-dataset transfer setting. This proves that the collaborative mechanism of the speed adaptive embedding module and the entropy-driven fusion prediction module can effectively compensate for the feature shift caused by the speed history and sampling differences across datasets, thereby enhancing the model's robust generalization ability to non-stationary operating conditions.

[0101] Figure 22 According to some embodiments of the present invention, a schematic diagram of the structure of an electronic device 100 is shown. It will be understood that the electronic device may incorporate all the models mentioned above. Figure 22As shown, the electronic device 100 includes a processor 101, a communication interface 103, and a memory 102. The processor 101, communication interface 103, and memory 102 can be interconnected via an internal bus 104, or they can communicate via wireless transmission or other means. This embodiment of the invention uses the connection via bus 104 as an example. Bus 104 can be a peripheral component interconnect express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. Bus 104 can be divided into address bus, data bus, control bus, etc. In addition to the data bus, bus 104 can also include a power bus, a control bus, and a status signal bus. For clarity, all buses are labeled as bus 104 in the figure.

[0102] Processor 101 may consist of at least one general-purpose processor, such as a central processing unit (CPU), or a combination of a CPU and a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Processor 101 executes various types of digital storage instructions, such as software or firmware programs stored in memory 102, enabling electronic device 100 to provide a variety of services.

[0103] Memory 102 may include volatile memory, such as random access memory (RAM); memory 102 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 102 may also include combinations of the above types.

[0104] The communication interface 103 can be a wired interface (e.g., an Ethernet interface), an internal interface (e.g., a PCIe bus interface), a wired interface (e.g., an Ethernet interface), or a wireless interface (e.g., a cellular network interface or a wireless LAN interface), for communicating with other devices or modules.

[0105] It needs to be explained that, Figure 22 This is merely one possible implementation of an embodiment of the present invention. In actual applications, the electronic device 100 may include more or fewer components, which will not be elaborated here.

[0106] In some embodiments, the present invention also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above aspects.

[0107] In some embodiments, the present invention also provides a computer program product comprising: computer program code that, when run on a computer, causes the computer to perform the methods described above.

[0108] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0109] It should be noted that the units / modules mentioned in the various device embodiments of the present invention are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problem proposed by the present invention. Furthermore, to highlight the innovative aspects of the present invention, the above-described device embodiments of the present invention have not introduced units / modules that are not closely related to solving the technical problem proposed by the present invention. This does not mean that the above-described device embodiments do not contain other units / modules.

[0110] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0111] Although the invention has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the scope of the invention.

Claims

1. A bearing fault diagnosis method based on a physically guided rotational speed adaptive multimodal large model, applied to electronic devices, wherein the electronic devices are equipped with a bearing fault diagnosis model, characterized in that... include: The test vibration signal and test speed signal of the bearing under test are acquired. A corresponding test time-frequency diagram is generated based on the test vibration signal, and a corresponding test speed change pattern is generated based on the test speed signal. The test time-frequency diagram and the test speed change pattern are input into the bearing fault diagnosis model to obtain the fault diagnosis result of the bearing under test output by the bearing fault diagnosis model. The bearing fault diagnosis model is obtained through the following steps: Step 1: Under variable speed conditions, collect vibration signals and speed signals of different test bearings, determine the fault state of the corresponding test bearing according to the preset fault category label information corresponding to the vibration signal, determine the speed change mode of the corresponding test bearing according to the speed signal, and determine the corresponding fault category label according to the fault state and speed change mode of different test bearings. Step 2: Convert the vibration signal into a time-frequency graph, generate a structured text description of the test bearing according to the rotational speed change pattern and the fault category label, and generate a training set and a test set according to the time-frequency graph and the structured text description corresponding to different test bearings; Step 3: Initialize the pre-trained multimodal large model and configure an efficient fine-tuning strategy for the parameters of the multimodal large model; Step 4: Use the training set to perform domain adaptation fine-tuning on the multimodal large model, wherein the domain adaptation fine-tuning includes: freezing the original parameters of the multimodal large model, and simulating parameter updates by introducing a low-rank matrix, so that the multimodal large model can be adapted to the variable speed bearing fault diagnosis task. Step 5: Using the speed adaptive embedding module in the multimodal large model, determine the visual embedding based on the time-frequency diagram, determine the fault category semantic embedding based on the structured text description, determine the similarity between the visual embedding and the fault category semantic embedding, determine the speed adaptive temperature coefficient based on the speed change pattern, and use the speed adaptive temperature coefficient to dynamically adjust the similarity to generate fuzzy weight vectors and fuzzy semantic embeddings corresponding to different fault categories. Step 6: Using the entropy-driven fusion prediction module in the multimodal large model, calculate the corresponding normalized entropy based on the fuzzy weight vector, and determine the dynamic fusion coefficient based on the normalized entropy; based on the classification probabilities of different fault categories output by the multimodal large model and the classification probabilities corresponding to the fuzzy weight vector, fuse them according to the dynamic fusion coefficient to obtain the fusion probabilities corresponding to different fault categories; iterate through the fusion probabilities corresponding to different fault categories, and determine the fault category corresponding to the fusion probability with the highest probability value as the final bearing diagnosis result; Step 7: Repeat steps 4 to 6 until the maximum number of iterations is reached, and then stop to obtain the bearing fault diagnosis model.

2. The method according to claim 1, characterized in that, The step of converting the vibration signal into a time-frequency diagram includes: The Blackman window is used as the transformation window function to determine the preset fixed window length and preset window overlap rate; After segmenting the different vibration signals using a sliding window of the corresponding length with the fixed window length and the preset window overlap rate, STFT is performed to generate the time-frequency diagrams corresponding to the different vibration signals, wherein the time-frequency diagrams of the different vibration signals are standardized time-frequency diagrams.

3. The method according to claim 1, characterized in that, The structured text description includes at least the following three dimensions of text description: The equipment attribute dimension includes at least the bearing model, pitch circle diameter, rolling element diameter, number of rolling elements, and bearing installation position parameters of the different test bearings; The dynamic operating condition dimension includes at least the different starting and ending speed values, speed change modes, and operating condition duration of the test bearings. The speed change modes include at least monotonous acceleration, monotonous deceleration, acceleration followed by deceleration, and deceleration followed by acceleration. The fault characteristic dimension includes at least different fault types and damage degrees. The fault types include at least normal, inner ring fault, outer ring fault, rolling element fault and compound fault types. The inner ring fault and the outer ring fault both include degrees of mild damage, moderate damage and severe damage.

4. The method according to claim 1, characterized in that, The step of generating training and testing sets based on the time-frequency diagrams and structured text descriptions corresponding to different test bearings includes: The time-frequency diagrams and structured text descriptions corresponding to different test bearings are used as the full sample. The preprocessed full sample is divided into the training set and the test set according to a preset ratio. The training set and the test set include complete data and restricted data. The complete data uses all segmented subsamples, while the restricted data directly uses the original continuous vibration signal without sample segmentation.

5. The method according to claim 1, characterized in that, The efficient parameter fine-tuning strategy employs LoRA as the fine-tuning strategy, wherein the fine-tuning strategy satisfies the following: the pre-trained original weights W of the multimodal large model remain frozen, and the parameter update amount ΔW is composed of two low-rank matrices. , The product approximation is used, where d is the feature dimension of the original weights of the pre-trained multimodal large model, and r is the rank of the low-rank decomposition of LoRA.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the adaptive temperature coefficient of rotational speed based on the rotational speed change pattern includes: The starting speed value X and the ending speed value Y are extracted from the speed change pattern. Calculate the average original speed: At maximum operating speed S max Perform normalization: , , where s is a parameter used to determine the speed adaptive temperature coefficient and is sent to the speed adaptive embedded module.

7. The method according to claim 6, characterized in that, The processing procedure of the speed adaptive embedding module includes: Step 5.1: Encode the time-frequency map using the built-in Vision Transformer visual encoder in the multimodal large model Qwen2.5-VL to output different visual embeddings at the image level. ; Step 5.2: Perform word embedding and masked average pooling on the structured text description to output semantic embeddings for different fault categories. ; Step 5.3, use the Pearson coefficient to measure the coefficient using the formula. Calculate the visual embedding semantic embedding with the fault category The similarity, among which, For the visual embedding The mean, semantic embedding for the fault category The mean; Step 5.4, using the formula Construct the speed-adaptive temperature coefficient, where τ0 is the base temperature, β is the adjustment range, and k is the steepness factor; Step 5.5: Combine the adaptive temperature with Softmax to generate the fuzzy weight vectors corresponding to different fault categories: ; Step 5.6: Weighted fusion of all the fault category semantic embeddings to generate the fuzzy semantic embedding: Where C represents the total number of fault categories.

8. The method according to claim 7, characterized in that, The processing steps of the entropy-driven fusion prediction module include: Step 6.1: Extract the terminal embedding features h from the output of the last layer of the multimodal large model. last Determine the embedding feature h last semantic embeddings of different fault categories The similarity, classified by the multimodal large model, is determined by the temperature coefficient τ. llm After scaling, the classification probabilities of different fault categories are obtained using softmax. ; Step 6.2: Calculate the information entropy based on the fuzzy weight vector α. , where ε is a local constant; Step 6.3, Normalized Information Entropy The dynamic fusion coefficient is determined based on the normalized entropy. ; Step 6.4: Weigh the fuzzy weight vector with the classification probabilities of different fault categories using the fusion coefficients to obtain the fusion probabilities corresponding to different fault categories. .

9. A computer-readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 8.

10. An electronic device, characterized in that, include: Memory is used to store instructions executed by one or more processors of an electronic device; And a processor, one of the processors of the electronic device, for executing instructions stored in the memory to implement the method of any one of claims 1 to 8.