A cross-domain fault diagnosis method based on time-frequency characteristics and multi-scale adapter

Abstract

The application provides a cross-domain fault diagnosis method based on time-frequency characteristics and a multi-scale adapter. First, the original signal of a rotating machine is collected and converted into a two-dimensional time-frequency image; second, local features are extracted from the time-frequency image to obtain a feature map; third, a Transformer backbone network based on a gated parallel axis attention mechanism and embedded with a multi-scale feature adapter is constructed to extract time sequence features and frequency band features of the signal and perform weighted fusion through a gating network; fourth, deep features are input into a multi-center cosine classification head to output a diagnosis result; and finally, when facing a new working condition of a target domain, a local parameter fine-tuning strategy is adopted, the feature extraction weight of the backbone network is fixed, and only a small number of parameters are fine-tuned by using a small number of target domain samples. The application improves the extraction effect of time-frequency features and alleviates the overfitting problem in the small sample cross-domain scene through local parameter fine-tuning.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of mechanical equipment health monitoring and artificial intelligence, specifically involving a cross-domain fault diagnosis method based on time-frequency features and multi-scale adapters for small sample and variable operating condition scenarios. Background Technology

[0002] Rotating machinery (such as electric motors, wind turbines, aircraft engines, and various industrial machine tools) serves as the core power load in modern industrial manufacturing, transportation, and energy production systems. Its operating status directly determines the safety, stability, and economic efficiency of the entire production system. Bearings and motor stators / rotors, as the most complex and frequently operating critical components within rotating machinery, are constantly exposed to high loads, alternating stresses, and complex environmental disturbances, making them highly susceptible to early failures such as wear, fatigue spalling, or breakage. Relevant industrial statistics indicate that a significant proportion of downtime accidents in rotating machinery systems are caused by the failure of these critical components. Therefore, research into efficient and intelligent fault diagnosis technologies is of great significance for ensuring infrastructure safety and promoting the intelligent transformation of the manufacturing industry.

[0003] Traditional fault diagnosis methods often rely on manual extraction of time-domain statistical indicators (such as root mean square, peak factor, and kurtosis) or frequency-domain analysis (such as Fast Fourier Transform spectral envelope extraction). However, the impact signals generated in the early stages of mechanical faults are usually weak and exhibit non-stationary and nonlinear characteristics, often masked by mechanical noise and electromagnetic interference in actual industrial settings. Traditional feature engineering, dependent on the experience of domain experts, struggles to adapt to the demands of massive industrial data and variable operating environments. Currently, with the development of deep learning technology, converting one-dimensional vibration or acoustic sensor signals into two-dimensional time-frequency maps, and then utilizing deep neural networks for end-to-end automatic feature extraction and fault classification, has become the mainstream approach in data-driven diagnostics.

[0004] However, existing deep learning diagnostic models still face some unresolved issues when applied in actual industrial settings.

[0005] Currently, existing network architectures typically treat the converted time-frequency image as a regular optical image, failing to fully consider its unique physical topology. Specifically, the horizontal axis represents the temporal evolution of the time period, while the vertical axis represents the energy frequency distribution of the fault impact. Conventional multi-head self-attention mechanisms, when processing time-frequency images, generally aggregate features from both the time and frequency axes globally, without distinguishing between these two types of physical information. This computational approach leads to model attentional distraction when faced with complex industrial background noise, and the true spatiotemporal evolution characteristics of the fault are easily masked by environmental clutter, affecting the model's ability to extract features from early, weak faults.

[0006] Meanwhile, the operating conditions (such as speed, torque, and load) and operating environment of equipment in industrial settings change frequently, leading to significant domain shifts in the distribution of data collected from the equipment. Currently, models trained under specific steady-state conditions often suffer from reduced diagnostic accuracy when directly applied to new operating conditions. However, in real-world industrial scenarios, labeled fault samples for new operating conditions are often scarce. Using these limited samples to fine-tune all parameters of a deep neural network with a large number of parameters is not only computationally expensive but also prone to causing the model to forget the general physical laws learned during pre-training, resulting in overfitting on the target domain data.

[0007] Furthermore, existing fault diagnosis models often rely on traditional linear classifiers consisting of fully connected layers and a softmax function for measurement at the output. However, under varying operating conditions, deep features extracted from the same mechanical defect undergo nonlinear shifts and scaling in the feature space. A single linear classification boundary struggles to accommodate the diverse variations of similar fault features under different operating conditions. Due to the lack of an active compensation mechanism for cross-domain drift in the feature space, the generalization ability of traditional classification boundaries is often limited when encountering unknown target domain data distributions. Therefore, a more complex relational formula is chosen for prediction, resulting in a more accurate prediction and stronger robustness.

[0008] To address the above issues, this invention provides a cross-domain fault diagnosis method based on Transformer and PEFT. Starting from the model architecture, this method designs a feature extractor that can better separate spatiotemporal noise and develops local parameter update and feature drift compensation strategies suitable for small-sample, variable-condition scenarios. Summary of the Invention

[0009] This invention provides a cross-domain fault diagnosis method based on time-frequency features and a multi-scale adapter. This method acquires one-dimensional vibration or sound signals of rotating machinery through sensors, then preprocesses the signals to convert them into two-dimensional time-frequency images containing information about the operating status of the rotating machinery. Subsequently, a constructed neural network model is used to classify faults in the time-frequency image, and the parameters of the existing source domain model can be updated using a small amount of target domain data, thus realizing cross-domain fault diagnosis of rotating machinery.

[0010] The cross-domain fault diagnosis method based on time-frequency characteristics and multi-scale adapters described in this invention specifically includes the following steps:

[0011] Step 1: Input the two-dimensional time-frequency image containing the operating status information of the rotating machinery into the front-end feature extraction module of the Convolutional Neural Network (CNN). The local features of the time-frequency image are extracted through multi-layer convolution to obtain the feature map.

[0012] Step 2: Input the feature map into a Transformer network block containing a gated parallel axis attention mechanism. The standard multi-head self-attention layer in the Transformer network block is replaced with a gated parallel axis attention mechanism, which performs independent feature extraction along the time axis and frequency axis, and uses a global context-based gated network to calculate dynamic weights to weighted fuse the two-path features.

[0013] Step 3: Embed multi-scale adapters in the network, specifically after the feature extraction module at the CNN front end, after the residual connections of the gated fusion layer in the Transformer Block, and after the residual connections of the FFN layer. Utilizing... Parallel depthwise convolutional branches extract multi-receptive field features, global average pooling is used to perceive the context, and dynamic fusion weight ratios of branches at each scale are generated by MLP calculation.

[0014] Step 4: Input the deep features into the multi-center cosine similarity classification head. Multiple prototype sub-centers are set for each fault category, and the maximum cosine similarity is extracted. During the calculation, a dynamic scaling factor based on feature magnitude and a drift compensation term based on domain offset are introduced to output the final diagnostic result.

[0015] Step 5: Use the PEFT strategy for cross-domain model updates. After pre-training using the source domain dataset, use only a small number of samples in the target domain, fix most of the parameters of the backbone network, and only perform local fine-tuning and optimization on the multi-scale adapter parameters, gating network parameters, and classifier head drift compensation parameters to adjust the model to a new model adapted to the target domain data. Attached Figure Description

[0016] Figure 1 This is the overall flowchart of the present invention.

[0017] Figure 2 This is a diagram of the overall structure of the neural network of the present invention.

[0018] Figure 3 This is a diagram of the experimental platform for acquiring fault signals in rotating machinery according to the present invention.

[0019] Figure 4 The original one-dimensional vibration waveform of the bearing.

[0020] Figure 5 The original one-dimensional audio waveform of the motor

[0021] Figure 6 STFT diagram for vibration signal conversion

[0022] Figure 7 Mel spectrogram for audio signal conversion

[0023] Figure 8 A structural diagram of the Transformer Block, which includes a gated parallel-axis attention mechanism and a multi-scale adapter.

[0024] Figure 9 Multi-scale adapter structure diagram

[0025] Figure 10 Structural diagram of a multicenter cosine classification head and drift compensation mechanism

[0026] Figure 11 Schematic diagram of the fine-tuning mechanism Detailed Implementation Plan

[0027] The present invention will be further described below with reference to the accompanying drawings:

[0028] like Figure 1 The diagram shows a flowchart of a cross-domain fault diagnosis method based on time-frequency features and a multi-scale adapter. The neural network model structure used is as follows: Figure 2 As shown, the specific steps include:

[0029] Step 1: Set up as follows Figure 3 The rotating machinery fault simulation test bench shown includes equipment such as the motor under test, transmission bearings, sound sensors, and data acquisition terminals. During operation, an accelerometer is used to collect one-dimensional discrete vibration signals from components such as bearings, or a high-fidelity microphone array is used to collect one-dimensional discrete acoustic signals from the motor. The acquired raw one-dimensional time series signal contains background noise from steady-state mechanical operation and weak transient impacts from faults. The waveform of the raw one-dimensional signal is as follows: Figure 4 and Figure 5 As shown.

[0030] In step two, the acquired one-dimensional sequence is converted into a two-dimensional physical time-frequency image. For vibration acceleration signals, this invention employs Short-Time Fourier Transform (STFT) to extract features, obtaining, as follows: Figure 6 The STFT diagram is shown. A sliding window function is introduced. (e.g., Hamming window), the discrete calculation formula for STFT is as follows:

[0031] (1) Among them, For time frame indexing, For frequency index, This is the step size for the window to slide. This represents the number of points in the Fourier transform. Subsequently, amplitude-frequency information is obtained by calculating the modulus of the complex matrix, generating a two-dimensional time-frequency plot matrix. :

[0032] (2)

[0033] For acoustic microphone signals, to reduce interference from high-frequency environmental white noise, this invention uses Mel-spectrum analysis to extract time-frequency features. First, the power spectrum of the signal is calculated. Then, the linear frequency is determined according to the Mel scale formula. Converted to nonlinear Mel frequency :

[0034] (3)

[0035] Construct a series of Mel filter banks arranged in a triangular pattern. The power spectrum is passed through a filter bank to obtain the Mel spectrum matrix. :

[0036] (4) Finally, a logarithmic amplitude conversion is performed to scale it to the decibel (dB) level to highlight the low-frequency resonance characteristics:

[0037] (5)

[0038] in To prevent the occurrence of a minimal constant with negative infinity when taking the logarithm, the generated two-dimensional matrix is ​​then reshaped into a two-dimensional time-frequency image input network with uniform spatial resolution using a bilinear interpolation algorithm. The resulting Mel spectrogram is shown below. Figure 7 As shown.

[0039] Step 3: Convert the two-dimensional time-frequency image The input is fed into the CNN front-end feature extraction module, where and These are the height and width of the image. This refers to the number of channels in the convolutional layer. This module contains cascaded convolutional layers, batch normalization layers, and the Swish activation function; the input features are adjusted by setting the convolution stride. Spatial downsampling is performed to obtain high-dimensional feature maps. .

[0040] Step 4: Convert the high-dimensional feature map Input to Figure 8 The Transformer block shown contains gated parallel-axis attention. This mechanism abandons global attention computation and extracts features independently along the time and frequency axes by reshaping and transposing the physical dimensions of the tensor. The specific computation includes the following 5 steps:

[0041] Step 1: Add to feature map Inject two-dimensional positional encodings into rows and columns, and generate query, key, and value matrices for row attention through projection, respectively. , , ) and column attention queries, key-value matrices ( , , );

[0042] Step 2: Row attention calculation: Calculate the height of the feature map. Dimensions are merged into batch dimensions, by width The dimension is treated as an independent sequence length, and self-attention computation is performed along the time axis to obtain row attention features:

[0043] (6)

[0044] Step 3: Column Attention Calculation: Transpose the feature map width Dimensions are merged into batch dimensions, based on height. The dimension is treated as an independent sequence length, and self-attention computation is performed along the frequency axis to obtain column attention features:

[0045] (7)

[0046] in, The scaling factor corresponding to the feature dimension;

[0047] Step 4: Calculate global context gating coefficients: for feature maps Perform global average pooling to generate a global context vector, and calculate the dynamic gating coefficients.

[0048] (8)

[0049] in It is the Sigmoid activation function. and These are learnable weights and bias parameters;

[0050] Step 5: Dynamic Fusion and Reshaping: The row attention features and column attention features are weighted and fused using the gating coefficients, and then flattened and reshaped back into a sequence tensor to obtain the final output features.

[0051] (9)

[0052] Step 5: Embed the following in the backbone network: Figure 9The multi-scale adapter is shown. Specifically, it is located after the feature extraction module in the CNN front end, after the residual connections of the gated fusion layer in the Transformer block, and after the residual connections of the FFN layer. The specific computation process of the multi-scale adapter includes the following five steps:

[0053] Step 1: Set the input features as After layer normalization (LN) processing, the scaling factor is used. Perform residual preprocessing: (10)

[0054] Step 2: via After convolution dimensionality reduction, we get Input it in parallel to The three depthwise separable convolutional branches yield multi-scale features. ;

[0055] Step 3: Sum the features of the three branches and perform a GAP operation to obtain the context vector. ;

[0056] Step 4: After applying the MLP and Softmax functions, a scale weight assignment matrix is ​​generated and split into... Weighted fusion is performed:

[0057] (11)

[0058] Step 5: Calculate the compensation characteristics of the adapter output.

[0059] (12)

[0060] in This is the GELU activation function.

[0061] Step Six: Input the deep features into... Figure 10 The multicenter cosine metric classification head shown. The specific prediction process includes the following four steps:

[0062] Step 1: For Each fault category is initialized separately. Subcenter Weight Matrix ;

[0063] Step 2: Extract global features Calculate its L2 modulus. And calculate the adaptive scaling factor. :

[0064] (13)

[0065] in Based on scaling parameters;

[0066] Step 3: Add features With all sub-centers Perform L2 normalization to calculate cosine similarity, and then calculate along the sub-center dimension. Perform max pooling to extract the maximum similarity. :

[0067] (14)

[0068] Step 4: Combine drift bias term Calculate and output classification results :

[0069] (15)

[0070] Finally, the Softmax function outputs the predicted probability for each fault category.

[0071] Step 7: When it is necessary to deploy the model to a new operating condition in the target domain, use methods such as... Figure 11 The PEFT strategy is used to update the model across domains, obtain a small number of labeled fault samples, and set most of the feature extraction network layers of the neural network to an untrainable state.

[0072] By locating via a computational graph, only the following parameters are enabled for gradient updates during backpropagation:

[0073] (1) The weights and biases of the gating layer in the gating parallel axis attention module described in step four;

[0074] (2) All parameters in the multi-scale feature adapter mentioned in step five, including convolutional branches, perceptron layers and residual scaling factors;

[0075] (3) The drift bias term in the multicenter cosine classification head described in step six and scaling base parameters .

[0076] By using small sample data to drive the updates of the aforementioned few parameters, a new model adapted to the new working conditions of the target domain is obtained while alleviating model overfitting.

Claims

1. A cross-domain fault diagnosis method based on time-frequency characteristics and a multi-scale adapter, characterized by the following steps: Step 1: Obtain a feature map that preserves the spatial topology by downsampling the two-dimensional time-frequency image of the rotating machinery through a front-end feature extraction network. The input is fed into stacked Transformer network blocks for deep feature extraction. The standard multi-head self-attention layer within these Transformer network blocks is replaced with a gated parallel-axis attention mechanism, which performs independent feature extraction along the time and frequency axes. The specific computation includes: Step 1: Add to feature map Inject two-dimensional positional encodings into rows and columns, and generate query, key, and value matrices for row attention through projection. , , ) and column attention queries, key-value matrices ( , , ); Step 2: Row attention calculation: Calculate the height of the feature map. Dimensions are merged into batch dimensions, by width. The dimension is treated as an independent sequence length, and self-attention computation is performed along the time axis to obtain row attention features: （1）； Step 3: Column Attention Calculation: Transpose the feature map width Dimensions are merged into batch dimensions, based on height. The dimension is treated as an independent sequence length, and self-attention computation is performed along the frequency axis to obtain column attention features: （2）； in, The scaling factor corresponding to the feature dimension; Step 4: Calculate global context gating coefficients: for feature maps Perform global average pooling to generate a global context vector, and calculate the dynamic gating coefficients. （3）； in It is the Sigmoid activation function. and These are learnable weights and bias parameters; Step 5: Dynamic Fusion and Reshaping: The row attention features and column attention features are weighted and fused using the gating coefficients, and then flattened and reshaped back into a sequence tensor to obtain the final output features. （4）； Step 2: Embed a multi-scale adapter in the backbone network, specifically after the front-end feature extraction module, after the residual connection of the gated fusion layer in the Transformer block, and after the residual connection of the fully connected feed-forward network (FFN) layer. The specific calculation process of the multi-scale feature adapter includes the following 5 steps: Step 1: Set the input features as After layer normalization (LN) processing, the scaling factor is used. Perform residual preprocessing: (5); Step 2: via After convolution dimensionality reduction, we get Input it in parallel to The three depthwise separable convolutional branches yield multi-scale features. ; Step 3: Sum the features of the three branches and perform global average pooling (GAP) to obtain the context vector. ; Step 4: After passing through a Multi-Layer Perceptron (MLP) and the Softmax function, a scale weight assignment matrix is ​​generated and split into... Perform weighted fusion: （6） ； Step 5: Calculate the compensation characteristics of the adapter output: （7）； in This is the activation function for the Gaussian Error Linear Unit (GELU). Step 3: Input the deep features into the multicenter cosine metric classification head shown in Figure 10. The specific prediction process includes the following four steps: Step 1: For Each fault category is initialized separately. Subcenter Weight Matrix ; Step 2: Extract global features Calculate its L2 modulus. And calculate the adaptive scaling factor. : （8）； in Based on scaling parameters; Step 3: Add features With all sub-centers Perform L2 normalization to calculate cosine similarity, and then calculate along the sub-center dimension. Perform max pooling to extract the maximum similarity. : （9）； Step 4: Combine drift bias term Calculate and output classification results : （10）； Finally, the input is fed into the Softmax function, which outputs the predicted probability of each fault category. Step 4: When it is necessary to deploy the model to a new working condition in the target domain, the Parameter-Efficient Fine-Tuning (PEFT) strategy is used to update the model across domains. First, the model is pre-trained using a large dataset from the source domain. When it is necessary to deploy the model to a new working condition in the target domain, a small number of labeled fault samples are obtained, and most of the feature extraction network layers of the backbone network are set to an untrainable state. By locating via a computational graph, only the following parameters are enabled for gradient updates during backpropagation: (1) The weights and biases of the gating layer in the gating parallel axis attention module described in step one; (2) All parameters in the multi-scale feature adapter mentioned in step two, including convolutional branches, perceptron layers and residual scaling factors; (3) The drift bias term in the multicenter cosine classification head described in step three and scaling base parameters ; By using small sample data to drive the updates of the aforementioned few parameters, a new model adapted to the new working conditions of the target domain is obtained while alleviating model overfitting.

Images