Lightweight rotating machine fault diagnosis method based on SEFormer

Abstract

The invention discloses a lightweight rotating machine fault diagnosis method based on SEFormer. The method comprises the following steps: 1) data acquisition: acquiring vibration signals of key transmission parts (such as a bearing and a gear) of a rotating machine in various health states; 2) data preprocessing: segmenting the vibration signals into sample sets through sliding window sampling, and dividing the sample sets into a training set, a verification set and a test set; 3) model construction: designing and developing a separable multi-scale deep convolution SMDC and an efficient self-attention ESA, and constructing an SEFormer model; 4) model training, verification and evaluation: training the model on the training set and the verification set, and evaluating the model performance on the test set; and 5) fault diagnosis: performing fault diagnosis on the rotating machine by using the trained model. According to the method, a lightweight SEFormer model is constructed for rotating machine fault diagnosis, multi-scale features are extracted and integrated from different channel dimensions of the vibration signals, key fine-grained features of the vibration signals are captured in a global range, and the method has the advantages of robustness, generalization ability and lightweight.

Description

Technical Field

[0001] This invention belongs to the field of rotating machinery fault diagnosis technology, specifically a lightweight rotating machinery fault diagnosis method based on SEFormer. Background Technology

[0002] With the rapid development of modern mechanical systems, rotating machinery has become indispensable in intelligent equipment, and its safety has received considerable attention from academia and industry. Under high-intensity working conditions, critical transmission components of rotating machinery (such as bearings and gears) inevitably experience failures such as wear, corrosion, deformation, and fracture. Failed transmission components directly affect the operational reliability of rotating machinery, potentially leading to serious accidents, economic losses, and even personal injury or death. Therefore, conducting fault diagnosis and predictive maintenance for rotating machinery is of significant research value.

[0003] Traditional data-driven fault diagnosis methods rely on expert experience to manually extract effective fault features from signals, which has certain limitations. Deep learning technology, due to its powerful fault feature extraction capabilities and end-to-end diagnostic characteristics, has become a research hotspot in the field of fault diagnosis. In particular, methods based on convolutional neural networks (CNNs) have performed well in various fault diagnosis tasks. Although CNNs excel at extracting local features from short time series, they lack the ability to establish global dependencies in long time series. When noise interference exists in the acquired signal, relying solely on local features to identify effective fault information will face difficulties.

[0004] As a rising star in the fields of Natural Language Processing (NLP) and Computer Vision (CV), the Transformer can capture fine-grained features and establish temporal correlations from long-term sequences by evaluating the similarity between sequences. However, the Transformer lacks the local correlation extraction and spatial inductive bias capabilities of CNNs. Therefore, a large number of training samples are usually required to ensure performance effectiveness, which poses a challenge for fault diagnosis tasks. Furthermore, vibration signals are periodic and continuous, so local features cannot be ignored.

[0005] Recently, the CNN-Transformer collaborative method, combining CNNs and Transformers, has been explored to jointly capture local features and global dependencies in signals. However, the cross-channel convolution mechanism of CNNs and the self-attention computation of Transformers make the collaborative model too complex. This complexity leads to high computational costs and limited industrial applicability. Therefore, it is particularly important to design and develop simple, lightweight, and efficient convolution mechanisms and self-attention computation methods while maintaining the overall performance of the model.

[0006] Technical comparison with patent CN113505830B "Method, System, Equipment and Storage Medium for Fault Diagnosis of Rotating Machinery"

[0007] Patent CN113505830B uses a lightweight convolutional neural network to extract rotating machinery fault signals, obtain spatial feature information, and perform feature recognition on the spatial feature information. This patent constructs a lightweight SEFormer model for rotating machinery fault diagnosis, extracting and integrating multi-scale features from different channel dimensions of vibration signals, and capturing key fine-grained features of vibration signals from a global perspective. The two patents differ fundamentally in their technical approaches.

[0008] Patent CN113505830B establishes a lightweight convolutional neural network fault diagnosis model, which is based on the CNN architecture and represents an improvement in network structure. This patent designs and develops a separable multi-scale deep convolutional (SMDC) module and an efficient self-attention (ESA) module. Based on these two modules, a lightweight SEFormer model for rotating machinery fault diagnosis is constructed. This model is based on the CNN-Transformer architecture and represents the design and development of network structures. The two differ fundamentally at the algorithmic level.

[0009] Technical Comparison with Patent CN117113170A "A Lightweight Fault Diagnosis Method Based on Multi-Scale Information Fusion"

[0010] Patent CN117113170A performs continuous wavelet transform on the raw vibration data of the acquired rotating machinery to obtain a time-frequency image dataset with enhanced state information; the sample data is a two-dimensional time-frequency image. This patent, however, directly segments the raw vibration signal of the acquired rotating machinery into a dataset through sliding window sampling; the sample data is a one-dimensional time-domain vibration signal. The two patents represent fundamentally different methods of data preprocessing for vibration signals.

[0011] Patent CN117113170A describes a two-stream network structure, including an upper branch structure and a lower branch structure; this model belongs to the two-stream network architecture. This patent designs and develops a separable multi-scale deep convolutional (SMDC) module and an efficient self-attention (ESA) module. Based on these two modules, a lightweight SEFormer model is constructed for rotating machinery fault diagnosis; the model is based on the CNN-Transformer architecture. The network structures of the two are fundamentally different. Summary of the Invention

[0012] To address the aforementioned technical problems, this invention proposes a lightweight rotating machinery fault diagnosis method based on SEFormer. The proposed method has two main objectives: (1) to design and develop a simple, lightweight, and efficient convolution mechanism and self-attention calculation method; and (2) to construct a CNN-Transformer model that combines robustness, generalization ability, and lightweightness for rotating machinery fault diagnosis.

[0013] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0014] A lightweight rotating machinery fault diagnosis method based on SEFormer includes the following steps:

[0015] Step 1: Data Collection

[0016] Collect vibration signals from key transmission components of rotating machinery under various health conditions;

[0017] Install at least one sensor at different locations or in different directions near the rotating mechanical transmission component that needs to be diagnosed, and use data acquisition equipment to collect vibration signals under various health conditions;

[0018] Step 2, Data Preprocessing:

[0019] Vibration signals are segmented into sample sets using sliding window sampling, and then divided into training, validation, and test sets.

[0020] Step 3: Model Building

[0021] Design and develop a separable multi-scale deep convolutional model (SMDC) and an efficient self-attention model (ESA) to construct the SEFormer model.

[0022] Step 4: Model Training, Validation, and Evaluation

[0023] Train the model on the training and validation sets, and evaluate the model performance on the test set;

[0024] Step 5: Fault Diagnosis

[0025] Use a trained model to diagnose faults in rotating machinery.

[0026] Furthermore, in the second step of the preprocessing process, to avoid test leakage, there is no overlap between adjacent windows during sliding window sampling, and a certain interval is maintained; each sample contains 1024 signal points; all samples are standardized by mean-std.

[0027] Furthermore, in step three, the SEFormer model consists of multiple feature extraction layers and an output layer connected in sequence, and the number of feature extraction layers can be adjusted according to the needs of different tasks.

[0028] In the feature extraction layer;

[0029] First, separable multi-scale depth convolution (SMDC) is used to extract and integrate multi-scale features from different channel dimensions of vibration signals;

[0030] Secondly, the key fine-grained features of vibration signals are captured from a global scale using efficient self-attention ESA;

[0031] Then, the risk of overfitting is reduced by residual connection Add, and batch normalization BN is used to stabilize the feature distribution;

[0032] Next, a feedforward network (FFN) is used to perform a nonlinear transformation on the features;

[0033] Finally, the extracted features are output through residual connection Add and batch normalized BN;

[0034] The separable multi-scale depth convolution SMDC;

[0035] First, we use parallel multi-scale depth convolution with different kernel sizes to extract multi-local receptive field features of vibration signals;

[0036] Then, the multi-scale features are concatenated along the channel dimension using Concat;

[0037] Next, pointwise convolution is used to integrate feature information and capture the correlation between channels;

[0038] Finally, batch normalized BN and Gaussian error linear unit GELU are used to stabilize the feature distribution and perform nonlinear mapping, as shown in the following equation:

[0039]

[0040] in, Let k represent the input, where C1 and L1 represent the channel dimension and time dimension of the input, respectively. l Indicates the size of the depthwise convolution kernel. Depth convolution kernel k l The weight, Depth convolution kernel k l The output, This represents the output of a multi-scale depthwise convolution. This represents the weights of the pointwise convolution kernel. The output is represented by n, which represents the number of convolutional kernels of different depths and sizes, and L2 represents the time dimension of the output.

[0041] The highly efficient self-attention ESA;

[0042] First, to expand the representational power of each feature and reduce computation, three parallel depthwise separable convolutions DSConv are used to generate self-attention input features: query matrix Q, key matrix K, and value matrix V.

[0043] Secondly, efficient attention is used to transmit information about global features, while reducing the computational cost of self-attention.

[0044] Then, Softmax is used as a normalization operation suitable for the form of Efficientattention. Softmax normalization is performed on matrix Q in the time dimension and on matrix K in the channel dimension.

[0045] Finally, the attention weights are calculated using matrix transpose and matrix multiplication, as shown in the following equation:

[0046] Efficient attention(Q,K,V)=Softmax L (Q)·[(Softmax C (K)) T ·V] (4)

[0047] in, and Let L and C represent the query matrix, key matrix, and value matrix, respectively, and let C represent the time dimension and channel dimension, respectively. T · and · represent matrix transpose and matrix multiplication, respectively. Softmax L (·) and Softmax C (·) represent Softmax normalization in the time dimension and channel dimension, respectively;

[0048] The residual connection Add is shown in the following formula:

[0049] x'=f(x)+x (5)

[0050] The batch normalized BN is as follows:

[0051]

[0052] Among them, h l ={h l(1) ,…,h l(N)} represents the input feature map of the l-th layer with a batch size of N, h l(n) ={h1 l (n) ,…,h k l(n)}, y j l(n) Let u represent the j-th output feature map of the l-th layer. j σ j 2 They represent h respectively j l The mean and variance are given by ε, where ε is a very small constant used to prevent invalid calculations when the variance is 0. These represent the scale and translation parameters that need to be learned;

[0053] The Gaussian error linear unit (GELU) is shown in the following equation:

[0054]

[0055] The feedforward network FFN consists of two layers of linear transformation and an intermediate nonlinear activation, as shown in the following equation:

[0056] FFN(x)=W2σ(W1x+b1)+b2 (11)

[0057] Where x represents the input, W1 and W2 represent the weights of the first and second linear transformations respectively, b1 and b2 represent the biases of the first and second linear transformations respectively, and σ represents the intermediate nonlinear activation function.

[0058] In the output layer;

[0059] First, we use global average pooling (GAP) to reduce the dimensionality of features in the time dimension.

[0060] Secondly, pointwise convolution PConv is used to integrate features in the channel dimension;

[0061] Then, batch normalized BN and Gaussian error linear unit GELU are used to stabilize the feature distribution and perform nonlinear mapping;

[0062] Finally, the high-dimensional features are mapped to the classification dimension of health status through a fully connected (FC) approach.

[0063] Furthermore, the SEFormer model is trained using a multi-class cross-entropy loss function to calculate the training loss, the AdamW optimization algorithm is used to update the model parameters, and the learning rate is dynamically adjusted using an adaptive decay strategy based on the loss of the validation set.

[0064] Compared with the prior art, the advantages of the present invention are as follows:

[0065] (1) A separable multi-scale depth convolution SMDC was designed to extract and integrate multi-scale features from different channel dimensions of vibration signals;

[0066] (2) A highly efficient self-attention ESA was developed to capture key fine-grained features of vibration signals from a global perspective;

[0067] (3) Based on separable multi-scale deep convolution SMDC and efficient self-attention ESA, a robust, generalizable and lightweight SEFormer model was constructed for rotating machinery fault diagnosis. Attached Figure Description

[0068] Figure 1This is a flowchart of the method of the present invention;

[0069] Figure 2 This is a schematic diagram of the SEFormer model of the method of the present invention;

[0070] Figure 3 This is a schematic diagram of the structure of the separable multi-scale depth convolutional SMDC of the present invention.

[0071] Figure 4 This is a schematic diagram of the structure of the efficient self-attention ESA of the method of the present invention. Detailed Implementation

[0072] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0073] In this embodiment, as Figure 1 As shown, a lightweight rotating machinery fault diagnosis method based on SEFormer includes the following steps: Step 1, Data Acquisition: Acquire vibration signals of key transmission components (such as bearings and gears) of rotating machinery under various health conditions; Step 2, Data Preprocessing: Segment the vibration signals into sample sets using sliding window sampling, and further divide them into training, validation, and test sets; Step 3, Model Building: Design and develop separable multi-scale deep convolutional model (SMDC) and efficient self-attention model (ESA) to construct the SEFormer model; Step 4, Model Training, Validation, and Evaluation: Train the model on the training and validation sets, and evaluate the model performance on the test set; Step 5, Fault Diagnosis: Use the trained model to diagnose faults in the rotating machinery.

[0074] In step one, one or more sensors are installed at different locations or in different directions near the rotating mechanical transmission components (such as bearings and gears) that need to be diagnosed, and vibration signals under various health conditions are collected using data acquisition equipment.

[0075] In step two, to avoid test leakage, there is no overlap between adjacent windows during sliding window sampling, and a certain interval is maintained; each sample contains 1024 signal points; all samples are standardized using mean-std.

[0076] In this embodiment, accelerometers are installed in the X and Y directions on the planetary gearbox housing to collect multi-sensor vibration signals from the bearings and gears, respectively. During the data acquisition process, the motor speed is 1800 r / min, and the sampling frequency is 20480 Hz. The planetary gearbox has four types of bearing faults and four types of gear faults. Vibration signals for nine healthy states, including the normal state, were collected, as shown in Table 1 below:

[0077]

[0078] In this embodiment, the vibration signal for each health state is segmented into 1200 samples using a sliding window sampling method. Of these, 500 samples are used for training, 300 for validation, and 400 for testing. To evaluate the model's noise robustness, two types of noise are added sequentially to the test samples, as shown in the following formula:

[0079] S i =(S i +α)×β (1)

[0080] Among them, S i Let represent the i-th signal point of the test sample. α ~ N(0,σ) and β ~ N(1,σ) represent additive Gaussian noise and scaled Gaussian noise, respectively. σ represents the variance of the Gaussian distribution; the larger the value of σ, the more significant the noise. To simulate the uncertainty of the noise, each test sample has a 50% probability of being randomly added with any type of noise.

[0081] In this embodiment, as Figure 2 As shown, in step three, the SEFormer model consists of three feature extraction layers and one output layer connected in sequence;

[0082] In the feature extraction layer, firstly, separable multi-scale deep convolutional neural networks (SMDC) are used to extract and integrate multi-scale features from different channel dimensions of the vibration signal. Secondly, efficient self-attention networks (ESA) are used to capture key fine-grained features of the vibration signal globally. Then, residual connections (Add) are used to reduce the risk of overfitting, and batch normalization (BN) is employed to stabilize the feature distribution. Next, a feedforward network (FFN) is used to perform a nonlinear transformation on the features. Finally, the extracted features are output through residual connections (Add) and batch normalization (BN).

[0083] like Figure 3 As shown, the separable multi-scale deep convolutional method (SMDC) first extracts multi-local receptive field features of the vibration signal using parallel multi-scale deep convolutions with different kernel sizes. Then, the multi-scale features are concatenated along the channel dimension. Next, pointwise convolution is used to integrate the feature information and capture the correlation between channels. Finally, batch normalized normalization (BN) and Gaussian error linear units (GELUs) are used to stabilize the feature distribution and perform nonlinear mapping, as shown in the following equation:

[0084]

[0085] in, Let k represent the input, where C1 and L1 represent the channel dimension and time dimension of the input, respectively. l Indicates the size of the depthwise convolution kernel. Depth convolution kernel k l The weight, Depth convolution kernel k l The output, This represents the output of a multi-scale depthwise convolution. This represents the weights of the pointwise convolution kernel. The output is represented by n, which represents the number of convolutional kernels of different depths and sizes, and L2 represents the time dimension of the output.

[0086] like Figure 4 As shown, the efficient self-attention ESA firstly generates the input features for self-attention using three parallel depthwise separable convolutions (DSConv) to expand the representational power of each feature and reduce computation: the query matrix Q, the key matrix K, and the value matrix V. Secondly, efficient attention is used to transfer information about the global features while reducing the computational cost of self-attention. Then, Softmax is used as a normalization operation suitable for the efficient attention form, applying Softmax normalization to matrix Q in the time dimension and to matrix K in the channel dimension. Finally, the attention weights are calculated through matrix transpose and matrix multiplication, as shown in the following equation:

[0087] Efficient attention(Q,K,V)=Softmax L (Q)·[(Softmax C (K)) T ·V] (5)

[0088] in, and Let L and C represent the query matrix, key matrix, and value matrix, respectively, and let C represent the time dimension and channel dimension, respectively. T · and · represent matrix transpose and matrix multiplication, respectively. Softmax L (·) and Softmax C (·) represent Softmax normalization in the time dimension and channel dimension, respectively;

[0089] The residual connection Add is shown in the following formula:

[0090] x'=f(x)+x (6)

[0091] The batch normalized BN is as follows:

[0092]

[0093] Among them, h l ={h l(1) ,…,h l(N)} represents the input feature map of the l-th layer with a batch size of N, hl(n) ={h1 l (n) ,…,h k l(n)}, y j l(n) Let u represent the j-th output feature map of the l-th layer. j σ j 2 They represent h respectively j l The mean and variance are given by ε, where ε is a very small constant used to prevent invalid calculations when the variance is 0. These represent the scale and translation parameters that need to be learned;

[0094] The Gaussian error linear unit (GELU) is shown in the following equation:

[0095]

[0096] The feedforward network FFN consists of two layers of linear transformation and an intermediate nonlinear activation, as shown in the following equation:

[0097] FFN(x)=W2σ(W1x+b1)+b2 (12)

[0098] Where x represents the input, W1 and W2 represent the weights of the first and second linear transformations respectively, b1 and b2 represent the biases of the first and second linear transformations respectively, and σ represents the intermediate nonlinear activation function.

[0099] In the output layer, firstly, global average pooling (GAP) is used to reduce the dimensionality of features in the time dimension. Secondly, pointwise convolutional polynomials (PConv) are used to integrate features in the channel dimension. Then, batch normalization (BN) and Gaussian error linear units (GELU) are employed to stabilize the feature distribution and perform non-linear mapping. Finally, a fully connected (FC) layer maps the high-dimensional features to the classification dimension of health status.

[0100] In this embodiment, the detailed parameter configuration of the SEFormer model is shown in Table 2 below:

[0101]

[0102]

[0103] Note: m represents the number of sensor channels for signal acquisition, d represents the dimension of the output channels, and k l The SMDC kernel size is represented by s, the stride by s, the padding by p, and the scaling factor by r for FFN. g This indicates the output time dimension of GAP, k represents the convolution kernel size of PConv, and d lThis indicates the time dimension of the FC output, where C represents the number of health status categories.

[0104] In this embodiment, m = 2 and C = 9.

[0105] In this embodiment, in step four, the SEFormer model is trained using a multi-class cross-entropy loss function to calculate the training loss, the AdamW optimization algorithm is used to update the model parameters, and the learning rate is dynamically adjusted using an adaptive decay strategy based on the loss of the validation set.

[0106] In this embodiment, the training hyperparameters of the SEFormer model are configured as follows: the initial learning rate of the AdamW optimization algorithm is 0.001, the batch size is 64, and the number of iterations is 100.

[0107] In this embodiment, seven existing advanced models were selected for comparative analysis with the SEFormer model, including three end-to-end fault diagnosis models based on CNN-Transformer: LiConvFormer, Convformer-NSE, and CLFormer; and four popular CNN models: MobileNetV2, MobileNet, MK-ResCNN, and ResNet18. It is worth noting that, for a fair comparison, all models used the same input signal length, training strategy, and training hyperparameter configuration. To reduce the impact of randomness, each experiment was repeated five times, and the mean and standard deviation (Std) of the diagnostic accuracy were used as the final experimental results for analysis. Furthermore, the number of parameters (Params) and the number of floating-point operations (FLOPs) were used as evaluation metrics for model computational complexity. The diagnostic accuracy and computational complexity of all models under different noise levels are shown in Table 3 below.

[0108]

[0109]

[0110] As shown in the table, without any added noise, MK-ResCNN and ResNet18 have slightly higher average accuracies than SEFormer. SEFormer's robustness advantage becomes significant as the noise level increases. SEFormer performs excellently at all three noise levels. Specifically, at σ = 0.2 and σ = 0.6, SEFormer achieves the highest average accuracy (96.63% and 72.72%), with smaller standard deviations (0.89% and 0.59%). When σ = 0.4, SEFormer's average accuracy is slightly lower than LiConvFormer, but its standard deviation is smaller. Overall, SEFormer is comparable to LiConvFormer in diagnostic performance, but has a significant advantage in model computational complexity. Although Convformer-NSE and CLFormer have relatively low model computational complexity, their diagnostic performance lags significantly behind other models.

[0111] As can be seen from the above embodiments, the present invention is a lightweight rotating machinery fault diagnosis method based on SEFormer. Compared with the prior art, the advantages of the present invention are: (1) a separable multi-scale deep convolution SMDC is designed to extract and integrate multi-scale features from different channel dimensions of vibration signals; (2) an efficient self-attention ESA is developed to capture key fine-grained features of vibration signals from a global perspective; (3) based on separable multi-scale deep convolution SMDC and efficient self-attention ESA, a SEFormer model with robustness, generalization ability and lightweight is constructed for rotating machinery fault diagnosis.

[0112] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A lightweight rotating machinery fault diagnosis method based on SEFormer, characterized by: The steps include: Step 1: Data collection: Collect vibration signals of key transmission components of rotating machinery under various health conditions; Install at least one sensor at different positions or directions near the rotating machinery transmission parts that need to be diagnosed, and use data acquisition equipment to collect vibration signals under various health conditions; Step 2: Data preprocessing: The vibration signal is divided into sample sets through sliding window sampling, and then divided into training set, validation set and test set; Step 3: Model building: Design and develop separable multi-scale deep convolution SMDC and efficient self-attention ESA, and build SEFormer model; Step 4: Model training, verification and evaluation: Train the model on the training set and validation set, and evaluate the model performance on the test set; Step 5: Fault diagnosis: Use the trained model to perform fault diagnosis on rotating machinery.

2. The SEFormer-based lightweight rotating machinery fault diagnosis method according to claim 1, characterized in that: In the second step, the preprocessing process is to avoid test leakage. When the sliding window is sampled, there is no overlap between adjacent windows and a certain interval is maintained. Each sample contains 1024 signal points. All samples are standardized by mean-standard deviation.

3. The SEFormer-based lightweight rotating machinery fault diagnosis method according to claim 1, characterized in that: In step 3, the SEFormer model is composed of multiple feature extraction layers and an output layer connected in sequence, and the number of feature extraction layers is adjusted according to different task requirements; In the feature extraction layer; Firstly, separable multi-scale deep convolution (SMDC) is used to extract and integrate multi-scale features from different channel dimensions of vibration signals. Secondly, efficient self-attention (ESA) is used to capture the key fine-grained features of vibration signals from a global perspective. Then, residual connection (Add) is used to reduce the risk of overfitting, and batch normalization (BN) is used to stabilize the feature distribution. Next, the feedforward network FFN is used to perform nonlinear transformation on the features; Finally, the extracted features are output through residual connection Add and batch normalization BN; The separable multi-scale deep convolution SMDC; Firstly, parallel multi-scale deep convolution with different kernel sizes is used to extract the multi-local receptive field features of the vibration signal. Then, the multi-scale features are concatenated along the channel dimension. Next, point-wise convolution is used to integrate feature information and capture the correlation between channels; Finally, batch normalization BN and Gaussian error linear unit GELU are used to stabilize feature distribution and perform nonlinear mapping, as shown in the following formula: in, represents the input, C1 and L1 represent the channel dimension and time dimension of the input respectively, k l represents the size of the depth convolution kernel, represents the depth convolution kernel k l The weight of represents the depth convolution kernel k l The output, represents the output of multi-scale depth convolution, represents the weight of the point-by-point convolution kernel, represents the output, n represents the number of different depth convolution kernel sizes, and L2 represents the time dimension of the output; The efficient self-attention ESA; First, in order to expand the representation capacity of each feature and reduce computation, three parallel depth-wise separable convolutions DSConv are used to generate the input features of self-attention: query matrix Q, key matrix K, and value matrix V; Secondly, efficient attention is used to transfer global feature information while reducing the cost of self-attention calculation; Then, Softmax is used as a normalization operation suitable for efficient attention. Softmax normalization is performed on the matrix Q in the time dimension and on the matrix K in the channel dimension. Finally, the attention weight is calculated through matrix transposition and matrix multiplication operations, as shown in the following formula: Efficient attention(Q,K,V)=Softmax L (Q)·[(Softmax C (K)) T ·V] (4) in, and denote the query matrix, key matrix, and value matrix respectively, L and C denote the time dimension and channel dimension respectively, (·) T and represent matrix transposition and matrix multiplication respectively, Softmax L (·) and Softmax C (·) represents the Softmax normalization in the time dimension and channel dimension respectively; The residual connection Add is shown in the following formula: x'=f(x)+x (5) The batch normalization BN is shown in the following formula: Among them, h l ={h l(1) ,…,h l(N) } represents the input feature map of the lth layer with a batch size of N, h l(n) ={h1 l(n) ,…,h k l (n) },y j l(n) represents the j-th output feature map of the l-th layer, u j , σ j 2 Respectively represent h j l The mean and variance of , ε represents a small constant used to prevent invalid calculations when the variance is 0, They represent the scale and translation parameters that need to be learned respectively; The Gaussian error linear unit GELU is shown in the following formula: The feedforward network FFN includes two layers of linear transformation and a nonlinear activation in the middle, as shown in the following formula: FFN(x) = W2σ(W1x+b1)+b2 (11) Among them, x represents the input, W1 and W2 represent the weights of the linear transformation of the first layer and the second layer respectively, b1 and b2 represent the bias of the linear transformation of the first layer and the second layer respectively, and σ represents the intermediate nonlinear activation function; In the output layer; First, global average pooling (GAP) is used to reduce the feature dimension in the time dimension; Secondly, point-by-point convolution PConv is used to integrate features in the channel dimension; Then, batch normalization BN and Gaussian error linear unit GELU are used to stabilize feature distribution and perform nonlinear mapping; Finally, the high-dimensional features are mapped to the categorical dimension of health status through fully connected FC.

4. The SEFormer-based lightweight rotating machinery fault diagnosis method according to claim 1, characterized in that: In step 4, the training of the SEFormer model uses a multi-classification cross entropy loss function to calculate the training loss, uses the AdamW optimization algorithm to update the model parameters, and uses an adaptive decay strategy to dynamically adjust the learning rate according to the loss of the validation set.

Images