ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting

Abstract

The present invention discloses a ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting, which is specifically implemented in the following steps: step 1, obtaining original vibration fault signals of different fault types from a public data set; step 2, preprocessing the original vibration fault signal obtained in step 1; step 3, establishing a CpcsViT model for mechanical equipment fault detection; step 4, inputting the preprocessed original vibration fault signal in step 2 into the CpcsViT model established in step 3, setting training parameters, and training the CpcsViT model; step 5, using the trained model for fault diagnosis. The present invention solves the problem of low detection accuracy in mechanical equipment fault detection tasks existing in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of fault diagnosis methods, and relates to a ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting. Background Art

[0002] Existing mechanical equipment fault detection methods usually rely on traditional convolutional neural networks (CNNs) or machine learning methods based on handcrafted features. With the development of deep learning technology, the Vision Transformer (ViT) model has gradually become an effective feature extraction tool due to its successful application in image classification tasks.

[0003] Although the ViT model has strong expressiveness when processing image information, how to effectively capture the relationship between local area and global features in mechanical equipment fault detection tasks remains a challenge. Existing ViT models often ignore the importance of multi-scale information and regional features in bearing fault images, resulting in unsatisfactory performance and low accuracy in mechanical equipment fault detection tasks. Summary of the invention

[0004] The purpose of the present invention is to provide a ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting, which solves the problem of low detection accuracy in mechanical equipment fault detection tasks existing in the prior art.

[0005] The technical solution adopted by the present invention is to combine the ViT cross-domain fault diagnosis method with adaptive spatial attention and patch feature weighting, which is specifically implemented in the following steps:

[0006] Step 1, obtaining original vibration fault signals of different fault types from a public data set;

[0007] Step 2, preprocessing the original vibration fault signal obtained in step 1;

[0008] Step 3, establishing the CpcsViT model for mechanical equipment fault detection;

[0009] Step 4, inputting the original vibration fault signal preprocessed in step 2 into the CpcsViT model established in step 3, setting training parameters, and training the CpcsViT model;

[0010] Step 5: Use the trained model to perform fault diagnosis.

[0011] The present invention is also characterized in that:

[0012] Step 2 is as follows:

[0013] The original vibration fault signal obtained in step 1 is segmented, and then the segmented signal is converted into a color three-dimensional fault signal image by applying short-time Fourier transform.

[0014] When segmenting the original vibration fault signal, the segmentation is performed according to the time step = 2048 and the overlap rate = 0.5.

[0015] The CpcsViT model for mechanical equipment fault detection in step 3 includes a feature enhancement module and a feature extraction module connected in sequence;

[0016] The feature enhancement module includes a hole convolution module, a spatial attention module, a pyramid pooling module, and a batch normalization layer connected in sequence;

[0017] The feature extraction module includes an embedding layer, a dropout layer, a channel attention block, a ViT encoder, a channel attention block, a ViT encoder, and a linear layer connected in sequence;

[0018] Each channel attention block and the subsequent ViT encoder form a stage, that is, the feature extraction module consists of two stages.

[0019] The first ViT encoder includes 4 layers of 8-head self-attention layers; the second ViT encoder includes 5 layers of 16-head self-attention layers.

[0020] In step 4, the original vibration fault signal preprocessed in step 2 is input into the CpcsViT model established in step 3. Assume that the image obtained after preprocessing in step 2 is , where B is the batch size, C is the number of channels, H and W are the height and width of the image, then the processing process of the CpcsViT model is as follows:

[0021] Step 4.1, The input is fed into the dilated convolution module, and the input is processed by dilated convolution according to the following formula:

[0022] in, is the feature map output by the hole convolution module, It is the dilated convolution processing;

[0023] Step 4.2, then Input to the spatial attention module to obtain the output feature map of the spatial attention module ;

[0024] Step 4.3, then Input to the pyramid pooling module to get the output feature map of the pyramid pooling module ;

[0025] Step 4.4, Batch normalization is performed through the batch normalization layer, specifically:

[0026]

[0027] in, For batch normalization, To obtain the feature map after batch normalization, ;

[0028] Step 4.5: Feature map obtained after processing by feature enhancement module Input into the feature extraction module, and in the embedding layer Perform embedding operation to obtain , specifically:

[0029] in, , N is the number of patches, D is the embedding dimension, For embedded operations;

[0030] Then in Add a class token to represent the final classification information, and we get , specifically:

[0031]

[0032] in, For splicing operation, are learnable parameters, which are ultimately used for classification tasks;

[0033] Give again Adding position embedding gives , specifically:

[0034]

[0035] in, Encode the embedded position information;

[0036] Step 4.6, Pass it into the Dropout layer to prevent overfitting, and get , ;

[0037] Step 4.7, As the input of the channel attention block, the feature is obtained after the channel attention weighting by the channel attention block ;

[0038] Step 4.8, the features As the input of the first ViT encoder, it is processed by the ViT encoder and output ;

[0039] Step 4.9, Enter the second stage and get the features after passing through the second ViT encoder ;

[0040] Step 4.10, pass the features after the second ViT encoder Take the class token for classification, specifically:

[0041]

[0042] in, To extract the class token with global information after training;

[0043] Step 4.12, then Input the linear layer and use the softmax function for probability matching classification, specifically:

[0044]

[0045] in, is the normalization function, is the weight matrix of the classification head, Weight bias vector for the classification head.

[0046] In step 4.2, the spatial attention module first Perform pooling operations, namely average pooling and maximum pooling operations, specifically:

[0047]

[0048] in, Get the result for the average pooling operation; The result obtained for the maximum pooling operation;

[0049] Then convolution is performed on the results obtained from the average pooling and maximum pooling operations to calculate the spatial attention to obtain the attention map , specifically:

[0050]

[0051] Among them, σ represents the Sigmoid activation function;

[0052] The attention map then weights each position to obtain the weighted output features of the spatial attention module: .

[0053] In step 4.3, the pyramid pooling module first Use the pooling window as , 2 Perform average pooling operation, specifically:

[0054]

[0055] in, For Use the pooling window as , 2 Perform an average pooling operation to obtain a collection of results. For The pooling window is The average pooling operation, For The pooling window is The average pooling operation, For The pooling window is Average pooling operation;

[0056] Then, the results of the average pooling operation are upsampled separately, specifically:

[0057]

[0058] in, is the result set of the upsampling operation, For conduct , For conduct , For conduct ;

[0059] Then concatenate the operations to get Specifically:

[0060] ;

[0061] in, For splicing operation.

[0062] The channel attention block in step 4.7 is The processing process is:

[0063] Perform global pooling:

[0064] and

[0065] in, is average pooling, is the maximum pooling, N is the number of patches;

[0066] Calculating channel attention :

[0067] in and They are the fully connected layers, is the activation function;

[0068] According to the channel attention Weighted to get weighted features :

[0069] Then the weighted features are Dropout processed and normalized, and finally the processed features are residually connected with the original features, specifically:

[0070] Dropout processing :

[0071]

[0072] Layer Normalization :

[0073]

[0074] Will and Perform residual connection to obtain :

[0075] ;

[0076] The processing of the ViT encoder in step 4.8 is:

[0077] Perform multi-head attention calculation:

[0078]

[0079] in, is query, key, value, is the dimension of the key, is the normalization function;

[0080] Then, according to the multi-head attention Then perform residual connection and layer normalization to obtain features :

[0081]

[0082] in Normalize the layer;

[0083] Then After being processed by the feedforward network, we get :

[0084]

[0085] It is a feed-forward network, which consists of two fully connected layers, ReLU activation function, and layer normalization;

[0086] The obtained Then input Perform residual connection and layer normalization to obtain features :

[0087] ;

[0088] In step 4.9, Enter the second stage. The processing process of the second stage is as follows:

[0089] The second channel attention block is Processing to obtain :

[0090]

[0091] in, represents the second channel attention block;

[0092] Then the second ViT encoder processes it to get :

[0093]

[0094] in, Represents the second ViT encoder.

[0095] When training the CpcsViT model in step 4, use the Adam optimizer, Onecycle scheduler, batch_size of 32, and learning rate of 0.001.

[0096] The beneficial effects of the present invention are:

[0097] The present invention strengthens the processed fault signal image features by introducing spatial attention, dilated convolution and pyramid pooling. Specifically: first, dilated convolution is used to enhance the local features of the image, so that the model can better capture the subtle changes of the fault; then, the spatial attention mechanism further optimizes the spatial features of the image, and highlights the key area information by weighting different areas in the image; then, spatial pyramid pooling is used to extract image features of different scales to provide richer multi-scale information, which is combined with the original features after upsampling, further enhancing the feature expression ability; after feature enhancement, the processed signal image is sent to the embedding The channel attention block adaptively weights the features of each patch, further improving the expressiveness of image features. The channel attention mechanism learns the global information of each patch and assigns different weights to different feature channels, thereby dynamically adjusting the weighting method of patch features. Through this weighting mechanism, more attention can be paid to the key patch features related to the fault, the representation of image features is optimized, and the performance of mechanical equipment fault detection is significantly improved. It can more accurately identify and distinguish subtle differences in faults, thereby improving the accuracy and robustness of fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 It is a structural diagram of the CpcsViT model in the ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting of the present invention;

[0099] Figure 2 It is a working principle diagram of the spatial attention module in the CpcsViT model of the present invention;

[0100] Figure 3 It is a structural diagram of the channel attention block in the CpcsViT model of the present invention;

[0101] Figure 4 It is a working principle diagram of the channel attention module in the channel attention block in the CpcsViT model of the present invention;

[0102] Figure 5 It is a structural diagram of the ViT encoder in the CpcsViT model of the present invention. DETAILED DESCRIPTION

[0103] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0104] Example 1

[0105] The present invention combines the ViT cross-domain fault diagnosis method with adaptive spatial attention and patch feature weighting, and is specifically implemented according to the following steps:

[0106] Step 1, obtaining original vibration fault signals of different fault types from a public data set;

[0107] Step 2, preprocessing the original vibration fault signal obtained in step 1;

[0108] Step 3, establishing the CpcsViT model for mechanical equipment fault detection;

[0109] Step 4, inputting the original vibration fault signal preprocessed in step 2 into the CpcsViT model established in step 3, setting training parameters, and training the CpcsViT model;

[0110] Step 5: Use the trained model to perform fault diagnosis.

[0111] Example 2

[0112] The present invention combines the ViT cross-domain fault diagnosis method with adaptive spatial attention and patch feature weighting, and is specifically implemented according to the following steps:

[0113] Step 1, obtaining original vibration fault signals of different fault types from a public data set;

[0114] Step 2, preprocessing the original vibration fault signal obtained in step 1;

[0115] Step 3: Establish the CpcsViT model for mechanical equipment fault detection. Its structure is as follows: Figure 1 As shown, it includes a feature enhancement module and a feature extraction module connected in sequence;

[0116] The feature enhancement module includes a hole convolution module, a spatial attention module, a pyramid pooling module (Spatial Pyramid Pooling, SPP), and a batch normalization layer connected in sequence;

[0117] The feature extraction module includes an embedding layer, a dropout layer, a channel attention block, a ViT encoder, a channel attention block, a ViT encoder, and a linear layer connected in sequence;

[0118] Each channel attention block and the subsequent ViT encoder form a stage, that is, the feature extraction module consists of two stages.

[0119] The first ViT encoder includes 4 layers of 8-head self-attention layers; the second ViT encoder includes 5 layers of 16-head self-attention layers;

[0120] Step 4, inputting the original vibration fault signal preprocessed in step 2 into the CpcsViT model established in step 3, setting training parameters, and training the CpcsViT model;

[0121] Step 5: Use the trained model to perform fault diagnosis.

[0122] The dilated convolution module of the present invention expands the receptive field through dilated convolution, and can capture a wider range of local features without increasing the amount of calculation. This operation is particularly important for small feature changes in fault diagnosis, and helps to enhance the perception of local details.

[0123] Spatial attention module Spatial attention can focus on important areas in the image and suppress the influence of irrelevant areas by weighting the spatial dimension of the input feature map, thereby improving the accuracy of fault diagnosis. This module further improves the model's ability to pay attention to different spatial regions by processing the average pooling and maximum pooling results at the same time.

[0124] The Spatial Pyramid Pooling (SPP) module uses multi-scale pooling operations to capture multi-scale features of images through pooling layers of different scales (such as 1x1, 2x2, and 4x4 pooling). After pooling, features of different scales are integrated through upsampling, thereby achieving richer feature fusion in spatial scale and providing a more comprehensive feature representation.

[0125] The Channel Attention Block combines the channel attention mechanism in the channel attention module with Dropout and LayerNorm to form a complete feature enhancement process, which specifically includes the following parts:

[0126] The channel attention mechanism helps the model better highlight the key features related to the fault by dynamically weighting the feature dimensions within each patch (i.e., each channel of the embedding). The weight coefficient of each patch is calculated by the attention weight after global pooling. Specifically, the feature channels of each patch are first globally pooled, and the feature representations of average pooling and maximum pooling are calculated respectively. Then, these pooling results are passed through the fully connected layer to generate channel-level weight coefficients, which are normalized by the sigmoid function, and finally the features of each patch are weighted. In this way, the channel attention mechanism can adjust the weights of each feature channel at the patch level, so that the model can more accurately focus on the key patch features that contribute significantly to the classification task, thereby optimizing the expression of image features and improving the accuracy of fault detection.

[0127] Dropout: To avoid overfitting, the Dropout layer randomly discards weighted features to enhance the generalization ability of the model.

[0128] Layer Normalization: The LayerNorm layer is used to further stabilize the training process and ensure that the output of the model force block is fused with the original input through the residual connection, so that the model can retain the information of the original features and introduce the features of channel attention enhancement to achieve information fusion and enhancement.

[0129] The feature extraction module uses multiple ViT encoders, each of which consists of multiple self-attention layers, with different head and layer configurations, specifically:

[0130] The first ViT encoder is an 8-head 4-layer encoder: this configuration enables the model to learn representations of multiple feature subspaces in parallel, and extract features layer by layer through 4 layers of self-attention layers to capture local and global information in the image. For fault detection, the low-head attention mechanism can focus on more obvious, local fault features, ensuring that the model accurately recognizes basic features.

[0131] The second ViT encoder is a 16-head, 5-layer encoder: this configuration enhances the model's expressiveness, with more heads enabling the model to capture features of different scales, directions, and contextual relationships in more subspaces, while the increased number of layers helps the model extract deeper complex patterns in the image. In the bearing fault detection task, this configuration can effectively identify small, imperceptible fault types.

[0132] The self-attention mechanism of the ViT encoder effectively captures the long-range dependencies in the image by learning the relationship between image patches, and improves the training efficiency and expressiveness of the model through parallelization.

[0133] In this way, channel attention helps the model focus on important feature dimensions within each patch, while the self-attention in the encoder calculates the relationship between each patch and other patches based on these weighted features, and generates prediction results through the classification head, thereby completing the information aggregation and classification tasks.

[0134] Example 3

[0135] Based on Example 2, step 2 is specifically as follows:

[0136] The original vibration fault signal obtained in step 1 is segmented, and then the segmented signal is converted into a color three-dimensional fault signal image by applying short-time Fourier transform.

[0137] When segmenting the original vibration fault signal, the segmentation is performed according to the time step = 2048 and the overlap rate = 0.5.

[0138] Example 4

[0139] On the basis of Example 3, the original vibration fault signal preprocessed in step 2 in step 4 is input into the CpcsViT model established in step 3. Assume that the image obtained after preprocessing in step 2 is , where B is the batch size, C is the number of channels, H and W are the height and width of the image, then the processing process of the CpcsViT model is as follows:

[0140] Step 4.1, The input is fed into the dilated convolution module, and the input is processed by dilated convolution according to the following formula:

[0141] in, is the feature map output by the hole convolution module, It is the dilated convolution processing;

[0142] Step 4.2, then Input to the spatial attention module, such as Figure 2 As shown in Figure 2, a spatial weighting coefficient is generated to enhance the relevant area, and the output feature map of the spatial attention module is obtained. , specifically:

[0143] The spatial attention module first Perform pooling operations, namely average pooling and maximum pooling operations, specifically:

[0144] in, Get the result for the average pooling operation; The result obtained for the maximum pooling operation;

[0145] Then convolution is performed on the results obtained from the average pooling and maximum pooling operations to calculate the spatial attention to obtain the attention map , specifically:

[0146]

[0147] Among them, σ represents the Sigmoid activation function;

[0148] The attention map then weights each position to obtain the weighted output features of the spatial attention module: .

[0149] Step 4.3, then Input to the pyramid pooling module to get the output feature map of the pyramid pooling module , specifically:

[0150] The pyramid pooling module first Use the pooling window as , 2 Perform average pooling operation, specifically:

[0151]

[0152] in, For Use the pooling window as , 2 Perform an average pooling operation to obtain a collection of results. For The pooling window is The average pooling operation, For The pooling window is The average pooling operation, For The pooling window is Average pooling operation;

[0153] Then, the results of the average pooling operation are upsampled separately, specifically:

[0154]

[0155] in, is the result set of the upsampling operation, For conduct , For conduct , For conduct ;

[0156] Then concatenate the operations to get Specifically:

[0157] ;

[0158] in, For splicing operation.

[0159] Step 4.4, Batch normalization is performed through the batch normalization layer, specifically:

[0160]

[0161] in, For batch normalization, To obtain the feature map after batch normalization, ,The purpose of batch normalization is to normalize the input of each layer, keeping the mean to 0 and the variance to 1 to speed up training;

[0162] Step 4.5: Feature map obtained after processing by feature enhancement module Input into the feature extraction module, and in the embedding layer Perform embedding operation to obtain , specifically:

[0163] in, , N is the number of patches, D is the embedding dimension;

[0164] Then in Add a class token to represent the final classification information, and we get , specifically:

[0165]

[0166] in, For splicing operation, are learnable parameters, which are ultimately used for classification tasks;

[0167] Give again Adding position embedding gives , specifically:

[0168] in, Encode the embedded position information;

[0169] Step 4.6, Pass it into the Dropout layer to prevent overfitting, and get , ;

[0170] Step 4.7, As the input of the channel attention block, the feature is obtained after the channel attention weighting by the channel attention block ;like Figure 3-4 As shown in Figure 2, the channel attention block includes a channel attention module, a Dropout layer, and a Layer Normalization layer connected in sequence. The channel attention block The processing process is:

[0171] Perform global pooling:

[0172] in, is average pooling, is the maximum pooling, N is the number of patches;

[0173] Calculating channel attention :

[0174] in and They are the fully connected layers, is the activation function;

[0175] According to the channel attention Weighted to get weighted features :

[0176] Then the weighted features are Dropout processed and normalized, and finally the processed features are residually connected with the original features, specifically:

[0177] Dropout processing :

[0178] Layer Normalization :

[0179] Will and Perform residual connection to obtain : ;

[0180] The function of the entire channel attention block is as follows:

[0181] Channel attention module: By weighting the input features, it highlights the features related to the task.

[0182] Dropout: Prevent overfitting and improve the generalization ability of the model.

[0183] Layer Normalization: Improves training stability and accelerates convergence.

[0184] Residual connection: By adding the original input to the processed features, the gradient vanishing problem is alleviated and information can be transmitted in the network.

[0185] Step 4.8, the features As the input of the first ViT encoder, it is processed by the ViT encoder and output ;like Figure 5As shown, the processing process of the ViT encoder is:

[0186] Perform multi-head attention calculation:

[0187] in, is query, key, value, is the dimension of the key, is the normalization function;

[0188] Then, according to the multi-head attention Then perform residual connection and layer normalization to obtain features :

[0189]

[0190] in Normalize the layer;

[0191] Then After being processed by the feedforward network, we get :

[0192] It is a feed-forward network, which consists of two fully connected layers, ReLU activation function, and layer normalization;

[0193] The obtained Then input Perform residual connection and layer normalization to obtain features : ;

[0194] Step 4.9, Enter the second stage and get the features after passing through the second ViT encoder , the processing of the second stage is as follows:

[0195] The second channel attention block is Processing to obtain :

[0196]

[0197] in, represents the second channel attention block;

[0198] Then the second ViT encoder processes it to get :

[0199]

[0200] in, Represents the second ViT encoder;

[0201] Step 4.10, pass the features after the second ViT encoder Take the class token for classification, specifically: ,in, To extract the class token with global information after training;

[0202] Step 4.12, then Input the linear layer and use the softmax function for probability matching classification, specifically:

[0203] in, is the normalization function, is the weight matrix of the classification head, Weight bias vector for the classification head.

[0204] The channel attention block and ViT encoders with different numbers of heads form a stage to process and identify fault types at different levels, ensuring that the model can notice changes in fault signals at different levels and improve the accuracy of the model.

[0205] Compared with the existing ViT model, the model of the present invention introduces dilated convolution and spatial pyramid pooling. Dilated convolution expands the receptive field of convolution by inserting holes in the convolution kernel, and can capture a wider range of contextual information. Unlike traditional convolution, dilated convolution does not increase the number of additional parameters, because it only adds a dilated structure to the existing convolution kernel, thereby covering a larger area without adding new convolution kernels or layers, which avoids the increase in the number of parameters.

[0206] Spatial pyramid pooling (SPP) helps capture multi-scale features by performing pooling operations at multiple scales. The SPP module uses a fixed pooling window and step size to avoid the large increase in parameters brought by the fully connected layer. It generates feature maps of different scales by pooling feature maps. This process does not involve additional trainable parameters, so it does not significantly increase the number of model parameters.

[0207] Although adding dilated convolution and spatial pyramid pooling enhances the expressiveness of the model, it does not significantly increase the number of parameters. In addition, the present invention also optimizes the attention mechanism, by controlling the number of heads and the number of layers, it maintains a high representation capability and avoids the introduction of too many parameters.

[0208] Example 5

[0209] Based on Example 4, when the CpcsViT model is trained in step 4, the Adam optimizer and scheduler Onecycle are used, the batch_size is 32, and the learning rate is 0.001.

[0210] Example 6

[0211] On the basis of Example 5, in this embodiment, B is 64, C is 3, H and W are both 224, patch_size is set to 16, N is 197, and D is 768.

[0212] Example 7

[0213] On the basis of Example 6, in order to verify the effectiveness of the present invention in mechanical equipment fault diagnosis, the Case Western Reserve Bearing (CWRU) data set was selected for experiment. In terms of signal preprocessing, we first segmented the original vibration fault signal according to time_steps (time step) = 2048 and overlap_ratio (overlap ratio) = 0.5. Then, the short-time Fourier transform (STFT) was applied to the segmented signal to convert it into a color three-dimensional fault signal image. After segmentation and conversion, each fault type was processed into 118 pictures.

[0214] This embodiment selects three different motor speed load conditions from the CWRU data set, which are 1750r / min, 1772r / min and 1797r / min, and the corresponding fault diameters are 0.007 inches, 0.014 inches and 0.021 inches, respectively. During the training and testing process, a unified classification standard is used to process the data under all conditions. Each fault type is uniformly classified into one label, so each label category will contain 356 pictures. Table 1 shows the specific classification method for one of the working conditions. The same classification method is used for the other two motor load and speed conditions.

[0215] Table 1

[0216]

[0217] This embodiment divides the data under different working conditions into three groups of data, A, B, and C, and conducts comparative tests on multiple models using a mode of training under one working condition and testing under another working condition. The model hyperparameters are set to: Adam optimizer, Onecycle scheduler, batch_size is 32, and learning rate is 0.001. The ViT model, EfficientNet-B2, and RegNetY-800MF use the Adam optimizer and StepLR scheduler. Since the Adam optimizer used by SqueezeNet and Vgg16 is particularly poor, the Adamax optimizer is used instead. Table 2 shows the best performance of different models in the experiment, and also compares the training parameters between different models:

[0218] Table 2

[0219]

[0220] It can be seen from the above table that in cross-domain fault detection, the traditional model performs better when the working condition difference is small, such as AB. However, when the motor speed and load difference is large, the detection effect of the traditional model is significantly different, such as CA. The method proposed in the present invention can still maintain stable performance and maintain a very high accuracy rate in the face of large differences in working conditions. This shows that the method proposed in the present invention has strong robustness and can effectively adapt to bearing fault detection under different working conditions.

[0221] The method proposed in the present invention has the least number of parameters in the training process, which not only can significantly save computing resources, but also improves the generalization ability and deployment flexibility of the model, and accelerates the training process, further improving its efficiency in practical industrial applications.

[0222] The present invention combines the channel-level and patch-level weighted attention mechanisms to significantly improve the accuracy and robustness of bearing fault detection. The experimental results verify the superior performance of the model on the Western Reserve Case bearing dataset and demonstrate its wide application potential in the field of industrial fault diagnosis.

Claims

1. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting is characterized by: The specific steps are as follows: Step 1, obtaining original vibration fault signals of different fault types from a public data set; Step 2, preprocessing the original vibration fault signal obtained in step 1; Step 3, establishing a CpcsViT model for mechanical equipment fault detection, including a feature enhancement module and a feature extraction module connected in sequence; The feature enhancement module includes a hole convolution module, a spatial attention module, a pyramid pooling module, and a batch normalization layer connected in sequence; The feature extraction module includes an embedding layer, a Dropout layer, a channel attention block, a ViT encoder, a channel attention block, a ViT encoder, and a linear layer connected in sequence; The channel attention block dynamically weights the feature dimensions within each patch through the channel attention mechanism. First, the feature channels of each patch are globally pooled, and the feature representations of the average pooling and the maximum pooling are calculated respectively. Then, these pooling results are passed through the fully connected layer to generate channel-level weighting coefficients, and normalized by the sigmoid function to finally weight the features of each patch; Each channel attention block and the subsequent ViT encoder form a stage, that is, the feature extraction module consists of two stages; Step 4, inputting the original vibration fault signal preprocessed in step 2 into the CpcsViT model established in step 3, setting training parameters, and training the CpcsViT model; Step 5: Use the trained model to perform fault diagnosis.

2. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 1 is characterized in that: The step 2 is specifically as follows: The original vibration fault signal obtained in step 1 is segmented, and then the segmented signal is converted into a color three-dimensional fault signal image by applying short-time Fourier transform.

3. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 2, characterized in that: When segmenting the original vibration fault signal, the segmentation is performed according to the time step = 2048 and the overlap rate = 0.

5.

4. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 3 is characterized in that: The first ViT encoder includes 4 layers of 8-head self-attention layers; the second ViT encoder includes 5 layers of 16-head self-attention layers.

5. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 4, characterized in that: In step 4, the original vibration fault signal preprocessed in step 2 is input into the CpcsViT model established in step 3. Assume that the image obtained after preprocessing in step 2 is , where B is the batch size, C is the number of channels, H and W are the height and width of the image, then the processing process of the CpcsViT model is as follows: Step 4.1, The input is fed into the dilated convolution module, and the input is processed by dilated convolution according to the following formula: in, is the feature map output by the hole convolution module, It is the dilated convolution processing; Step 4.2, then Input to the spatial attention module to obtain the output feature map of the spatial attention module ; Step 4.3, then Input to the pyramid pooling module to get the output feature map of the pyramid pooling module ; Step 4.4, Batch normalization is performed through the batch normalization layer, specifically: in, For batch normalization, To obtain the feature map after batch normalization, ; Step 4.5: Feature map obtained after processing by feature enhancement module Input into the feature extraction module, and in the embedding layer Perform embedding operation to obtain , specifically: in, , N is the number of patches, D is the embedding dimension, For embedded operations; Then in Add a class token to represent the final classification information, and we get , specifically: in, For splicing operation, are learnable parameters, which are ultimately used for classification tasks; Give again Adding position embedding gives , specifically: in, Encode the embedded position information; Step 4.6, Pass it into the Dropout layer to prevent overfitting, and get , ; Step 4.7, As the input of the channel attention block, the feature is obtained after the channel attention weighting by the channel attention block ; Step 4.8, the features As the input of the first ViT encoder, it is processed by the ViT encoder and output ; Step 4.9, Enter the second stage and get the features after passing through the second ViT encoder ; Step 4.10, pass the features after the second ViT encoder Take the class token for classification, specifically: in, To extract the class token with global information after training; Step 4.12, then Input the linear layer and use the softmax function for probability matching classification, specifically: in, is the normalization function, is the weight matrix of the classification head, Weight bias vector for the classification head.

6. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 5, characterized in that: In step 4.2, the spatial attention module first Perform pooling operations, namely average pooling and maximum pooling operations, specifically: in, Get the result for the average pooling operation; The result obtained for the maximum pooling operation; Then convolution is performed on the results obtained from the average pooling and maximum pooling operations to calculate the spatial attention to obtain the attention map , specifically: Among them, σ represents the Sigmoid activation function; The attention map then weights each position to obtain the weighted output features of the spatial attention module: .

7. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 5, characterized in that: In step 4.3, the pyramid pooling module first Use the pooling window as , 2 Perform average pooling operation, specifically: in, For Use the pooling window as , 2 Perform an average pooling operation to obtain a collection of results. For The pooling window is The average pooling operation, For The pooling window is The average pooling operation, For The pooling window is Average pooling operation; Then, the results of the average pooling operation are upsampled separately, specifically: in, is the result set of the upsampling operation, For conduct , For conduct , For conduct ; Then concatenate the operations to get Specifically: ; in, For splicing operation.

8. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 5, characterized in that: The channel attention block in step 4.7 is The processing process is: Perform global pooling: in, is average pooling, is the maximum pooling, N is the number of patches; Calculating channel attention : in, and They are the fully connected layers, is the activation function; According to the channel attention Weighted to get weighted features : Then the weighted features are Dropout processed and normalized, and finally the processed features are residually connected with the original features, specifically: Dropout processing : Layer Normalization : Will and Perform residual connection to obtain : ; The processing process of the ViT encoder in step 4.8 is: Perform multi-head attention calculation: in, is query, key, value, is the dimension of the key, is the normalization function; Then, according to the multi-head attention Then perform residual connection and layer normalization to obtain features : in Normalize the layer; Then After being processed by the feedforward network, we get : It is a feed-forward network, which consists of two fully connected layers, ReLU activation function, and layer normalization; The obtained Then input Perform residual connection and layer normalization to obtain features : ; In step 4.9, Enter the second stage. The processing process of the second stage is as follows: The second channel attention block is Processing to obtain : in, represents the second channel attention block; Then the second ViT encoder processes it to get : in, Represents the second ViT encoder.

9. The ViT cross-domain fault diagnosis method combining adaptive spatial attention and patch feature weighting according to claim 5, characterized in that: When the CpcsViT model is trained in step 4, the Adam optimizer and scheduler Onecycle are used, the batch_size is 32, and the learning rate is 0.001.

Images