A small sample fault diagnosis method based on feature migration and transformer

Abstract

The application provides a small sample fault diagnosis method based on feature migration and a Transformer, and comprises the following three steps: (1) data sampling, (2) feature migration LEN-Transformer modeling and training, and (3) fault diagnosis test; the application has the beneficial effects that: a soft weight feature fusion method based on double-channel scoring is used to provide a quantitative basis for channel screening and weight allocation, the contribution difference of different channels to fault category distinction is fully reflected, and the adaptive ability of the feature fusion process is improved; an intermittent source domain mixed model training strategy is designed, the source domain gradient is prevented from suppressing target domain feature learning in the early training stage, and the problem that the model training precision is not high due to the large sample quantity difference between the source domain and the target domain under the small sample condition is effectively overcome; a fault diagnosis method based on feature migration LEN-Transformer is constructed, and the problem that the target domain sample quantity is small and overfitting is prone to occur in the later training stage is solved; the small sample fault diagnosis performance is considered, the calculation resource consumption is not increased, and the application scenario is good.

Description

Technical Field

[0001] This invention relates to the field of small-sample fault diagnosis, and more specifically, to a small-sample fault diagnosis method based on feature transfer and Transformer. Background Technology

[0002] Rotating machinery is a crucial component of various mechanical equipment, widely serving industrial equipment fields such as power systems, transportation, intelligent robots, high-end equipment manufacturing, mining, and metallurgy. Bearings are the core fundamental components of rotating machinery systems, undertaking important functions such as supporting rotating components, defining relative positions, and connecting related motion units. Their operating status directly affects the safety, stability, and reliability of the entire system. Research shows that bearing failure has a high probability of occurrence, becoming one of the important factors affecting the normal operation of rotating machinery. Once a bearing malfunctions, it not only affects the normal operation of the equipment but can also lead to accidents resulting in mechanical failure and loss of life, causing huge direct economic losses and adverse social impacts. Therefore, accurate diagnosis of bearing failures is of great significance for the safe and stable operation of equipment and reducing the risk of accidents.

[0003] In actual industrial equipment operation, it operates normally most of the time, while fault conditions occur infrequently. This makes fault sample acquisition difficult and costly, resulting in a very limited number of data samples for some fault categories. Insufficient sample data leads to unstable training of the fault diagnostic tool, poor generalization performance, and low diagnostic accuracy. Data augmentation algorithms are typically used to expand small samples before fault diagnosis and classification. However, when there are too many target domain categories, each data augmentation consumes significant additional computational resources. To avoid increasing the computational cost of data augmentation, and because the source domain data volume is much larger than the target domain, the large difference in sample size between the two domains leads to the model training being dominated by the source domain number in the early stages. To address the issue of overfitting to target domain data in the mid-to-late stages, a fault diagnosis method based on feature transfer LEN-Transformer is proposed. This method utilizes publicly available source domain datasets and employs a soft-weight feature fusion method based on dual-channel scoring to provide quantitative basis for channel selection and weight allocation, thereby improving the adaptive capability of the feature fusion process. An intermittent source domain mixing model training strategy is adopted to avoid source domain gradients suppressing target domain feature learning in the early stages of training. Combining the powerful feature extraction capabilities of Transformer and the cross-domain generalization ability of transfer learning, this method compensates for the limited number of fault label samples in the target domain through cross-operating conditions, cross-speed, and cross-equipment knowledge transfer, thus improving the accuracy of small-sample bearing fault diagnosis and ensuring the safe and stable operation of equipment. This is essential. Summary of the Invention

[0004] In order to at least solve one or more of the problems mentioned above, this invention discloses a small-sample fault diagnosis method based on feature transfer and Transformer.

[0005] The present invention adopts the following technical solution: Step 1: Data Sampling Using a sliding window sampling method, n sets of data were sampled from the KAIST dataset (released by the Noise and Control Center of the Korea Advanced Institute of Science and Technology) as source domain samples. S source n+m sets of data were sampled from the CWRU dataset (publicly released by the Bearing Data Center at Case Western Reserve University). S target,original m sets of data are used as samples for the target domain. S target Sample n sets of data as fault diagnosis test samples S test .

[0006] Step 2: Feature Transfer LEN-Transformer Modeling and Training First, a feature transfer LEN-Transformer model is established. This model comprises a feature fusion module, a parameter-sharing backbone, a source domain branch, and a target domain branch. The feature fusion module is a soft-weighted feature fusion module based on dual-channel scoring. The parameter-sharing backbone consists of 1D convolution, a lightweight Transformer, another 1D convolution, and another lightweight Transformer. The lightweight Transformer is a Transformer module with a global attention mechanism. The source domain branch is a softmax classifier. The target domain branch consists of a LEN-Transformer module, a temporal module, and a softmax classifier, respectively. Then, the feature transfer LEN-Transformer model is trained using source domain data. S source Target domain data S target As input, source domain data S source The input is trained using an intermittent source-domain hybrid module, and the source-domain fused features are obtained through dual-channel soft-weight feature fusion. f source,fusion Features fused with the target domain f target,fusion Then, the common features of the domain are obtained through the parameter sharing backbone. f source,com and f target,com ,Willf source,com The classification result is obtained by inputting the source domain branch. ,Will f target,com Input the target domain branch to obtain the classification result Calculate the loss, perform backpropagation, and update the model parameters. If the termination training condition is reached, save the feature transfer LEN-Transformer model parameters and apply the test data. S test Input the data into the model for fault diagnosis testing.

[0007] (1) The soft weight feature fusion module based on dual-channel scoring evaluates multi-source signal channels from the perspectives of cross-channel correlation classification and discrimination ability by using correlation scoring based on phase locking value (PLV) and classification scoring based on permutation importance. The two types of scoring results are weighted and fused to obtain the final channel weighted score. Then, the score is normalized to construct a dual-branch soft weight. The dual-branch soft weight is multiplied by the original data channel by channel as the input tensor for dual-branch convolution feature extraction, thereby obtaining two sets of feature tensors. The two sets of feature tensors are then concatenated to obtain multi-source fusion features.

[0008] Let the original data be

[0009] In the formula, C Indicates the number of fault categories. N This represents the number of samples for each type of fault. M Indicates the number of feature channels. L This represents the data length of each feature channel, for any class of the first feature channel. n The sample, the first i Each channel signal is represented as

[0010] Correlation scoring based on phase-locked values S relat Classification scoring based on the importance of permutations S class The final weighted score is obtained by weighted fusion.

[0011] In the formula, α These are weighting coefficients. α ∈[0,1]; , ;

[0012] Normalization of channel scores within each class Sfinal The normalized score is obtained as

[0013] In the formula, dim is the channel. It is a subtraction of each channel. It is an extremely small constant;

[0014] A two-branch soft weight structure is constructed based on the normalized scores, with higher weights... W high Low weight W low They are respectively

[0015] Apply the two-branch soft weights to the original data. X Channel-by-channel multiplication is used as the input tensor for bi-branch convolution feature extraction, resulting in two sets of feature tensors.

[0016] In the formula, h and l These represent the number of high-weight and low-weight convolutional layers, respectively. It is the activation function after each convolutional layer. It involves multiplying each channel sequentially.

[0017] By concatenating the two sets of feature tensors, the multi-source fused features are obtained.

[0018] 1) The correlation scoring based on phase-locked values ​​uses instantaneous phase synchronization relationships to calculate the coupling degree between each channel and the other channels, in order to reflect the correlation structure within the multi-source signal. The specific scoring process is as follows: First, its analytic signal is constructed using the Hilbert transform. In the formula, represents the Hilbert transform. j It is the imaginary unit.

[0019] For the n Any two feature channels of a sample i and k The PLV is

[0020] In the formula, For the first n Any two feature channels of each channel i and k The instantaneous phase difference, ,in , The first n Any two feature channels of each channel i and k The instantaneous phase, and , arg( • ) represents the phase angle operation of complex numbers.

[0021] when , The closer the value is to 1, the stronger the phase consistency between the two channels and the more obvious the synchronization characteristics; when it is close to 0, it indicates that the phase relationship between the two channels is more discrete and the coupling is weaker; for the sample n Its PLV matrix can be further constructed as follows:

[0022] To convert the PLV matrix into channel-level ratings, calculate the first... i The correlation score of each channel is the average of its PLV with all other channels.

[0023] This score reflects the consistency of a particular feature channel in phase synchronization with other feature channels in the current sample; a higher score indicates that the channel is in a more central or stable position in the multi-source coupling structure; to facilitate unified comparison between different samples and different categories, the score is adjusted accordingly. n The first sample i The correlation scores of each channel are subjected to global min-max normalization to obtain...

[0024] In the formula, s r, min and s r, max Let represent the minimum and maximum correlation scores for all samples and all channels, respectively. ε It is an extremely small positive number to prevent division by zero.

[0025] The channel correlation score for each type of fault is obtained by averaging the channel correlation scores of N sets of samples.

[0026] 2) The classification scoring based on permutation importance uses statistical features and the permutation importance of the classification model to evaluate the contribution of each channel to distinguishing fault categories. The specific scoring process is as follows: For any sample, the th iThe channel signal is used to extract three statistical features: root mean square (RMS), standard deviation (Std), and kurtosis. The resulting 3D statistical feature tensor... f i =[ f RMS , f Std , f Kurtosis ]= [ f i,1 , f i,2 , f i,3 The feature tensors of each channel are concatenated and expanded to obtain the classification input matrix of XGBoost.

[0027] XGBoost's classifier uses a multi-class softprob objective function, resulting in a mapping relationship between input features and class probability distributions.

[0028] In the formula, Y For fault category labels, Y ∈{0,1,2,…,C}.

[0029] After obtaining the trained classifier, permutation importance is used to measure the classification contribution of each feature. Keeping other features in the validation set unchanged, the values ​​of a certain feature are randomly shuffled, and the magnitude of the performance drop is observed. The more significant the performance drop, the more important the feature. k The importance of a first permutation of each feature is:

[0030] In the formula, Acc base The original validation set accuracy. Acc k perm This represents the model's accuracy.

[0031] The feature is then permuted 10 times, and the average value is used to obtain the importance score of the feature. To avoid negative values ​​interfering with the scoring, negative permutation importance values ​​are truncated. ,get

[0032] By combining all importance values, we get 3× M The feature-level importance vector is

[0033] No. i The original categorical importance of each channel is defined as follows:

[0034] In the formula, I i,1 , I i,2 and I i,3 The first i The importance of RMS, Std, and Kurtosis features corresponding to each channel.

[0035] Therefore, the original importance tensor of the feature channels can be obtained as follows: Then perform min-max normalization to obtain the th i The global importance of each feature channel is:

[0036] In the formula, I min and I max These are the minimum and maximum values ​​of the original importance of all channels, respectively. It represents the global classification contribution of each feature channel in the overall classification task.

[0037] Using only global importance ignores the dynamic differences in channel performance across different samples; therefore, we further refine the global permutation importance using the statistical feature strength of the current sample to obtain the importance of each sample. i The classification score for each channel is

[0038] In the formula, g i For the first i The overall strength of each channel ,in In each sample i Locally normalized values ​​of the 3D statistical characteristics of each channel. , f i,min and f i,max For the first i The minimum and maximum values ​​among the three statistical characteristics of each channel.

[0039] For the first n The classification scores of each sample are subjected to global min-max normalization to obtain... In the formula, sc, min and s c, max These represent the minimum and maximum classification scores for all samples and all channels, respectively.

[0040] For each fault category, N The average of the group samples is used to obtain the category-level channel classification score. .

[0041] (2) The intermittent source domain hybrid module training strategy described above has the same algorithm flow as Algorithm 1. In this strategy, the target domain data is used as the main training data stream (which must participate in the update in each iteration step), and the source domain data is only inserted into the training at certain intervals. This ensures that the parameter sharing backbone has a stable discrimination ability while avoiding the source domain gradient suppressing the target domain feature learning in the early stage of training. It effectively overcomes the disadvantages of the large difference between the source domain and target domain sample sizes in the case of small samples, which leads to the model being dominated by the source domain number in the early stage of training and overfitting the target domain data in the middle and late stages of training.

[0042]

[0043] In the source domain branch, shared features are directly input into the source domain Softmax classifier to obtain the prediction result. And adopt cross-entropy loss Constraining its ability to discriminate source domain data, cross-entropy loss is defined as... In the formula, N s It is the current step The number of source domain samples.

[0044] The supervisory signal is backpropagated to the parameter-sharing backbone, enabling the shared features to have stable separability in the source domain, thereby providing a transferable discriminative structure.

[0045] In the target domain branch, to enhance the feature learning ability for small sample target domains, a LEN-Transformer module is introduced to strengthen the learning ability for dissimilar features in the target domain, so as to obtain the final target domain features. f target The classification result is obtained by using a target domain softmax classifier. And using Focal loss Constraining the discriminative ability of target domain data, using Center loss. Auxiliary domain learning, , The loss is defined as follows:

[0046] In the formula, N t It is the current step The number of samples in the target domain.

[0047]

[0048] In the formula, yes Feature center ;

[0049] To maintain the discriminative power of the source domain while reducing the distribution difference between the two domains, thereby improving the ability of the parameter-shared backbone to extract similar features, a difference loss L between the feature distributions of the source and target domains is introduced in addition to the two-domain classifier loss. MMD Its definition is

[0050] In the formula, MMD is the Maximum Mean Discrepancy (MMD) calculation.

[0051] Based on the above reasons, a total loss function is constructed to optimize the overall objective, and its expression is:

[0052] In the formula, , , and These are the weight coefficients for the source domain cross-entropy loss, the target domain Focal loss, the target domain Center loss, and the domain difference loss, respectively. , These are the dynamic weighting coefficients for the target domain Center loss and the domain difference loss, respectively. , , e These are the parameters for the dynamic weighting coefficients. step These are parameters for determining whether to train in combination with the source domain. epochs This is the preset total training period.

[0053] Step 3: Fault Diagnosis Test Using test data S test The input is fed into the LEN-Transformer model trained in step two, and a fault diagnosis test is performed. The effectiveness of the method is verified by using macro-precision (Macro-P), macro-recall (Macro-R), and macro-F1 score (Macro-F1) as performance metrics.

[0054] In the formula, TP i , FP i and FN i They are the first i True examples, false positive examples, and false negative examples of the same type of fault.

[0055] The beneficial effects and advantages of this invention are as follows: This invention employs a soft-weight feature fusion method based on dual-channel scoring to provide a quantitative basis for channel selection and weight allocation, fully reflecting the differences in contribution of different channels to fault category differentiation and improving the adaptive capability of the feature fusion process; it designs an intermittent source domain mixing model training strategy to avoid the source domain gradient suppressing the target domain feature learning in the early stages of training, effectively overcoming the low model training accuracy caused by the large difference in the sample size between the source and target domains in the case of small samples; it constructs a fault diagnosis method based on feature transfer LEN-Transformer to solve the problem of easy overfitting in the later stages of training due to the small sample size of the target domain; it ensures high accuracy in fault diagnosis with small samples without increasing computational resource consumption (such as data augmentation), and has a good application scenario. Attached Figure Description

[0056] Figure 1 This is a flowchart of a small-sample fault diagnosis method based on feature transfer and Transformer according to the present invention; Figure 2 This is a diagram of the feature transfer LEN-Transformer structure of the present invention; Figure 3 This is a structural diagram of the dual-channel scoring feature fusion module of the present invention; Figure 4 This is the model training loss curve based on the CWRU dataset of this invention; Figure 5 It is the model training accuracy curve based on the CWRU dataset; Figure 6 It is a diagnostic confusion matrix based on the CWRU dataset. Detailed Implementation

[0057] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments: See attached diagram. Figure 1 This is a flowchart of a small-sample fault diagnosis method based on feature transfer and Transformer according to the present invention, which includes the following three steps: (1) data sampling, (2) feature transfer LEN-Transformer modeling and training, and (3) fault diagnosis test.

[0058] Step 1: Data Sampling The datasets used are the publicly available KAIST dataset released by the Noise and Control Center of the Korea Advanced Institute of Science and Technology (KAIST) and the publicly available CWRU dataset released by the Bearing Data Center of Case Western Reserve University. Data details are shown in Tables 1 and 2, respectively. Using sliding window sampling, 100 sets of data were sampled for each fault type in the KAIST dataset, representing axial vibration, radial vibration, temperature, and current, as source domain samples. S source Each data set has a length of 2048; 120 data sets were randomly sampled from the CWRU dataset using a sliding window sampling method. S target,original 20 groups as target domain samples S target 100 sets were used as fault diagnosis test samples. S test .

[0059]

[0060]

[0061] Step 2: Feature Transfer LEN-Transformer Modeling and Training See attached diagram. Figure 2 This is a structural diagram of the LEN-Transformer feature transfer model of the present invention. The LEN-Transformer model comprises a feature fusion module, a parameter-sharing backbone, a source domain branch, and a target domain branch. The feature fusion module is a soft-weighted feature fusion module based on dual-channel scoring. See Appendix. Figure 3 The relevance scoring based on PLV and the classification scoring based on permutation importance are weighted and fused to obtain the final channel-weighted score. This score is then normalized to construct a dual-branch soft weight. The dual-branch soft weight is then multiplied channel-by-channel with the original data as the input tensor for dual-branch convolutional feature extraction, resulting in two sets of feature tensors. These two sets of feature tensors are then concatenated to obtain the source-domain fusion feature. f source,fusion Features fused with the target domain f target,fusion The parameter-sharing backbone consists of a 1D convolution, a lightweight Transformer, another 1D convolution, and another lightweight Transformer, with the lightweight Transformer being a Transformer module with a global attention mechanism. The source domain branch is a softmax classifier. The target domain branch consists of a LEN-Transformer module, a Temporal module, and a softmax classifier.

[0062] The feature transfer LEN-Transformer model was trained according to Algorithm 1, and the initialization parameters of the feature transfer LEN-Transformer model were shown in Table 3; using source domain data S source Target domain data S target As input, source domain data S source The input is given according to the intermittent source-domain hybrid module training strategy, and then the source-domain fusion features are obtained by the dual-channel soft-weight feature fusion module based on equations (1)-(23). f source,fusion Features fused with the target domain f target,fusion Then, the common features of the domain are obtained through the parameter sharing backbone. f source,com and f target,com ,Will f source,com The classification result is obtained by inputting the source domain branch. ,Will f target,com Input the target domain branch to obtain the classification result The cross-entropy loss is calculated using equation (24). Formula (25) calculates the Focal loss. Equation (26) calculates the Center loss L. center Equation (27) calculates the loss L due to the large mean difference. MMD Equation (28) is used to calculate the total loss L. total It then performs backpropagation and updates the model parameters until training is complete; training is then completed and each... split All epoch The loss value, the accuracy on the source domain test set, the accuracy on the target domain test set, and the accuracy based on... S test The accuracy (which does not affect any loss value) and each split The model parameters for the cycle with the best results after completion.

[0063] Table 3 Initial parameters of the feature transfer LEN-Transformer model

[0064] See attached diagram. Figure 4The graph shows the model training loss curves based on the CWRU dataset. As can be seen, under different random seeds (31, 3, 1, 9), all losses exhibit a consistent convergence trend, indicating that the constructed model demonstrates good stability and repeatability during training. Overall, the total loss decreases rapidly in the early stages of training and then gradually stabilizes, suggesting that the model parameters can be effectively optimized within a relatively small number of training epochs. center It dominates in the early stages of training, with a value significantly higher than other losses, and then decreases rapidly as training progresses. This indicates that the model learns to gradually reduce the distance between similar samples and the class center, increase the distance between different classes, and effectively enhance the constraint of the domain difference loss. and It can also quickly converge to a lower level, indicating that both source domain supervision and target domain Focal constraints can effectively improve the network's learning ability on target domain data and softmax classification accuracy; in addition, since the source domain data is intermittently mixed into the training, L mmd The trend did not resemble L. center Similarly, it converges from high to low, at 500 epoch The fact that the loss between the left and right sides exceeds the center and remains within a certain range reflects that the main optimization pressure in the later stages of the lightweight Transformer part shifts to the alignment of the domain distribution. However, the fact that it does not approach 0 for a long time indicates that there are still distribution differences in the shared features between the source and target domains.

[0065] See attached diagram. Figure 5 The graph shows the model training accuracy curves based on the CWRU dataset. As can be seen from the graph, under different random seeds (31, 3, 1, 9), the accuracy of the source domain test set, the accuracy of the target domain test set, and the accuracy based on the CWRU dataset all increase. S test The accuracy rates showed a trend of rapid increase in the early stages of training, slow improvement in the middle stages, and stable performance in the later stages. The later stages did not show a continuous decline or significant divergence, indicating that the network optimization process was relatively stable and the parameter update direction was accurate. Moreover, in the middle and later stages of training, all three accuracies were greater than 90% and close to 100%, indicating that the features learned by the model have strong discriminative power. At the same time, this trend did not change significantly with the random seed, indicating good robustness.

[0066] Step 3: Fault Diagnosis Test Using test data S test Input the feature transfer LEN-Transformer model trained in step two into the fault diagnosis test and save the corresponding confusion matrix (see attached figure). Figure 6The diagram shows the diagnostic confusion matrix based on the CWRU dataset. As can be seen from the figure, the accuracies under different random seeds (31, 3, 1, 9) are 98.9%, 97.7%, 97.6%, and 97.5%, respectively. This indicates a slight deficiency in the network's generalization performance, demonstrating some sensitivity to random partitioning, but with minimal fluctuation. The main issue is that a small number of samples fall into a few categories outside the diagonal, meaning confusion exists only between a few categories, while the remaining categories are almost all correctly identified. Furthermore, the misdiagnosis occurs with labels 7, 8, and 9, representing rolling body failure. This suggests that the confusion is more likely due to the source domain dataset KAIST not providing rolling body failure features, rather than training instability or overall failure. Overall, the main problem is not widespread misclassification, but rather a small proportion of confusion concentrated at the boundaries of a few categories. The initial network parameters with random seed 31 handle these boundaries best, resulting in the highest accuracy.

[0067] To verify the effectiveness of the proposed feature transfer LEN-Transformer model in fault diagnosis under small sample conditions, ablation experiments were conducted to evaluate the impact of transfer methods, feature learning, loss design, and training strategies on fault diagnosis performance under different modalities. The detailed arrangement of the ablation experiments is shown in Table 4.

[0068]

[0069] With 10 random seeds, the Macro-P, Macro-R, and Macro-F1 of the five methods were calculated according to Equations (29) to (31) for three different target domain datasets, as shown in Table 5. The table shows that in the case of small samples, M0, which uses a simple convolutional neural network to extract features and combines it with a softmax classifier, has weak diagnostic performance, with Macro-P, Macro-R, and Macro-F1 all below 82%. M1, by introducing a Transformer to improve the feature extraction process, shows improvements in Macro-P, Macro-R, and Macro-F1, but due to the small sample size, it still does not produce good diagnostic results. M2 incorporates feature information from multi-source data, improving all indicators, but none exceed [the required standard]. The 90% improvement indicates that enhancing the model's feature extraction capabilities and fusing multi-source data can improve model performance to some extent, but cannot fundamentally address the problems caused by small sample sizes. M3 significantly improves performance by optimizing loss feature transfer, with Macro-R reaching 95.64%, a 15.23% improvement over M0, but still lagging behind current best methods. M4 enhances the influence of target domain samples by intermittently mixing source domain samples to interfere with the training process, resulting in substantial improvements in Macro-P, Macro-R, and Macro-F1, all exceeding 99.50%, representing increases of 6.40%, 3.97%, and 5.20% compared to M3, respectively. To further validate the diagnostic performance of the proposed method, it was compared with DFATN... [1] CSTAN [2] PCFM [3] and MRFSN [4] The methods were compared, and the results are shown in Table 6. Among these methods, the highest Macro-P, Macro-R, and Macro-F1 scores were 97.81, 97.64, and 97.72, respectively. The M4 method improved these scores by 1.76%, 1.97%, and 1.87%, respectively. The results show that M4, in the case of small samples, only adopts the intermittent source-domain mixing training strategy, without increasing the time cost and computational overhead required by other additional training (such as data augmentation), and obtains stable and effective diagnostic results, demonstrating high practical application value and engineering promotion significance.

[0070]

[0071]

Claims

1. A small-sample fault diagnosis method based on feature transfer and Transformer, used for small-sample fault diagnosis without data augmentation, which can achieve high accuracy in small-sample fault diagnosis without increasing computational resource consumption, includes the following three steps: (1) data sampling, (2) feature transfer LEN-Transformer modeling and training, and (3) fault diagnosis testing, characterized in that: A feature transfer LEN-Transformer model is established, comprising a feature fusion module, a parameter-sharing backbone, a source domain branch, and a target domain branch. This addresses the issue of overfitting in the later stages of training due to the small sample size in the target domain. The feature fusion module is a soft-weighted feature fusion module based on dual-channel scoring. The parameter-sharing backbone consists of 1D convolution, a lightweight Transformer, another 1D convolution, and another lightweight Transformer. The lightweight Transformer is a Transformer module with a global attention mechanism. The source domain branch is a softmax classifier, and the target domain branch consists of a LEN-Transformer module, a temporal module, and a softmax classifier. The feature transfer LEN-Transformer model is then trained using source domain data. S source Target domain data S target As input, source domain data S source The input is trained using an intermittent source-domain hybrid module, and the source-domain fused features are obtained through dual-channel soft-weight feature fusion. f source,fusion Features fused with the target domain f target,fusion Then, the common features of the domain are obtained through the parameter sharing backbone. f source,com and f target,com ,Will f source,com The classification result is obtained by inputting the source domain branch. ,Will f target,com Input the target domain branch to obtain the classification result Calculate the loss, perform backpropagation, and update the model parameters. If the termination training condition is reached, save the feature transfer LEN-Transformer model parameters and apply the test data. S test Input the data into the model for fault diagnosis testing.

2. The small-sample fault diagnosis method based on feature transfer and Transformer according to claim 1, characterized in that, The soft-weight feature fusion module based on dual-channel scoring evaluates multi-source signal channels from two perspectives: cross-channel correlation classification and discrimination ability. It employs correlation scoring based on Phase Locking Value (PLV) and classification scoring based on permutation importance to jointly assess these channels. The two sets of scores are then weighted and fused to obtain a final channel-weighted score. This score is then normalized to construct a dual-branch soft weight. The dual-branch soft weight is multiplied channel-by-channel by the original data to serve as the input tensor for dual-branch convolution feature extraction, resulting in two sets of feature tensors. These two sets of feature tensors are then concatenated to obtain multi-source fusion features. This dual-channel scoring-based soft-weight feature fusion module provides a quantitative basis for channel selection and weight allocation, fully reflecting the differences in the contribution of different channels to fault category differentiation and improving the adaptability of the feature fusion process. Let the original data be ， In the formula, C Indicates the number of fault categories. N This represents the number of samples for each type of fault. M Indicates the number of feature channels. L This represents the data length of each feature channel, for any class of the first feature channel. n The sample, the first i Each channel signal is represented as ， Correlation scoring based on phase-locked values S relat Classification scoring based on the importance of permutations S class The final weighted score is obtained by weighted fusion. ， In the formula, α These are weighting coefficients. α ∈[0,1]; , ; Normalization of channel scores within each class S final The normalized score is obtained as ， In the formula, dim is the channel. It is a subtraction of each channel. It is an extremely small constant; A two-branch soft weight structure is constructed based on the normalized scores, with higher weights... W high Low weight W low They are respectively ， Apply the two-branch soft weights to the original data. X Channel-by-channel multiplication is used as the input tensor for bi-branch convolution feature extraction, resulting in two sets of feature tensors. ， In the formula, h and l These represent the number of high-weight and low-weight convolutional layers, respectively. It is the activation function after each convolutional layer. It involves multiplying channel by channel; concatenating the two sets of feature tensors yields the multi-source fused features. ； (1) The correlation scoring based on phase-locked values ​​uses instantaneous phase synchronization relationships to calculate the coupling degree between each channel and the other channels, so as to reflect the correlation structure inside the multi-source signal. The specific scoring process is as follows: First, its analytic signal is constructed using the Hilbert transform. ， In the formula, Represents the Hilbert transform. j The imaginary unit; for the first n Any two feature channels of a sample i and k The PLV is ， In the formula, For the first n Any two feature channels of each channel i and k The instantaneous phase difference, ,in , The first n Any two feature channels of each channel i and k The instantaneous phase, and , arg( • () represents complex number phase angle operations; , The closer it is to 1, the stronger the phase consistency between the two channels and the more obvious the synchronization characteristics. When it approaches 0, it indicates that the phase relationship between the two channels is more discrete and the coupling is weaker; for the sample n Its PLV matrix can be further constructed as follows: ， To convert the PLV matrix into channel-level ratings, the calculation of the first... i The correlation score of each channel is the average of its PLV with all other channels. ， This score reflects the consistency of a particular feature channel in phase synchronization with other feature channels in the current sample; a higher score indicates that the channel is in a more central or stable position in the multi-source coupling structure; to facilitate unified comparison between different samples and different categories, the score is adjusted accordingly. n The first sample i The correlation scores of each channel are subjected to global min-max normalization to obtain... ， In the formula, s r, min and s r, max Let represent the minimum and maximum correlation scores for all samples and all channels, respectively. ε The number should be an extremely small positive number to prevent division by zero. The channel correlation score for each type of fault is obtained by averaging the channel correlation scores of N sets of samples. ， (2) The classification scoring based on permutation importance uses statistical features and the permutation importance of the classification model to evaluate the contribution of each channel to the distinction of fault categories. The specific scoring process is as follows: For any sample, the first... i The channel signal is used to extract three statistical features: root mean square (RMS), standard deviation (Std), and kurtosis. The resulting 3D statistical feature tensor... f i =[ f RMS , f Std , f Kurtosis ]= [ f i,1 , f i,2 , f i,3 The feature tensors of each channel are concatenated and expanded to obtain the classification input matrix of XGBoost. ， XGBoost's classifier uses a multi-class softprob objective function, resulting in a mapping relationship between input features and class probability distributions. ， In the formula, Y For fault category labels, Y ∈{0,1,2,…,C}; After obtaining the trained classifier, the permutation importance metric is used to measure the classification contribution of each feature. Keeping other features in the validation set unchanged, the values ​​of a certain feature are randomly shuffled, and the magnitude of the classification performance decline is observed. The more significant the performance decline, the more important the feature is. No. k The importance of a first permutation of each feature is: ， In the formula, Acc base The original validation set accuracy. Acc k perm The model accuracy is calculated by performing the permutation of this feature 10 times and averaging the results to obtain the feature's importance score. To avoid negative values ​​interfering with the scoring, negative permutation importance values ​​are truncated. ,get ， By combining all importance values, we get 3× M The feature-level importance vector is ， No. i The original categorical importance of each channel is defined as follows: ， In the formula, I i,1 , I i,2 and I i,3 The first i The RMS, Std, and Kurtosis feature importance values ​​corresponding to each channel; thus, the original feature channel importance tensor can be obtained as follows: Then perform min-max normalization to obtain the th i The global importance of each feature channel is: ， In the formula, I min and I max These are the minimum and maximum values ​​of the original importance of all channels, respectively. It represents the global classification contribution of each feature channel in the overall classification task; Using only global importance ignores the dynamic differences in channel performance across different samples; therefore, we further refine the global permutation importance using the statistical feature strength of the current sample to obtain the importance of each sample. i The classification score for each channel is ， In the formula, g i For the first i The overall strength of each channel ,in In each sample i Locally normalized values ​​of the 3D statistical characteristics of each channel. , f i,min and f i,max For the first i The minimum and maximum values ​​among the three statistical characteristics of each channel; For the first n The classification scores of each sample are subjected to global min-max normalization to obtain... ， In the formula, s c, min and s c, max Let represent the minimum and maximum classification scores for all samples and all channels, respectively. For each fault category, ... N The average of the group samples is used to obtain the category-level channel classification score. 。 3. The small-sample fault diagnosis method based on feature transfer and Transformer according to claim 1, characterized in that, The intermittent source-domain hybrid module training strategy described above has the same algorithm flow as Algorithm 1. In this strategy, the target domain data is used as the main training data stream (which must participate in the update in each iteration step), and the source domain data is only inserted into the training at certain intervals. This ensures that the parameter-sharing backbone has stable discriminative ability while avoiding the source domain gradient suppressing the target domain feature learning in the early stage of training. It effectively overcomes the drawbacks of the large difference between the source domain and target domain sample sizes in the case of small samples, which leads to the model being dominated by the source domain number in the early stage of training and overfitting the target domain data in the middle and late stages of training. ， In the source domain branch, shared features are directly input into the source domain Softmax classifier to obtain the prediction result. And adopt cross-entropy loss Constraining its ability to discriminate source domain data, cross-entropy loss is defined as... ， In the formula, N s It is the current step The number of source domain samples is determined; the supervisory signal is backpropagated to the parameter-sharing backbone, enabling the shared features to have stable separability in the source domain, thus providing a transferable discriminative structure; in the target domain branch, to enhance the feature learning ability for small sample target domains, a LEN-Transformer module is introduced to strengthen the learning ability for dissimilar features in the target domain, so as to obtain the final features of the target domain. f target The classification result is obtained by using a target domain softmax classifier. And using Focal loss Constraining the discriminative ability of target domain data, using Center loss. Auxiliary domain learning, , The loss is defined as follows: ， In the formula, N t It is the current step Number of samples in the target domain; ， In the formula, yes Feature center ; To maintain the discriminative power of the source domain while reducing the distribution difference between the two domains, thereby improving the ability of the parameter-shared backbone to extract similar features, a difference loss L between the feature distributions of the source and target domains is introduced in addition to the two-domain classifier loss. MMD Its definition is ， In the formula, MMD stands for Maximum Mean Discrepancy (MMD). Based on the above reasons, a total loss function is constructed to optimize the overall objective, and its expression is: ， In the formula, , , and These are the weight coefficients for the source domain cross-entropy loss, the target domain Focal loss, the target domain Center loss, and the domain difference loss, respectively. , These are the dynamic weighting coefficients for the target domain Center loss and the domain difference loss, respectively. , , e These are the parameters for the dynamic weighting coefficients. step These are parameters for determining whether to train in combination with the source domain. epochs This is the preset total training period.

Images