Transfer learning method considering non-shared class samples

Abstract

The application discloses a kind of transfer learning methods considering non-common category sample, it is related to bearing fault diagnosis method technical field.The method includes the following steps: first, the original vibration signal is processed into graph sample;Then it is extracted using recursive multi-head graph attention residual network;Finally, using MDD dynamic migration mode, the source domain feature is migrated to target domain, wherein, in the process of dynamic migration, non-target domain class filter is used to realize non-target domain class migration;The method can improve the utilization efficiency of big data source domain and fault diagnosis precision.

Description

Technical Field

[0001] This invention relates to the field of bearing fault diagnosis methods, and in particular to a transfer learning method that considers samples of non-common categories. Background Technology

[0002] Wheelset bearings are one of the key components of high-speed trains. During operation, they are susceptible to high loads and high speeds, leading to localized wear and damage. Wheelset bearing failure can cause significant vibration and noise, and even catastrophic consequences such as sudden stops and derailments. Therefore, fault diagnosis of high-speed train wheelset bearings is crucial for ensuring the safe and reliable operation of trains.

[0003] With the development of artificial intelligence technology, deep learning-based bearing fault diagnosis methods are gaining increasing attention due to their ability to automatically extract features and classify faults without requiring additional feature engineering. The improvement in industrial big data and cloud computing capabilities has also significantly enhanced the automation, reliability, and accuracy of end-to-end fault diagnosis algorithms. Fault diagnosis models, represented by convolutional neural networks, heavily rely on large amounts of labeled data. Currently, it is difficult to collect large quantities of fault data for wheelset bearings. This makes it difficult for such models to achieve the expected fault diagnosis accuracy. Furthermore, deep learning models require that the training and testing datasets have the same spatial distribution for fault diagnosis. However, in actual industrial applications, the different operating conditions of wheelset bearings, such as speed and load, result in significant differences in the spatial distribution of the acquired samples, making models trained under a single operating condition unsuitable for other conditions.

[0004] To address the aforementioned issues, transfer learning has been applied to the field of bearing fault diagnosis, resolving problems that deep learning cannot effectively solve. Transfer learning offers two advantages in bearing fault diagnosis. First, it effectively addresses the unsupervised adaptation problem, enabling the efficient use and diagnosis of collected unlabeled samples. Second, it can fit sample features under different operating conditions, allowing for the effective diagnosis of fault samples across different distribution spaces. While existing transfer learning methods can learn from one source domain to another target domain, the number of samples in the target domain is typically comparable to that in the source domain during training. However, in industrial applications, source domain samples are usually collected from a single, concentrated tracking experiment or laboratory data. Target domain data is typically collected only after a fault diagnosis operation has begun. Therefore, the operating conditions, fault types, and sample size of the source domain data are far greater than those of the target domain. These issues limit the practicality of current transfer learning models.

[0005] By analyzing the above problems, it is not difficult to find that there are two main problems in transfer learning that need to be solved. (1) How to make full use of existing samples and the effective information contained therein. (2) How to increase the applicability of transfer learning models using only a single model. Summary of the Invention

[0006] The technical problem to be solved by this invention is how to provide a transfer learning method that considers non-common class samples, which can improve the utilization efficiency of big data domains and the accuracy of fault diagnosis.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a transfer learning method considering samples of non-shared categories, comprising the following steps:

[0008] First, the original vibration signal is processed into a graphic sample;

[0009] Then, a recursive multi-head graph attention residual network is used to extract features from it;

[0010] Finally, the MDD dynamic transfer method is used to transfer the source domain features to the target domain;

[0011] In this process, a non-target domain category filter is used to achieve non-target domain category migration during dynamic migration.

[0012] A further technical solution involves using a weighted e-RN algorithm to process the original vibration signal into graphical samples, specifically including the following steps:

[0013] The vibration signal of the bearing is a nonlinear time series x with n points. x is divided and truncated with the same time delay to obtain m points. A set of vectors of dimension is represented as follows:

[0014]

[0015] The recursion graph G of the vibration signal is represented by a two-dimensional recursion matrix R. The degree of recursion between two vectors is obtained by calculating the arbitrary norm between them. A threshold parameter ε is set as the threshold for determining whether recursion has occurred. The recursion matrix obtained by calculation can represent whether two vectors are a recursive pair. The calculation method of the recursion matrix is ​​as follows:

[0016]

[0017] in Denote the phase space norm of two vectors; cosine similarity is used. As a metric; Let Hevi-Synes function be defined as: If the two vectors are a recursive pair, then =1, otherwise =0;

[0018] Since graph neural networks require removing self-loops from the recursive matrix R, the recursive matrix R is transformed into an adjacency matrix A for computation, as defined below:

[0019]

[0020] in, For having The identity matrix of dimension; the recursive graph G is composed of the vertex set G. and the set of edges connecting vertices Composition; of which Represents a vertex. Let p and q represent the number of vertices and the number of edges, respectively.

[0021] The connection density ρ of the recursive graph is defined as follows: ρ is used to adaptively adjust e for each graph, where e is a parameter in the e-RN algorithm.

[0022]

[0023] Where a represents an element of the adjacency matrix A of the recursive graph; We can analyze the proportion of edges in the recursive graph to the total number of edges in a fully connected graph, thus ensuring that the connection density parameter of the constructed recursive graph remains consistent. Since the recursion range is different for each graph group, we first need to obtain the degree of recursion between each pair of vectors in the graph, calculated as follows:

[0024]

[0025] with maximum value and minimum value As a range of values, the calculation enables the connection density to be related to the set value. The closest ε is used as the parameter for this recursive graph;

[0026]

[0027] To extract as much prior knowledge as possible from the samples, we propose weighting the recursive graph using the norm information of the vertex set. First, we obtain the phase space norm matrix L of the vertex set, defined as follows:

[0028]

[0029] in Represents the phase space norm matrix Elements in the matrix; Normalized and adjacency matrix By multiplying the positions, we obtain the weighted recursive graph adjacency matrix. The calculation formula is as follows:

[0030]

[0031] because and Both are symmetric matrices, so multiplying the two matrices will give the result. A matrix; if two vertices are a recursive pair, then the value at the corresponding position in matrix Q is its spatial norm, otherwise it is 0.

[0032] A further technical solution involves constructing a recursive multi-head graph attention residual network, which includes the following steps:

[0033] The spatial distance between two vertices is calculated in the regenerated Hilbert space to give it a complete inner product space; the Gaussian kernel function is defined as follows:

[0034]

[0035] in It is the bandwidth of the kernel function; a multi-kernel Gaussian function requires first setting an initial... The Gaussian kernel function was calculated by obtaining k bandwidths at certain intervals and then averaging them as follows:

[0036]

[0037] Where r is the margin size. Since the graph attention mechanism focuses on edges, only recursive pairs with edges are calculated for attention coefficients during training; the attention coefficient refers to the attention magnitude of the recursive pair under the current training condition; the attention coefficient of each recursive pair is calculated, the attention matrix is ​​constructed, and training parameters are introduced:

[0038]

[0039] Where P is the propagation matrix of the proposed graph attention matrix. Let be the training parameters of the t-th layer; the proposed graph attention mechanism can be represented as:

[0040]

[0041] To improve the model's stability, it is extended to a multi-head graph attention mechanism; K independent attention mechanisms are used to analyze the input features, and these are concatenated as the output.

[0042]

[0043] in The aggregation process of the multi-head graph attention mechanism is as follows: (This indicates a connection; the process is as follows:)

[0044]

[0045] Based on the multi-head image attention network, a recursive multi-head image attention residual network is constructed. The residual network model is mainly divided into three parts: feature encoding part, feature extraction part, and classification part. A multilayer perceptron is used as the feature encoding part, with the same input and output dimensions.

[0046] We treat a graph attention layer, a sigmoid activation function, and a meanNorm layer as a single graph attention module. Each module aggregates the input features, and the output of each layer is as follows:

[0047]

[0048] Using the Sigmoid activation function and MeanNorm layers can suppress the oversmoothing of the model to some extent. ;

[0049] A residual block is introduced before the fully connected layer to connect the encoded features with the output features, which alleviates gradient vanishing and reduces overfitting. Finally, Softmax is used in the fully connected layer to obtain the final classification result.

[0050] The loss in the source domain is obtained, and then applied to a sample set in the target domain that has the same fault type as the source domain. The cross-entropy loss is used for calculation, namely:

[0051]

[0052] Where m refers to the number of categories. These are the sample labels. The output of the common category on the subnetwork is used to update parameters through the backpropagation algorithm.

[0053] A further technical solution involves using a non-target domain category sample filter to achieve non-target domain category transfer:

[0054] In the sub-network, the source domain sample set of non-target domain categories is compared based on the encoded features. The difference in Hilbert space distribution between the non-target domain samples and the target domain samples; the MMD algorithm is used to calculate the difference in Hilbert space distribution between the non-target domain samples and the target domain samples. With target domain graph samples The distribution distance can be obtained as follows:

[0055]

[0056] in Let k represent the spatial distribution distance between a vertex in the source domain and the target domain, and k be the Gaussian kernel function. The non-target domain category filter can classify samples into usable and unusable samples by setting a confidence filtering threshold γ; γ is set to 0.1. Therefore, the non-target domain category compensation loss can be obtained as follows:

[0057]

[0058] in The function is an indicator function; if the condition is met, the output is 1, otherwise it is 0, thus filtering out invalid compensation generated by low-confidence samples. Using the same method, the above filtering mechanism is applied to filter the cross-entropy loss of the samples, and the filtered loss of non-target domain category samples is obtained as follows:

[0059]

[0060] in, This refers to the output of non-shared categories on the subnetwork; therefore, it represents the total loss for non-target domain category classification. It should be noted that, and The losses are calculated separately, but they all share a common classifier. .

[0061] A further technical solution involves a method for transferring features from the source domain to the target domain, which includes the following steps:

[0062] From the perspective of classifiers and labels, a classifier mask matrix M is used to process the target domain samples. The values ​​of the mask matrix M are as follows:

[0063]

[0064] When assigning labels to source domain samples, one-hot encoding is used; labels for common categories are placed first, and labels for non-common categories are placed last.

[0065] The Marginal Domain Discrepancy (MDD) algorithm is used during migration.

[0066] Adversarial domain differential transfer uses two classifiers and of The divergence is used to measure the difference in distribution between two datasets; the difference between two classifiers lies in... The classification loss of the model is obtained through normal classification. Marginal distribution differences are captured through gradient flipping; adversarial transfer is achieved by minimizing the classification loss and maximizing the marginal distribution differences; therefore, the loss of the proposed model can be integrated as follows:

[0067]

[0068] Where 1 represents the indicator function for the two datasets, used to measure the two classifiers respectively. and In the dataset and The prediction difference is E, which represents the cross-entropy loss between the two datasets. and This represents the output after passing through the feature extraction network and classifier, where the target domain needs to be processed using a mask matrix M, i.e.:

[0069]

[0070]

[0071] in, This represents the positional multiplication of two vectors; when MDD is used as a measure of prediction difference, it is:

[0072] in, The argmax expression represents the MDD loss between the source and target domains; argmax represents... The predicted labels of the classifier; m is a hyperparameter of MDD, representing the minimum margin of the marginal distribution, used to balance generalization and optimization; by minimizing the loss This can maximize the similarity of samples within the same category, thus enabling a model trained in the source domain to adapt to data in the target domain.

[0073] The beneficial effects of adopting the above technical solution are as follows: The method described in this application uses an MMD-weighted recurrent multi-head graph attention residual network (ResGANet) for feature extraction of wheelset bearing samples, which can explicitly fit the relationship between samples and improve the extraction effect of effective features. A non-common class sample transfer mechanism is proposed, so that samples with additional fault types in the source domain can be effectively utilized. A spatial distribution difference weighting strategy is used to filter samples that will produce negative transfer, thereby improving the utilization rate of non-common class samples. The method is used for fault diagnosis of high-speed train wheelset bearings, improving the utilization efficiency of big data domain and the accuracy of fault diagnosis. Attached Figure Description

[0074] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0075] Figure 1 This is the transfer learning network structure diagram (NSCTLN) that considers samples of non-shared categories as described in the embodiments of the present invention.

[0076] Figure 2 This is a weighted recursive graph in an embodiment of the present invention;

[0077] Figure 3 This is a diagram of the feature aggregation process of the graph attention mechanism in an embodiment of the present invention;

[0078] Figure 4 This is a physical image of the high-speed train wheelset bearing integrated test bench in an embodiment of the present invention;

[0079] Figure 5 This is a detailed diagram of the experimental platform model and its structure in an embodiment of the present invention;

[0080] Figure 6 These are photographs of bearings in six different health states as described in this invention embodiment;

[0081] Figures 7a-7b This is a graph showing the experimental results of parameters in an embodiment of the present invention;

[0082] Figure 8 This is a graph showing the experimental results of five experiments in an embodiment of the present invention.

[0083] Figure 9 This is a graphical representation of the experimental results in the embodiments of the present invention. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0085] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0086] Migration problem description

[0087] Given a spatially distributed source domain Among them, the source domain Includes Class of faults, and contains T labeled samples. ,in .in This represents the training samples and their corresponding labels. It also provides the target domain, which has only one spatial distribution. He was from a wheelset bearing that needed diagnosis, which contained... This type of failure. It is important to note that... Target domain It contains t unlabeled samples ,in ,and Then, define the sample set of label categories common to both the source and target domains in the source domain as follows: The sample set of categories not included in the target domain is .so All contained in the source domain A sample set of fault types. The source domain and the target domain have different probability distributions P and Q. Therefore... In addition, it is also believed that... Therefore, this application establishes a fault diagnosis method that utilizes existing data as much as possible, capable of [from...] Towards To transfer knowledge. The domain adaptation process of transfer learning is as follows: Figure 1 As shown.

[0088] Analysis of the above dataset reveals the following challenges in diagnosing wheelset bearing migration faults: 1) Significant spatial distribution differences may exist between the source and target domains. 2) Samples from non-common categories may cause negative migration. The first challenge stems primarily from the substantial signal differences in wheelset bearings under various operating conditions. When sufficient training samples are unavailable, other types of rotating machinery may even be considered as source domain samples. This places higher demands on feature extraction accuracy and the comprehensiveness of information obtainable from the samples. Enabling the model to explicitly acquire relationships between samples can significantly increase the prior knowledge acquired by the model. The second challenge arises from the fact that some samples from non-common fault categories may generate negative migration. Employing a filtering mechanism will increase the weight of useful samples and remove samples that may cause negative migration. By combining these objectives, the existing samples can be fully utilized to improve the diagnostic performance of the wheelset bearing migration model.

[0089] Graph Convolutional Networks

[0090] Most fault feature extraction methods are based on Euclidean space, neglecting information from non-Euclidean space data. Graph samples, as a type of non-Euclidean space data, contain more information. Given the limited availability of large-scale locomotive wheelset bearing data, extracting more fault information from graph samples allows for the construction of a more reliable feature extraction network. Graph Neural Networks (GNNs) are models that can explicitly fit structured data from raw samples. They have gained widespread attention because they improve classification accuracy by aggregating node relationships in graph data. GNNs can also be interpreted as an extension of convolutional operations on a two-dimensional network. Typically, iterations in graph convolutional networks are considered further aggregations of the output features of the previous layer. GNNs fit their own features and neighbor node information, outputting the classification result for each vertex.

[0091] Define a graph with n nodes. .in , where is the vertex, representing the original information of each sample. The edges between vertices represent the relationships between samples. If there is an edge between two vertices, it means that the two samples are related under the decision condition. The adjacency matrix of G can be obtained through E. If there is an edge between two vertices, the element at the corresponding position is 1. A typical GNN usually alternates between a forward propagation layer and a perceptron. Represents the features of the t-th layer. Let be the propagation matrix of this layer. Therefore, a single-layer propagation layer can be defined as:

[0092]

[0093] in, This represents the output of the propagation layer. The propagation matrix here... , where D is the degree matrix of graph G, representing the number of edges each element has. Its elements can be computed by A. This method employs a local averaging operation. A typical propagation layer reflects the adjacency pattern of a graph G. The propagation layer is embedded into a GNN, and the perceptron parameter matrix is ​​constructed. Then the output of layer t+1 can be defined as:

[0094]

[0095] in, It has the same dimensions as the adjacency matrix. is the activation function. Invariant properties of graph samples are learned by iterating through multiple propagation and perceptron layers. Even if two samples are significantly different, they will still be classified into the same class due to their similar neighboring vertex features and structure. Typical GNNs have proven to be very powerful in multiple fields. Graph Convolutional Networks, developed based on this, are lightweight yet powerful networks. A typical GCN is a network with two propagation layers and a perceptron, which can be represented as:

[0096]

[0097]

[0098] Propagation matrix in graph convolutional networks ,in , It is an identity matrix. Graph Convolutional Networks (GCNs) are inspired by signal processing. Since the number of neighbors of a vertex in the graph space is not fixed and there is no absolute order relationship, GCNs transform the graph into the spectral domain using the Laplacian matrix for convolution operations. Although GCNs have demonstrated strong classification capabilities, in wheelset bearing fault diagnosis, due to the lack of clear inter-sample relationships, a more interpretable algorithm is first needed to construct graph samples from the vibration signals. Furthermore, the weights between edges in a GCN are fixed, making it impossible to change the importance of edges in a timely manner during training. Based on these issues, this application proposes a weighted e-RN and MMD-weighted graph attention network as a graph sample construction method and feature extraction tool to improve the accuracy and effectiveness of feature extraction for high-speed train wheelset bearings.

[0099] In general, this invention discloses a transfer learning method that considers samples with non-shared classes. The method uses a transfer learning network that considers samples with non-shared classes (NSCTLN), the structure of which is as follows: Figure 1 As shown, the method includes the following steps:

[0100] Before feature extraction, the original vibration signal needs to be processed into graph samples (using a weighted e-RN algorithm). Then, the proposed recurrent multi-head graph attention mechanism is used for feature extraction (using a recurrent multi-head graph attention residual network). To avoid the negative transfer problem caused by non-target domain categories, a non-target domain category filter is proposed to improve the transfer effect. Finally, to address the problem of the small number of target domain samples for wheelset bearings, the MDD transfer method is used to transfer the source domain features to the target domain.

[0101] Weighted e-RN

[0102] A graph is a data structure that can describe specific relationships between samples. There are multiple methods for constructing graph samples for each type of data. However, unlike other data with inherently defined topological structures, bearing vibration signals do not possess a natural topological structure, thus requiring an algorithm to describe the physical meaning between vibration signal samples. Vibration signals are a typical nonlinear time series, and the nonlinear structure between samples can be obtained using complex network analysis methods. e-RN is a nonlinear time series complex network analysis method based on phase space; it studies the recursiveness of time series in phase space from a geometric perspective and constructs a recursive graph. This application takes the recursiveness of vibration signals as its starting point and proposes a weighted e-RN algorithm.

[0103] The recursiveness of a nonlinear time series refers to the fact that after a certain time, a signal will always have a previous state that is the same as or similar to the current state. Therefore, the purpose of weighted e-RN is to analyze whether recursion occurs between two signals. The vibration signal of an existing bearing is a nonlinear time series x with n points. x is divided and truncated with the same time delay. This yields m points with... A set of vectors of dimension is represented as follows:

[0104]

[0105] The recursion graph G of a vibration signal can be represented by a two-dimensional recursion matrix R to study the recursiveness of the phase space trajectory of the vibration signal. The degree of recursion between two vectors is obtained by calculating the arbitrary norm between them. A threshold parameter ε is set as the threshold for determining whether recursion has occurred. The recursion matrix obtained by calculation can represent whether two vectors are a recursive pair. The calculation method of the recursion matrix is ​​as follows:

[0106]

[0107] in Let represent the phase space norm of two vectors. This application uses cosine similarity. As a metric. The heaviside function is defined as follows: If the two vectors are a recursive pair, then It is 1 if it is not 1, otherwise it is 0.

[0108] Since graph neural networks require operations to remove self-loops from the recursive matrix R, R is transformed into an adjacency matrix A for the operations, as defined below:

[0109]

[0110] in, Let G be an identity matrix of dimension c. The recursive graph G is formed by the set of vertices G = c * 1. and the set of edges connecting vertices Composition. Among them... Represents a vertex. Let p represent the number of vertices and q, respectively.

[0111] The above analysis shows that 'e' is a crucial parameter in the e-RN algorithm, affecting the recursion degree of the graph. A value that is too large or too small will affect the usability of the acquired recursive graph. However, the value of 'e' is usually largely based on experience. Furthermore, factors such as rotational speed, load, and sampling frequency during vibration signal acquisition all influence the value of 'e', ​​requiring individual adjustment for each graph. Therefore, the value of 'e' cannot be standardized. To address this issue, this application proposes using the connection density ρ of the recursive graph as a unified index for recursive graph samples, adaptively adjusting 'e' for each graph by setting ρ. The definition of the connection density ρ of the recursive graph is as follows:

[0112]

[0113] Where 'a' represents an element of the adjacency matrix A of the recursive graph. This allows us to analyze the proportion of edges in the recursive graph relative to the total number of edges in a fully connected graph. This ensures that the connection density parameters of the constructed recursive graph remain consistent. Since the recursion range differs for each graph group, we first need to determine the degree of recursion between each pair of vectors in the graph, calculated as follows:

[0114]

[0115] with maximum value and minimum value As a range of values, the calculation enables the connection density to be related to the set value. The closest ε is used as the parameter for this recursive graph.

[0116]

[0117] The original recursive graph only retains information about recursive pairs within the vertex set, but it fails to preserve the rich information about the degree of recursion during computation. Graph neural networks can explicitly fit this information. To capture as much prior knowledge as possible from the samples, this application proposes weighting the recursive graph with the norm information of the vertex set. First, the phase space norm matrix L of the vertex set is obtained, defined as follows:

[0118]

[0119] in Represents the phase space norm matrix The elements in the matrix. Normalized and adjacency matrix By multiplying the positions, we obtain the weighted recursive graph adjacency matrix. The calculation formula is as follows:

[0120]

[0121] because and Both are symmetric matrices, so multiplying the two matrices will give the result. A matrix. If two vertices are a recursive pair, the value at the corresponding position in matrix Q is its spatial norm; otherwise, it is 0. The weighted recursive graph and matrix Q are shown below. Figure 2 As shown.

[0122] Recursive Multi-Head Graph Attention Residual Network

[0123] Introducing attention mechanisms is one of the important development directions of graph neural networks. The key to attention mechanisms is analyzing the importance of edges in the network, allowing the network to focus on those important edges during training. However, existing graph attention networks still have some problems. First, the graph samples they target are usually asymmetric, but the proposed weighted recursive graph is a typical symmetric graph, which leads to redundancy in training parameters when fitting symmetric graph samples. Second, graph neural networks typically use Euclidean distance as a metric, which is a typical method for calculating inner product space and lacks completeness. When the noise component or variation trend of wheelset bearings is large, it can cause excessive bias in the attention matrix. Therefore, this application proposes a multi-kernel Gaussian kernel graph attention mechanism, which calculates the spatial distribution distance between two vertices in the regenerated Hilbert space, giving it a complete inner product space. The definition of the Gaussian kernel function is as follows:

[0124]

[0125] in This refers to the bandwidth of the kernel function. A multi-kernel Gaussian function requires first setting an initial... The Gaussian kernel function was calculated by obtaining k bandwidths at certain intervals and then averaging them as follows:

[0126]

[0127] Where r is the margin size. Since the graph attention mechanism focuses on edges, only recursive pairs with existing edges will have their attention coefficients calculated during training. The attention coefficient refers to the magnitude of attention for the recursive pair under the current training condition. The attention coefficients for each recursive pair are calculated, an attention matrix is ​​constructed, and training parameters are introduced:

[0128]

[0129] Where P is the propagation matrix of the proposed graph attention matrix. Let be the training parameters for the t-th layer. The proposed graph attention mechanism can be represented as:

[0130]

[0131] To improve the model's stability, it is extended to a multi-head graph attention mechanism. K independent attention mechanisms are used to analyze the input features, and these are concatenated as the output.

[0132]

[0133] in This indicates a connection. It's important to note that the final layer input of the multi-head attention mechanism needs to use an average value to maintain consistency between the output dimension and the input dimension. The aggregation process of the multi-head graph attention mechanism is as follows: Figure 3 As shown.

[0134]

[0135] Based on multi-head graph attention networks, a recursive multi-head graph attention residual network is constructed. As shown in the figure, the model mainly consists of three parts: feature encoding, feature extraction, and classification. To enhance the feature representation ability of samples and provide features as a non-target domain category filter in subsequent steps, this application utilizes a multilayer perceptron as the feature encoding part. Its input and output dimensions are the same.

[0136] We treat a graph attention layer, a sigmoid activation function, and a meanNorm layer as a single graph attention module. Each module aggregates the input features, and the output of each layer is as follows:

[0137]

[0138] Using the Sigmoid activation function and MeanNorm layers can suppress the oversmoothing of the model to some extent. .

[0139] A residual block is introduced before the fully connected layer to connect the encoded features with the output features, mitigating gradient vanishing and reducing overfitting. Finally, Softmax is applied in the fully connected layer to obtain the final classification result.

[0140] Similar to other models, the loss is obtained from the source domain, and then applied to a sample set with the same fault type in both the target and source domains. The cross-entropy loss is used for calculation, namely:

[0141]

[0142] Where m refers to the number of categories. These are the sample labels. This refers to the output of the common category on the subnetwork. Parameters are updated using the backpropagation algorithm.

[0143] Non-target domain category migration

[0144] To fully utilize existing data and provide more fault information for high-speed train wheelset bearing migration, this application proposes to apply non-target domain category samples from the source domain to the model. However, the features of these samples differ significantly from the target domain; if used directly, the calculated loss value will cause negative migration in the model. Therefore, this application proposes a non-target domain category sample filter.

[0145] In the sub-network, this application compares source domain sample sets that are not of the target domain category based on the encoded features. The difference in Hilbert space distribution between the non-target domain samples and the target domain samples is analyzed. The MMD algorithm is used to calculate the difference in Hilbert space distribution between the non-target domain samples and the target domain samples. With target domain graph samples From the distribution distance, we can obtain the following:

[0146]

[0147] in Let represent the spatial distribution distance between a vertex in the source domain and the target domain, and k be the Gaussian kernel function. The non-target domain category filter, by setting a confidence threshold γ, can classify samples into usable and unusable samples. In this application, γ is set to 0.1. Therefore, the non-target domain category compensation loss can be obtained as:

[0148]

[0149] in The function is an indicator; it outputs 1 if the condition is met, and 0 otherwise. This filters out invalid compensation from low-confidence samples. Simultaneously, this part of the loss can reduce the negative transfer effect of non-target domain category samples in the graph attention layer. During training, the model fully learns from non-target domain category samples, and its loss comes from the labels. However, low-confidence samples can cause significant negative transfer to the model, leading to error accumulation. Using the same method, the above filtering mechanism is applied to filter the cross-entropy loss of the samples, resulting in the filtered loss for non-target domain category samples as follows:

[0150]

[0151] in, This refers to the output of non-shared categories on the subnetwork. Therefore, it represents the total loss for non-target domain category classification. It is important to note that... and The losses are calculated separately, but they all share a common classifier. Non-shared domain category sample filters can reduce the impact of low-confidence samples during the initial domain adaptation phase of the model and improve the utilization of useful fault features.

[0152] Target domain sample transfer

[0153] The proposed model has two main characteristics during transfer learning. First, because the source domain dataset has more categories than the target domain dataset, and both share the same classifier, special processing is required. Second, large-scale collection of high-speed train wheelset bearing fault signals is difficult, resulting in a smaller overall sample size compared to the source domain.

[0154] To address the issue of the inability to share a single classifier between the source and target domains, this application takes a classifier and label-based approach, employing a classifier mask matrix M to process the target domain samples. The values ​​of the mask matrix M are as follows:

[0155]

[0156] When labeling source domain samples, one-hot encoding is used. Labels for common categories appear first, and labels for non-common categories appear last. Therefore, the label encoding for target domain samples will not have any last few bits set to 1. The classifier's output for target domain samples should not have excessively high probabilities for non-common categories. Failure to address this may affect the final classification result.

[0157] To address the issue of a relatively small total sample size in the target domain, instead of using minimum difference as a measure of probability distribution difference between the source and target domains during transfer learning, Marginal Domain Discrepancy (MDD) was employed. Compared to MMD, MDD focuses only on the mean difference within the same class when measuring distance, thus better utilizing information from a small number of samples of the same class for classification. MMD, on the other hand, requires calculating the distance between all samples, which may lead to inaccurate measurement of differences between samples in the case of small samples. Furthermore, MDD is a distance metric based on marginal distribution, directly measuring the difference between two domains and avoiding interference between different domains.

[0158] Adversarial domain differential transfer uses two classifiers and of The divergence is used to measure the difference in distribution between two datasets. As can be seen from the graph, the difference between the two classifiers lies in... The classification loss of the model is obtained through normal classification. Gradient flipping is used to capture the marginal distribution difference. Adversarial transfer is achieved by minimizing the classification loss and maximizing the marginal distribution difference. Therefore, the loss of the proposed model can be integrated as follows:

[0159]

[0160] Where 1 represents the indicator function for the two datasets, used to measure the two classifiers respectively. and In the dataset and The prediction difference is E, which represents the cross-entropy loss between the two datasets. and This represents the output after passing through the feature extraction network and classifier, where the target domain needs to be processed using a mask matrix M, i.e.:

[0161]

[0162]

[0163] in, This represents the positional multiplication of two vectors. When MDD is used as a measure of prediction difference, it is:

[0164] in, This represents the MDD loss between the source and target domains. argmax represents... The predicted labels of the classifier. `m` is a hyperparameter of the marginal distribution (MDD), representing the minimum difference between the marginal distributions, used to balance generalization and optimization. This is achieved by minimizing the loss... This can maximize the similarity of samples within the same category, thus enabling a model trained in the source domain to adapt to data in the target domain. It helps the model learn a domain-invariant representation, allowing the model to achieve good performance in both the source and target domains.

[0165] Experimental Analysis:

[0166] High-speed train wheelset bearing experimental dataset:

[0167] The experimental dataset was primarily collected from a comprehensive experimental platform for high-speed train wheelset bearings. Photos of the experimental platform are shown below. Figure 4 As shown in the figure, the experimental platform mainly consists of a motor, a loading device, a support bearing, and the bearing under test. The sensors used in the experiment and related details are as follows. Figure 5 As shown. During the experiment, the loading device applied positive pressure to the wheelset bearings via the hydraulic device on the upper part of the experimental platform. Then, the wheelset was rotated by a drive wheel driven by a motor at the bottom, simulating on-rail travel. An accelerometer (356A25) with a sensitivity of 2.505 mv / m / s² was installed above the experimental bearing to collect vibration signals during operation. The sampling frequency for signal acquisition was set to 12.8 kHz. The bearings used in the experiment mainly included six health states: inner ring fault (IR), outer ring fault (OR), rolling element fault (RE), outer ring-rolling element combined fault (OR), inner ring-outer ring combined fault (IO), and no fault (NC). The experimental bearings are as follows. Figure 6 As shown.

[0168] Four experimental sample sets were collected using the above-mentioned bearings under four operating conditions, and the constructed datasets are shown in the table. All datasets contain data for all health conditions. However, since the probability of the two combined faults occurring in actual operation is relatively low, the dataset for the target domain only includes four health conditions (IR, OR, RE, NC). The main parameters adjusted under each operating condition are operating speed and working load. The collected vibration signals were segmented using a sliding window. Each sample has 1024 data points. 2000 samples were obtained for each health condition when used as source domain samples. Therefore, a total of 12000 samples were obtained for the six faults. These samples were used to construct a weighted recurrence graph using weighted e-RN. 200 samples were randomly selected from the samples to construct a recurrence graph. It should be noted that samples in a recurrence graph have the same fault type. In the source domain, 48 recurrence graphs were used for training, and 12 recurrence graphs were used for testing. 1600 samples were obtained for each health condition when used as target domain samples. Therefore, a total of 6400 samples were obtained for the four types of faults. There are 24 recursive graphs in the target domain used for training and 8 recursive graphs used for testing.

[0169] Table 1. Experimental Data Set for Wheelset Bearings

[0170]

[0171] Model parameter analysis experiment

[0172] It is not difficult to see from the loss value that the hyperparameters , This affects the model's fault diagnosis accuracy. To investigate its impact on diagnostic accuracy, an experiment was designed to study the model's diagnostic accuracy under different parameters. These two parameters control the filtering of non-shared class samples and the degree of adaptive transfer of the adversarial domain, respectively. First, by observing the output accuracy, the model's diagnostic accuracy was analyzed. Impact on model diagnostic performance. Coefficient of adversarial transfer. We'll temporarily use 0.1 for the experiment. Then we'll obtain... After finding the optimal value, then... The experiment was conducted. During training, the learning rate was set to 0.0002. Each training iteration used one recursive graph, therefore the mini-batch size was 200. and The values ​​are selected from {0, 0.001, 0.01, 0.1, 0.5, 1}. Each value is trained 5 times and the average is taken to eliminate random error. DatasetA is used as the source domain and DatasetB as the target domain for the experimental samples. The proposed method is written in Python 3.7. The deep learning model uses PyTorch 1.3. The hardware platform for training the model is an Intel Xeon Gold 6148 CPU and an RTX 2080 Ti GPU. Experimental results are as follows: Figures 7a-7b As shown.

[0173] from Figure 7a It can be seen from this that The accuracy rate reaches its highest at 0.1, at 91.32%. The model exhibits the worst diagnostic performance when the value is 0.0001. The overall trend of the experimental results is a gradual increase followed by a gradual decrease. This indicates that the non-common class filtering loss can reduce negative transfer, but when the loss value is too large, the introduced loss offsets the benefits. In this case, the proposed model may not achieve optimal training. Then... Set to 0.1, for Conduct the test. From Figure 7b It can be seen from this that A value of 0.01 maximizes the model's accuracy to 94.37%. The overall trend of the experimental results is a gradual increase followed by a decrease. This indicates that when the value is too small, the model prioritizes source domain classification accuracy, reducing its domain adaptation ability during adversarial transfer. Conversely, when the value is too large, the model pays insufficient attention to the source domain, resulting in incomplete fitting of shared fault features between the source and target domains. In conclusion, and All values ​​need to be taken within a reasonable range to balance the relationship between the source domain and the target domain, as well as between common and non-common categories, in order to improve fault diagnosis performance.

[0174] Classification accuracy comparison experiment

[0175] To analyze the effectiveness and superiority of the model under various working conditions, this application compared it with five other models: WDCNN, DANN, CORAL, and DAGCN. WDCNN is a widely used and effective model for bearing vibration signal fault diagnosis. It is built upon a convolutional neural network. It uses a large kernel in the first convolutional layer to capture the impact features of the vibration signal, and then uses a small kernel to further extract the fault features. Since it lacks a transfer learning strategy, it directly uses samples from the target domain for training and testing. Furthermore, convolutional neural network-based models cannot explicitly fit the relationships between samples; therefore, these models are trained using only the samples themselves. DANN is a standard adversarial network-based transfer learning model, consisting of a domain classifier and a sample classifier. Similarly, DANN is trained using only the samples themselves. CORAL is a convolutional neural network-based transfer learning model. It uses CORAL loss to adjust the features between the source and target domains. DAGCN is a bearing fault diagnosis method based on MMD and multi-receptive-domain graph convolutional networks. It uses domain adversarial transfer to classify samples from the target domain. It consists of three multi-receptive-domain graph convolutional layers and a domain adversarial classifier. DAGCN, due to its ability to explicitly fit relationships between samples, is trained using a recursive graph. Furthermore, this application analyzes the accuracy of different parts of the network. It also studies training the target domain samples using only a graph attention network (ResGANet), masking the non-shared domain compensation loss (NSCTLN). Changes in diagnostic accuracy after =0). Due to parameter This is a parameter used in NSCTLN for adversarial domain adaptation. If it is masked, the migration cannot be completed, so this case will not be studied.

[0176] Based on the experimental set and model type, four sets of experiments were designed: A->B, B->C, C->D, and D->A. Since WDCNN lacks a transfer strategy, source domain samples were used for training and target domain samples for testing. The same classifier masking method as the proposed model was used to achieve transfer from six health statuses to four. Each model was trained five times to eliminate error values. The average fault diagnosis accuracy on the test set was used as the final result. The standard deviation of the test set accuracy was used as the metric for model stability. The experimental results are shown in Table 2. The experimental results for Task A->B of WDCNN, ResGANet, DANN, CORAL, DAGCN, and NSCTLN are shown in Table 2. Figure 8 As shown, the view is as follows Figure 9 As shown.

[0177] Table 2 Comparison of experimental results

[0178]

[0179] As can be seen from the table, the proposed method significantly outperforms the other five comparative methods in the wheelset bearing fault diagnosis task under different working conditions. The accuracy of all four transfer tasks exceeds 97%, indicating that the proposed NSCTLN has good adaptability to transfer between the same machine and different working conditions. NSCTLN's feature distribution boundaries are clear, with fewer misclassified samples. The multi-channel decision mechanism enhances effective sample features, greatly improving the model's training stability. The diagnostic accuracy of NSCTLN's three channels reaches 96%, 93%, and 94%, respectively. This demonstrates that non-common class samples, under the dual effect of filters and masks, can be effectively used for feature extraction. Experiments removing non-common class sample filter loss also verify this conclusion. When the value is 0, the accuracy of the multi-channel model decreases to some extent, indicating that unfiltered non-shared class samples will cause some negative transfer to the model.

[0180] In the other five comparative experiments, although the transfer effect reached about 90%, it still lagged behind the proposed NSCTLN. In the non-domain adaptive methods WDCNN and ResGANet, the diagnostic accuracy for all tasks was poor, indicating a significant difference in feature distribution between the source and target domains, making direct cross-condition diagnosis impossible. As a popular passive domain adaptive method, DANN showed some effectiveness in transferring data across different conditions, but its accuracy was still significantly lower than the proposed NSCTLN. The CORAL method showed some improvement in accuracy compared to DANN. However, its accuracy was significantly affected by negative transfer due to its inability to properly handle non-shared class samples. DAGCN, by explicitly fitting the relationships between samples, improved the model's diagnostic robustness, resulting in relatively high accuracy among the comparative models, but still falling short of the proposed model. Furthermore, its unique multi-receptor domain structure led to relatively high computational cost. In summary, the proposed NSCTLN demonstrated better performance in filtering non-shared class samples.

Claims

1. A transfer learning method considering samples with non-shared categories, characterized in that... Includes the following steps: First, the original vibration signal is processed into a graphic sample; Then, a recursive multi-head graph attention residual network is used to extract features from it; Finally, the MDD dynamic transfer method is used to transfer the source domain features to the target domain to achieve fault identification in the target domain. The bearing dataset of the operating condition in the source domain includes 6 health states: inner ring fault (IR), outer ring fault (OR), rolling element fault (RE), outer ring-rolling element combined fault (OR), inner ring-outer ring combined fault (IO), and no fault (NC). The bearing dataset of the operating condition in the target domain includes 4 health states: inner ring fault (IR), outer ring fault (OR), rolling element fault (RE), and no fault (NC). In particular, a non-target domain category filter is used to achieve non-target domain category migration during the dynamic migration process; The method for transferring features from the source domain to the target domain includes the following steps: From the perspective of classifiers and labels, a classifier mask matrix M is used to process the target domain samples. The values ​​of the mask matrix M are as follows: ； When assigning labels to source domain samples, one-hot encoding is used; labels for common categories are placed first, and labels for non-common categories are placed last. The MDD algorithm is used during transfer, and two classifiers are used for adversarial domain differential transfer. and of The divergence is used to measure the difference in distribution between two datasets; the difference between two classifiers lies in... The classification loss of the model is obtained through normal classification. Marginal distribution differences are captured through gradient flipping; adversarial transfer is achieved by minimizing the classification loss and maximizing the marginal distribution differences; therefore, the loss of the proposed model can be integrated as follows: ； Where 1 represents the indicator function for the two datasets, used to measure the two classifiers respectively. and In the dataset and The predicted difference in the dataset is E, which represents the cross-entropy loss between the two datasets. and This represents the output after passing through the feature extraction network and classifier, where the target domain needs to be processed using a mask matrix M, i.e.: ； ； in, This represents the positional multiplication of two vectors; when MDD is used as a measure of prediction difference, it is: ； in, The argmax expression represents the MDD loss between the source and target domains; argmax represents... The predicted labels of the classifier; m is a hyperparameter of MDD, representing the minimum margin of the marginal distribution, used to balance generalization and optimization; by minimizing the loss This can maximize the similarity of samples within the same category, thus enabling a model trained in the source domain to adapt to data in the target domain.

2. The transfer learning method considering samples with non-shared categories as described in claim 1, characterized in that, The weighted e-RN algorithm is used to process the original vibration signal into graphical samples, which includes the following steps: The vibration signal of the bearing is a nonlinear time series x with n points. x is divided and truncated with the same time delay to obtain m points. A set of vectors of dimension 1 is represented as follows: ； The recursion graph G of the vibration signal is represented by a two-dimensional recursion matrix R. The degree of recursion between two vectors is obtained by calculating the arbitrary norm between the two vectors. A threshold parameter ε is set as the threshold for determining whether recursion has occurred; the recursion matrix is ​​calculated to represent whether two vectors are a recursive pair; the calculation method for the recursion matrix is ​​as follows: ； in Denote the phase space norm of two vectors; cosine similarity is used. As a metric; Let Hevi-Synes function be defined as: ; If the two vectors are a recursive pair, then =1, otherwise =0; Since graph neural networks require removing self-loops from the recursive matrix R, the recursive matrix R is transformed into an adjacency matrix A for computation, as defined below: ； in, The identity matrix is ​​of dimension c; the recursive graph G is composed of the vertex set G. and the set of edges connecting vertices Composition; of which Represents a vertex. Let p and q represent the number of vertices and the number of edges, respectively. The connection density ρ of the recursive graph is defined as follows: ρ is used to adaptively adjust e for each graph, where e is a parameter in the e-RN algorithm. ； Where a represents an element of the adjacency matrix A of the recursive graph; We can analyze the proportion of edges in the recursive graph to the total number of edges in a fully connected graph, thus ensuring that the connection density parameter of the constructed recursive graph remains consistent. Since the recursion range is different for each graph group, we first need to obtain the degree of recursion between each pair of vectors in the graph, calculated as follows: ； with maximum value and minimum value As a range of values, the calculation enables the connection density to be related to the set value. The closest ε is used as the parameter for this recursive graph; ； To extract as much prior knowledge as possible from the samples, we propose weighting the recursive graph using the norm information of the vertex set. First, we obtain the phase space norm matrix L of the vertex set, defined as follows: ； in Represents the phase space norm matrix Elements in the matrix; Normalized and adjacency matrix By multiplying the positions, we obtain the weighted recursive graph adjacency matrix. The calculation formula is as follows: ； because and Both are symmetric matrices, so multiplying the two matrices will give the result. A matrix; if two vertices are a recursive pair, then the value at the corresponding position in matrix Q is its spatial norm, otherwise it is 0.

3. The transfer learning method considering samples with non-shared categories as described in claim 1, characterized in that, The construction method of a recursive multi-head graph attention residual network includes the following steps: The spatial distance between two vertices is calculated in the regenerated Hilbert space to give it a complete inner product space; the Gaussian kernel function is defined as follows: ； in It is the bandwidth of the kernel function; a multi-kernel Gaussian function requires setting an initial value first. The Gaussian kernel function was calculated by obtaining k bandwidths at certain intervals and then averaging them as follows: ； Where r is the margin size; since the graph attention mechanism focuses on edges, only recursive pairs with edges will have their attention coefficients calculated during training; the attention coefficient refers to the attention magnitude of the recursive pair under the current training condition; Calculate the attention coefficients for each recursive pair, construct the attention matrix, and introduce training parameters: ； Where P is the propagation matrix of the proposed graph attention matrix. Let be the training parameters of the t-th layer; the proposed graph attention mechanism can be represented as: ； To improve the model's stability, it is extended to a multi-head graph attention mechanism; K independent attention mechanisms are used to analyze the input features, and these are concatenated as the output. ； in The aggregation process of the multi-head graph attention mechanism is as follows: (This indicates a connection; the process is as follows:) ； Based on the multi-head image attention network, a recursive multi-head image attention residual network is constructed. The residual network model is mainly divided into three parts: feature encoding part, feature extraction part, and classification part. A multilayer perceptron is used as the feature encoding part, with the same input and output dimensions. We treat a graph attention layer, a sigmoid activation function, and a meanNorm layer as a single graph attention module. Each module aggregates the input features, and the output of each layer is as follows: ； Using the Sigmoid activation function and MeanNorm layers can suppress the oversmoothing of the model to some extent. ; A residual block is introduced before the fully connected layer to connect the encoded features with the output features, which alleviates gradient vanishing and reduces overfitting. Finally, Softmax is used in the fully connected layer to obtain the final classification result. The loss in the source domain is obtained, and then applied to a sample set with the same fault type in both the target and source domains. The cross-entropy loss is used for calculation, namely: ； Where m refers to the number of categories. These are the sample labels. The output of the common category on the subnetwork is used to update parameters through the backpropagation algorithm.

4. The transfer learning method considering samples with non-shared categories as described in claim 1, characterized in that, Implement non-target domain category transfer using a non-target domain category sample filter: In the sub-network, the source domain sample set of non-target domain categories is compared based on the encoded features. The difference in Hilbert space distribution between the non-target domain samples and the target domain samples; the MMD algorithm is used to calculate the difference in Hilbert space distribution between the non-target domain samples and the target domain samples. With target domain graph samples The distribution distance can be obtained as follows: ； in Let k represent the spatial distribution distance between a vertex in the source domain and the target domain, and k be the Gaussian kernel function. The non-target domain category filter can classify samples into usable and unusable samples by setting a confidence filtering threshold γ; γ is set to 0.

1. Therefore, the non-target domain category compensation loss can be obtained as follows: ； in The function is an indicator function; if the condition is met, the output is 1, otherwise it is 0, thus filtering out invalid compensation generated by low-confidence samples. Using the same method, the above filtering mechanism is applied to filter the cross-entropy loss of the samples, and the filtered loss of non-target domain category samples is obtained as follows: ； in, This refers to the output of non-shared categories on the subnetwork; Therefore, the total loss for non-target domain category classification It should be noted that, and The losses are calculated separately, but they all share a common classifier. .

Images