Double-path incremental fault diagnosis method and system fusing attention mechanism and prototype

Abstract

The invention belongs to the technical field of fault diagnosis, and discloses a double-path incremental fault diagnosis method fusing an attention mechanism and a prototype, and the method comprises the steps: designing a double-branch network, and carrying out the division of labor through a heterogeneous network: extracting a time domain feature through a Hu moment-based convolutional neural network, extracting a frequency domain feature through a scale invariant feature conversion-based convolutional neural network, and carrying out the classification of the time domain feature and the frequency domain feature; the time domain branch network captures the global features of the operation trend of the equipment, and the frequency domain branch network extracts the local features of abnormal signals of a specific frequency band to form a time-frequency complementary feature space, thereby achieving the comprehensiveness of feature coverage. Considering that local frequency domain features of a high frequency band are generally more sensitive to early faults and time domain global features can visually reflect the overall operation state, an attention mechanism is introduced to realize adaptive distribution of weights, and the accuracy of feature extraction is further enhanced. Random projection is adopted to increase feature dimensions, the linear separability of features is improved, the problem of category overlapping in a low-dimensional feature space is solved, and category boundaries are expanded.

Description

Technical Field

[0001] The present invention belongs to the field of fault diagnosis technology, and in particular relates to a dual-path incremental fault diagnosis method and system integrating an attention mechanism and a class prototype. Background Art

[0002] In the field of industrial equipment fault diagnosis, deep neural networks are rapidly developing due to their ability to automatically mine fault-related knowledge and their high level of intelligence. However, existing deep learning-based diagnostic models are mostly built using offline operating data from one or several industrial equipment failures. These models are trained and applied to specific operating conditions, specific equipment models, and specific fault types. Due to fluctuations in equipment operating conditions, instrument aging, environmental noise, and newly added fault information, offline data struggles to fully capture the complex and fluctuating monitoring conditions and fault information. Models are prone to degradation and failure, necessitating retraining based on new data. In general, when data distribution differs, the diagnostic accuracy of existing diagnostic models in the new environment is significantly reduced or even completely ineffective. Updating the original model with new data can easily cause it to forget its original diagnostic knowledge, while completely retraining the model using both new and old data wastes significant resources.

[0003] Incremental learning can autonomously adjust network parameters based on time-varying data streams, providing a solution to the complete retraining problem and enabling sustainable learning. Incremental learning can be broadly categorized into replay-based, regularization-based, and network structure-based methods. Replay-based methods typically require a large amount of space to store previous data samples, enabling "review and learning" in subsequent incremental steps. Because they retain the original data, storage costs are high and data storage poses privacy risks. Regularization-based methods mitigate catastrophic forgetting by imposing constraints on the loss function. These methods employ regularization techniques to effectively suppress noise and enhance features, while constraining parameter updates to prevent overfitting. However, when new and old tasks differ significantly, constraints can make it difficult to align the feature space, leading to task-dependency dependencies. Network structure-based methods achieve incremental learning through dynamic adjustments to the network structure. Complex network models capture the coordinated failure modes of multiple devices through topological analysis of nodes and edges, achieving global feature extraction capabilities. However, these methods often scale linearly with the number of tasks, leading to parameter inflation. Furthermore, they often adapt to new fault samples and failure modes by horizontally stacking diagnostic modules, which can lead to model redundancy and waste. In summary, existing methods still have limitations such as large data storage volume, complex calculations, poor feature sharing, data privacy issues, and easy expansion and redundancy of network structures.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows:

[0005] (1) The number of parameters in existing methods increases linearly with the number of tasks, which can easily lead to parameter expansion. In addition, most of them adapt to new fault samples and fault modes by horizontally superimposing diagnostic modules, which can easily cause model redundancy and waste.

[0006] (2) Existing methods still have limitations such as large data storage volume, complex calculations, poor feature sharing, data privacy issues, and easy expansion and redundancy of network structures. Summary of the Invention

[0007] In response to the problems existing in the prior art, the present invention provides a dual-path incremental fault diagnosis method that integrates attention mechanism and class prototype.

[0008] The present invention is implemented as follows: a dual-path incremental fault diagnosis method integrating attention mechanism and class prototype includes:

[0009] Step 1: The collected one-dimensional time series monitoring data is converted into a two-dimensional envelope spectrogram (frequency domain features) and a two-dimensional recurrence graph (time domain features). The backbone networks respectively use a convolutional neural network CNN1 based on Hu moments and a convolutional neural network CNN2 based on scale-invariant feature transform (SIFT);

[0010] In step 2, backbone network 1 is mainly used to extract global features F1 of time domain features, and backbone network 2 is mainly used to extract local features F2 of frequency domain features. Through the dual-branch network of time domain, frequency domain, local and global features, and the introduction of attention mechanism, accurate feature extraction is achieved;

[0011] Step 3: Project the extracted features into a higher-dimensional space through random projection to enhance the linear separability of the features. The resulting projection features are H1 and H2 respectively. The idea of ​​class prototype is introduced to obtain class prototype matrices P1 and P2.

[0012] Step 4: Pass the result to the Softmax classifier and get the diagnosis result by summing up each element.

[0013] Furthermore, the two-dimensional envelope spectrum:

[0014] (1) First, the envelope signal is extracted by Hilbert transform of the original signal x(t), as shown in formula (1), and its complex form is expressed as:

[0015] z(t)=x(t)+jH[x(t)] (1)

[0016] Where z is the constructed analytical signal, H(t) is the Hilbert transform, and the modulus of the envelope signal is:

[0017] e(t)=|z(t)|=x2 (t)+H 2 [x(t)] (2)

[0018] Then, the envelope signal is Fourier transformed (3) to obtain the envelope spectrum;

[0019]

[0020] The envelope spectrum is used as input and SIFT is used to extract local features. First, the SIFT key points are enhanced and the SIFT key points Q(4) in the envelope spectrum are detected to highlight the significant frequency components.

[0021]

[0022] Where fq is the position, σq is the scale, and θq is the direction; q, N is the enhancement from the q=1 key point to the q=N key point; then the key point position is used to generate the attention mask A(5), and then CNN2 is frozen, that is, the fixed parameter θ is not updated during the training process, which is expressed by formula (6);

[0023]

[0024] in, is the loss function; finally, the weighted CNN2 outputs the local frequency domain feature F2(7):

[0025] F2=CNN2(E(f))⊙A(Q) (7)

[0026] Among them, γ is the adjustment parameter of the Gaussian kernel; ⊙ is the element-by-element multiplication, and A is the attention mask.

[0027] Furthermore, the two-dimensional recurrence plot (RecurrencePlot):

[0028] (1) First, the phase space of the one-dimensional signal is reconstructed by formula (8). Given a one-dimensional signal x(t), a high-dimensional trajectory vector is constructed by delay embedding.

[0029] r i =[x(i),x(i+τ),...,x(i+(m-1)τ)]∈R m (8)

[0030] Among them, r i is the m-dimensional reconstruction vector, m is the embedding dimension, τ is the delay time, R m is an m-dimensional real vector space; then use formula (9) to calculate the Euclidean distance between trajectory points and generate the recursive matrix R i,j , similarity of reaction signals;

[0031] Ri,j =θ(ε-||r i -r j ||) (9)

[0032] Where θ(·) is the Heaviside step function, ε is the similarity threshold, and ‖·‖ is the Euclidean distance;

[0033] The recurrence graph is used as input and the global feature extraction is guided by Hu moments. First, the seven Hu moments of the recurrence graph are calculated using formula (10) and used as additional feature input to enhance the Hu moments.

[0034] h=[h 1...... h7]

[0035]

[0036] Among them, η pq is the normalized central moment; CNN1 is then frozen, i.e. the fixed parameter θ is not updated during the training process (11)

[0037]

[0038] in, is the loss function;

[0039] Then, a shallow CNN is used to extract the local features of RP, and then the frozen CNN features and Hu moments are connected to output the global temporal features F1(12);

[0040]

[0041] in, is the splicing operation, and Wh is the linear transformation matrix of Hu moment.

[0042] Furthermore, the freezing:

[0043] The frozen backbone network has two main parts: forward propagation and backpropagation;

[0044] Forward propagation

[0045] The entire model is divided into two parts: the backbone network, whose parameters are θb (frozen) and the new head, whose parameters are θh (trainable); the calculation process of the input data x through the model is shown in formula (13):

[0046] y=f h (f b (x;θ b );θ h ) (13)

[0048] Where y is the model's final prediction of data x, fb is the forward calculation of the backbone network, and the parameter θb is fixed; fh is the forward calculation of the newly added head, and the parameter θh is trainable;

[0049] Backpropagation

[0050] When backpropagation is used to calculate the gradient, the gradient of the frozen parameters is shielded, and only the gradient of the trainable part is calculated. Given the loss function ζ(y, ytarget), the total loss is calculated as shown in formula (14):

[0051] ζ=ζ(f h (f b (x,θ b θ b );θ h ),y target ) (14)

[0052] Where ytarget is the true label of the model, that is, the correct answer corresponding to each sample in the dataset; then the normal gradient calculation of θh is performed as shown in formula (15)

[0053]

[0054] Among them, △θhζ is the gradient of the loss value, which indicates the adjustment direction and amplitude of the parameter θhd for updating;

[0055] The θb part is frozen, the gradient calculation is blocked, and no update is performed;

[0056]

[0057] Furthermore, the attention mechanism:

[0058] 1) A K-domain channel attention module is used to amplify the weights of important channels and realize the information association between channels. First, the global average pooling (GAP) operation without reducing the dimension is performed on each channel of the input feature block using formula (17), so that the output feature map has a dimension of 1×1×C. This output feature map reflects the global receptive field.

[0059]

[0060] Where g(f) represents the global average pooling (GAP) operation at the channel level, W and H represent the width and height of the feature block, respectively, and F represents the input feature. Then, the local interaction information between each channel and its K channel neighbors is captured by the Conv1d (one-dimensional convolution) operation with a convolution kernel of K using formula (18);

[0061] f {W} =W m g(f) (18)

[0062] Where f{W} is the interaction function of the K-channel domain, and Wm is expressed as a C×C parameter matrix; then, the Sigmoid activation function is used to output the weights with a dimension of 1×1×C through formula (19); ω=σ(f {W} (g(f))), now let y = g(f), then ω = σ(Wy), for each weight channel weight:

[0063]

[0064] Where ΩKi is the set of neighboring channels of the yi-th channel. The learning parameters of all channels are set to be the same, that is, ωi is the same. At this time, the algorithm can be implemented with a one-dimensional convolution kernel of , and the formula can be further simplified to formula (20):

[0065] ω=σ(Conv1d K (y)) (20)

[0066] Among them, Conv1d K Represents a one-dimensional convolution with a convolution kernel of ; the calculation formula of the convolution kernel is as follows:

[0067] K=N in +2P+1-SN out (twenty one)

[0068] Among them, Nin is the input feature map, Nout is the output feature map, P is the number of padded pixels, and S is the step size, that is, the number of steps the convolution kernel slides each time.

[0069] Finally, the input feature map and the weight obtained by the activation function are multiplied by formula (22), thereby redistributing the weights of each channel feature;

[0070]

[0071] Spatial attention adjusts the position weights to encourage the diagnostic network to capture effective spatial positions. First, the convolution operation is performed using formula (23). The convolution kernel is slid across the image to calculate the dot product, generate a feature map, extract more advanced features, and obtain the final feature map F.

[0072]

[0073] Next, for each channel position, global average pooling and global maximum pooling are performed on the spatial dimension according to formula (24) and formula (25) to obtain two pooled feature vectors; then, according to formula (26), the two feature vectors of length C are concatenated to obtain a more abundant feature vector Fconcat of length 2C, which contains the average value and maximum value information of the channel in the spatial dimension;

[0074]

[0075] F contac =[F avg ; F max ] (26)

[0076] Where FC(i, j) represents the eigenvalue of the cth channel at position (i, j), and “;” represents vertically concatenating the two vectors. Then, a fully connected layer is used to map it to a new space, and the sigmoid activation function is used to obtain the weight coefficient distribution and generate the weight coefficient vector Wv. The output of the fully connected layer can be expressed as formula (27):

[0077] W v =σ(FC(F contact )) (27)

[0078] Where FC represents the fully connected linear transformation, which can be expressed as FC(x) = Wfcx + bfc, Wfc is the weight matrix, and bfc is the bias vector; the Sigmoid activation function maps the output of the fully connected layer to the interval (0, 1), which represents the weight coefficient; the weight coefficient vector Wv is then reshaped into a shape H×W that matches the spatial dimension of the original feature map according to formula (28) in a specific dimension, so as to assign a weight value to each spatial position;

[0079] W reshape =reshape(W,(H,W)) (28)

[0080] Next, apply the exponential function to each element in Wreshape, calculate the sum of the exponential values ​​of all elements, and divide the exponential value of each element by the sum to obtain the weight map M, where the value of each pixel represents the importance of the position, and the sum of all weights is 1; the formula (29) of the softmax function is:

[0081]

[0082] Among them, W reshape,ij Represents the weight matrix W reshape The weight value at position (i, j); finally, the spatial attention weight map is multiplied element-by-element with the original feature map to obtain a weighted feature map to emphasize important areas.

[0083] Further, incremental training:

[0084] 1) Incremental training is performed using a prototype-like approach. After feature extraction, projection features H1 and H2 are derived using formulas (30) and (31). This feature transformation enhances linear separability. Where Wfrozen is the frozen weight; Φ is a nonlinear function that randomly projects the feature F into a high-dimensional space.

[0085]

[0086] 2) Update the class prototype library and correct data deviations through class prototype aggregation; when aggregating the new and old class prototypes, use formula (32) to perform weighted aggregation according to the sample ratio to obtain the latest representative class prototype; where μg c,t-1 represents the c-class prototype existing in the class prototype library, μg c represents the c-class prototype obtained by aggregating the sample data in the current stage, Dt-1c represents the training sample set of category c in the past task, and Dt c represents the training sample set of category c in the current task;

[0087]

[0088] 3) A variant of LDA for continuous learning is used to remove the correlation between class prototypes, making the decision boundary of the model clearer and improving the generalization ability of the model; the Gram matrix G of the feature H is extracted in an iterative manner:

[0089]

[0090] Among them, Nt is the number of training samples in each task, T is the number of incremental tasks, represents the outer product;

[0091] 4) Based on the latest representative class prototypes obtained in 32), the class prototype matrix C is obtained by concatenating them in series:

[0092]

[0093] Among them, y is the class label;

[0094] 5) Use formula (35) to obtain the weight P to represent the decorrelation class prototype:

[0095] P=(G+λI) -1 C (35)

[0096] Where I is the identity matrix and λ is the ridge regression parameter. The parameter λ is updated after each task and is optimized by randomly splitting the training data for that task in an 80:20 ratio and selecting the λ value that minimizes the mean squared error between the target and prediction sets.

[0097] 6) Calculate the final fault category by predicting the logits L using formula (36) and summing the elements using formula (37);

[0098] L=H test P (36)

[0099] Here, Htest represents the encoded features of the test sample after the random projection layer. Next, a softmax layer is applied on the logits of each model to obtain the probability, and an element-wise addition layer is applied to make the final prediction.

[0100]

[0101] in, To predict the probability, L1 and L2 are the logits values ​​of the frozen CNN2 network based on SIFT and the logits values ​​of the frozen CNN1 network based on Hu moment, respectively. They are calculated for each c class, and the maximum result of the final prediction is selected as the output to achieve the purpose of predicting the fault category.

[0102] Another object of the present invention is to provide a dual-path incremental fault diagnosis system integrating attention mechanism and class prototype, including:

[0103] The conversion module is used to convert the collected one-dimensional time series monitoring data into a two-dimensional envelope spectrum and a two-dimensional recurrence graph. The backbone networks use the convolutional neural network CNN1 based on Hu moment and the convolutional neural network CNN2 based on scale-invariant feature transform (SIFT).

[0104] The extraction module is used in backbone network 1 to extract global features F1 of time domain features, and backbone network 2 to extract local features F2 of frequency domain features. Through the dual-branch network of time domain, frequency domain, local and global features, and the introduction of attention mechanism, accurate feature extraction is achieved;

[0105] The projection module is used to project the extracted features into a higher-dimensional space through random projection to enhance the linear separability of the features. The obtained projection features are H1 and H2 respectively. The idea of ​​class prototype is introduced to obtain the class prototype matrices P1 and P2.

[0106] The summation module is used to pass the results to the Softmax classifier and obtain the diagnosis result by summing up each element.

[0107] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the dual-path incremental fault diagnosis method that integrates the attention mechanism and the class prototype.

[0108] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the dual-path incremental fault diagnosis method that integrates the attention mechanism and the class prototype.

[0109] Another object of the present invention is to provide an information data processing terminal, which is used to implement the dual-path incremental fault diagnosis system that integrates the attention mechanism and the class prototype.

[0110] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0111] An accurate feature extraction module can effectively expand the classification boundary and improve the sharing ability of features before and after the incremental process. The traditional single-branch model is difficult to capture the features of the time domain and frequency domain at the same time. To solve this problem, the present invention designs a dual-branch network and uses heterogeneous network division of labor: a convolutional neural network based on Hu moment extracts time domain features, a convolutional neural network based on scale-invariant feature conversion extracts frequency domain features, the time domain branch network captures the global features of the equipment operation trend, and the frequency domain branch network extracts the local features of abnormal signals in a specific frequency band, forming a time-frequency complementary feature space to achieve comprehensive feature coverage. Taking into account that the local frequency domain features of the high frequency band are generally more sensitive to early faults, while the global features of the time domain can intuitively reflect the overall operating status, the present invention introduces an attention mechanism to achieve adaptive weight allocation and further enhance the accuracy of feature extraction. Random projection is then used to increase the feature dimension, further improve the linear separability of the features, solve the problem of category overlap in the low-dimensional feature space, and expand the category boundary.

[0112] Replay-based incremental learning requires the storage of a large amount of historical data, resulting in high storage costs and high privacy risks. This technology compresses information through class prototypes (i.e., the central representation of the feature space of each class), and only needs to save a small number of prototype vectors, reducing storage overhead by more than 90%. At the same time, the backbone network parameters are frozen to avoid the computational burden of fine-tuning the entire model. In the incremental stage, only the class prototype matrix needs to be optimized, which significantly improves computational efficiency. Moreover, when new categories are added, there is no need to horizontally superimpose diagnostic modules on the network, and the network structure is fixed. Freezing the backbone network can also retain the existing feature extraction capabilities, and only adjusts the class prototypes (classification layer parameters) to separate the decision boundaries of the new and old categories, thereby minimizing the impact on the classification performance of old categories when new categories are introduced, and effectively suppressing the forgetting of old knowledge caused by frequent adjustments to model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] Figure 1 This is a flow chart of a dual-path incremental fault diagnosis method that integrates an attention mechanism and a class prototype, provided by an embodiment of the present invention.

[0114] Figure 2 This is a structural block diagram of a dual-path incremental fault diagnosis system that integrates an attention mechanism and a class prototype, provided by an embodiment of the present invention.

[0115] Figure 3 This is a technical block diagram of a dual-path incremental fault diagnosis method that integrates an attention mechanism and a class prototype, as provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0116] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0117] Dual-path: Two parallel feature extraction branches, which respectively extract the time domain features and frequency domain features of the monitoring data, and design global feature extraction modules and local feature extraction modules based on the characteristics of different domain features.

[0118] Attention mechanism: Imitates human visual function, enabling feature extraction to focus on the important parts of the input data.

[0119] Class prototype: An example or reference vector that can represent the characteristics of a certain category of data, rather than the original data of this category.

[0120] Incremental: Data is acquired in a streaming manner. When new data appears, the diagnostic system can retain old knowledge while learning new knowledge. In other words, it can continuously learn new knowledge from new data while retaining existing knowledge.

[0121] This invention aims to address technical bottlenecks in current fault diagnosis scenarios, including weak model generalization, poor incremental learning performance, chaotic coupling between local and global features, and insufficient shallow perception of attention mechanisms. Traditional diagnostic methods based on convolutional neural networks (CNNs) typically rely on fixed input patterns and static parameter structures, and are unable to effectively address the heterogeneity of representations across multi-source time series signals. In particular, in incremental learning scenarios, catastrophic forgetting can occur, significantly reducing feature discrimination and category aggregation.

[0122] To address these issues, the present invention proposes a dual-branch architecture that guides feature learning paths based on recursive graphs and envelope spectrograms, respectively. By introducing the Hu moment and SIFT operators to enhance image structure priors, the architecture extracts global temporal and local frequency domain features, respectively, enhancing the network's ability to capture texture information at different scales in both spatial and frequency domains. Compared to traditional end-to-end single-pathway architectures, this dual-branch architecture enhances feature diversity and complementarity through different perception pathways, thereby improving the model's adaptability to multimodal input signals.

[0123] In terms of feature fusion, this paper innovatively integrates channel attention and spatial attention. The former uses the K-channel local perception mechanism to regulate the activation of important channels, while the latter drives the accuracy of mask distribution with spatial saliency. This enables the network to not only enhance its responsiveness at the feature amplitude level but also provide targeted weight allocation capabilities for spatial positioning. This mechanism effectively addresses common problems of deep models such as feature redundancy, fuzzy responses, and missed targets.

[0124] To improve classification performance, the present invention introduces a random projection mechanism after feature encoding, mapping the original features into a high-dimensional latent space using a fixed kernel function. This increases the inter-class distances between samples and improves the intra-class aggregation, enhancing the linear separability of features and the discriminative power of the model. Furthermore, this step can be used in conjunction with the frozen network strategy to ensure the stability of the original knowledge transfer and reduce the interference of new class learning on the old knowledge representation.

[0125] In multi-round incremental learning scenarios, this paper introduces a meta-learning mechanism based on class prototype aggregation. This mechanism constructs a dynamic prototype library by weighted superposition of historical prototypes and new prototypes for the current task. This is combined with a variant of LDA to orthogonalize prototype correlations and optimize inter-class decision boundaries, thereby mitigating inter-class overlap and improving generalization capabilities under conditions where new and old classes coexist. This strategy is particularly suitable for scenarios with small sample increments and complex class distributions with uneven classes.

[0126] Ultimately, based on dual-path feature encoding, attention-guided mechanisms, random mapping enhancement, and prototype alignment, the model outputs a joint voting prediction of the logits generated by the Hu moment and SIFT paths, with softmax normalization achieving robust output of fault types. The overall workflow is highly modular, structurally cohesive, and makes precise decisions, providing a new paradigm for intelligent fault diagnosis with high interpretability, portability, and adaptability.

[0127] like Figure 1 、 Figure 3 As shown, the embodiment of the present invention provides a dual-path incremental fault diagnosis method that integrates the attention mechanism and the class prototype, including the following steps:

[0128] S101, converting the collected one-dimensional time series monitoring data into a two-dimensional envelope spectrum and a two-dimensional recurrence graph. The backbone networks respectively use a convolutional neural network CNN1 based on Hu moments and a convolutional neural network CNN2 based on scale-invariant feature transform (SIFT).

[0129] S102, backbone network 1 is mainly used to extract global features F1 of time domain features, and backbone network 2 is mainly used to extract local features F2 of frequency domain features; through the dual-branch network of time domain, frequency domain, local and global features, and the introduction of attention mechanism, accurate feature extraction is achieved;

[0130] S103, projecting the extracted features into a higher-dimensional space through random projection to enhance the linear separability of the features. The resulting projection features are H1 and H2 respectively. The idea of ​​class prototype is introduced to obtain class prototype matrices P1 and P2.

[0131] S104: pass the result to the Softmax classifier, and obtain the diagnosis result by summing up each element.

[0132] The two-dimensional envelope spectrum provided by the embodiment of the present invention is:

[0133] (1) First, the envelope signal is extracted by Hilbert transform of the original signal x(t), as shown in formula (1), and its complex form is expressed as:

[0134] z(t)=x(t)+jH[x(t)] (1)

[0135] Where z is the constructed analytical signal, H(t) is the Hilbert transform, and the modulus of the envelope signal is:

[0136] e(t)=|z(t)|=x 2 (t)+H 2 [x(t)] (2)

[0137] Then, the envelope signal is Fourier transformed (3) to obtain the envelope spectrum;

[0138]

[0139] The envelope spectrum is used as input and SIFT is used to extract local features. First, the SIFT key points are enhanced and the SIFT key points Q(4) in the envelope spectrum are detected to highlight the significant frequency components.

[0140]

[0141] Where fq is the position, σq is the scale, and θq is the direction. The key point positions are then used to generate the attention mask A(5). CNN2 is then frozen, meaning that the fixed parameter θ is not updated during training, as expressed in formula (6).

[0142]

[0143] in, is the loss function, γ is the adjustment parameter of the Gaussian kernel; finally, the weighted CNN2 outputs the local frequency domain feature F2(7):

[0144] F2=CNN2(E(f))⊙A(Q) (7)

[0145] Among them, ⊙ is the element-wise multiplication and A is the attention mask.

[0146] The two-dimensional recurrence plot (RecurrencePlot) provided by the embodiment of the present invention:

[0147] (1) First, the phase space of the one-dimensional signal is reconstructed by formula (8). Given a one-dimensional signal x(t), a high-dimensional trajectory vector is constructed by delay embedding.

[0148] r i =[x(i),x(i+τ),...,x(i+(m-1)τ)]∈R m (8)

[0149] Among them, r i is the m-dimensional reconstruction vector, m is the embedding dimension, and τ is the delay time; then the Euclidean distance between the trajectory points is calculated using formula (9) to generate a recursive matrix that reflects the similarity of the signals;

[0150] R i,j =θ(ε-||r i -r j ||) (9)

[0151] Where θ(·) is the Heaviside step function, ε is the similarity threshold, and ‖·‖ is the Euclidean distance;

[0152] The recurrence graph is used as input and the global feature extraction is guided by Hu moments. First, the seven Hu moments of the recurrence graph are calculated using formula (10) and used as additional feature input to enhance the Hu moments.

[0153] h=[h1......h7]

[0154]

[0155] Among them, η pq is the normalized central moment; CNN1 is then frozen, i.e. the fixed parameter θ is not updated during the training process (11)

[0156]

[0157] in, is the loss function;

[0158] Then, a shallow CNN is used to extract the local features of RP, and then the frozen CNN features and Hu moments are connected to output the global temporal features F1(12);

[0159]

[0160] in, is the splicing operation, and Wh is the linear transformation matrix of Hu moment.

[0161] The freezing provided by the embodiment of the present invention:

[0162] The frozen backbone network has two main parts: forward propagation and backpropagation;

[0163] Forward propagation

[0164] The entire model is divided into two parts: the backbone network, whose parameters are θb (frozen) and the new head, whose parameters are θh (trainable); the calculation process of the input data x through the model is shown in formula (13):

[0165] y=f h (f b (x;θ b );θ h ) (13)

[0166] Where y is the model's final prediction of data x, fb is the forward calculation of the backbone network, and the parameter θb is fixed; fh is the forward calculation of the newly added head, and the parameter θh is trainable;

[0167] Backpropagation

[0168] When backpropagation is used to calculate the gradient, the gradient of the frozen parameters is shielded, and only the gradient of the trainable part is calculated. Given the loss function ζ(y, ytarget), the total loss is calculated as shown in formula (14):

[0169] ζ=ζ(f h (f b (x,θ b θ b );θ h ),y target ) (14)

[0170] Where ytarget is the true label of the model, that is, the correct answer corresponding to each sample in the dataset; then the normal gradient calculation of θh is performed as shown in formula (15)

[0171]

[0172] Among them, △θhζ is the gradient of the loss value, which indicates the adjustment direction and amplitude of the parameter θhd for updating;

[0173] The θb part is frozen, the gradient calculation is blocked, and no update is performed;

[0174]

[0175] The attention mechanism provided by the embodiment of the present invention:

[0176] 1) A K-domain channel attention module is used to amplify the weights of important channels and realize the information association between channels. First, the global average pooling (GAP) operation without reducing the dimension is performed on each channel of the input feature block using formula (17), so that the output feature map has a dimension of 1×1×C. This output feature map reflects the global receptive field.

[0177]

[0178] Where g(f) represents the global average pooling (GAP) operation at the channel level, W and H represent the width and height of the feature block, respectively, and F represents the input feature. Then, the local interaction information between each channel and its K channel neighbors is captured by the Conv1d (one-dimensional convolution) operation with a convolution kernel of K using formula (18);

[0179] f {W} =W m g(f) (18)

[0180] Where f{W} is the interaction function in the K-channel domain, and Wm is expressed as a C×C parameter matrix. Then, the Sigmoid activation function is used to output weights with a dimension of 1×1×C through formula (19). Now let y=g(f), then ω=σ(Wy), and the weight of each weight channel is:

[0181]

[0182] Where ΩKi is the set of neighboring channels of the yi-th channel. The learning parameters of all channels are set to be the same, that is, ωi is the same. At this time, the algorithm can be implemented with a one-dimensional convolution kernel of , and the formula can be further simplified to formula (20):

[0183] ω=σ(Conv1d K (y)) (20)

[0184] Among them, Conv1d K Represents a one-dimensional convolution with a convolution kernel of ; the calculation formula of the convolution kernel is as follows:

[0185] K=N in +2P+1-SN out (twenty one)

[0186] N in, N out, P

[0187] Finally, the input feature map and the weight obtained by the activation function are multiplied by formula (22), thereby redistributing the weights of each channel feature;

[0188]

[0189] Spatial attention adjusts the position weights to encourage the diagnostic network to capture effective spatial positions. First, the convolution operation is performed using formula (23). The convolution kernel is slid across the image to calculate the dot product, generate a feature map, extract more advanced features, and obtain the final feature map F.

[0190]

[0191] Next, for each channel position, global average pooling and global maximum pooling are performed on the spatial dimension according to formula (24) and formula (25) to obtain two pooled feature vectors; then, according to formula (26), the two feature vectors of length C are concatenated to obtain a more abundant feature vector Fconcat of length 2C, which contains the average value and maximum value information of the channel in the spatial dimension;

[0192]

[0193] F contac =[F avg ; F max ] (26)

[0194] Where FC(i, j) represents the eigenvalue of the cth channel at position (i, j), and “;” represents vertically concatenating the two vectors. Then, a fully connected layer is used to map it to a new space, and the sigmoid activation function is used to obtain the weight coefficient distribution and generate the weight coefficient vector Wv. The output of the fully connected layer can be expressed as formula (27):

[0195] W v =σ(FC(F contact )) (27)

[0196] Where FC represents the fully connected linear transformation, which can be expressed as FC(x) = Wfcx + bfc, Wfc is the weight matrix, and bfc is the bias vector; the Sigmoid activation function maps the output of the fully connected layer to the interval (0, 1), which represents the weight coefficient; the weight coefficient vector Wv is then reshaped into a shape H×W that matches the spatial dimension of the original feature map according to formula (28) in a specific dimension, so as to assign a weight value to each spatial position;

[0197] W reshape =reshape(W,(H,W)) (28)

[0198] Next, apply the exponential function to each element in Wreshape, calculate the sum of the exponential values ​​of all elements, and divide the exponential value of each element by the sum to obtain the weight map M, where the value of each pixel represents the importance of the position, and the sum of all weights is 1; the formula (29) of the softmax function is:

[0199]

[0200] Among them, W reshape,ij Represents the weight matrix W reshape The weight value at position (i, j); finally, the spatial attention weight map is multiplied element-by-element with the original feature map to obtain a weighted feature map to emphasize important areas.

[0201] The incremental training provided by the embodiment of the present invention:

[0202] 1) Incremental training is performed using a prototype-like approach. After feature extraction, projection features H1 and H2 are derived using formulas (30) and (31). This feature transformation enhances linear separability. Where Wfrozen is the frozen weight; Φ is a nonlinear function that randomly projects the feature F into a high-dimensional space.

[0203]

[0204] 2) Update the class prototype library and correct data deviations through class prototype aggregation; when aggregating the new and old class prototypes, use formula (32) to perform weighted aggregation according to the sample ratio to obtain the latest representative class prototype; where μg c,t-1 represents the c-class prototype existing in the class prototype library, μg c represents the c-class prototype obtained by aggregating the sample data in the current stage, Dt-1c represents the training sample set of category c in the past task, and Dt c represents the training sample set of category c in the current task;

[0205]

[0206] 3) A variant of LDA for continuous learning is used to remove the correlation between class prototypes, making the decision boundary of the model clearer and improving the generalization ability of the model; the Gram matrix G of the feature H is extracted in an iterative manner:

[0207]

[0208] Among them, Nt is the number of training samples in each task, T is the number of incremental tasks, represents the outer product;

[0209] 4) Based on the latest representative class prototypes obtained in 32), the class prototype matrix C is obtained by concatenating them in series:

[0210]

[0211] Among them, y is the class label;

[0212] 5) Use formula (35) to obtain the weight P to represent the decorrelation class prototype:

[0213] P=(G+λI) -1 C (35)

[0214] Where λ is the ridge regression parameter, which is updated after each task and optimized by randomly splitting the training data for that task in an 80:20 ratio and selecting the λ value that minimizes the mean squared error between the target and prediction sets;

[0215] 6) Calculate the final fault category by predicting the logits L using formula (36) and summing the elements using formula (37);

[0216] L=H test P (36)

[0217] Here, Htest represents the encoded features of the test sample after the random projection layer. Next, a softmax layer is applied on the logits of each model to obtain the probability, and an element-wise addition layer is applied to make the final prediction.

[0218]

[0219] Among them, L1 and L2 are the logits values ​​of the frozen CNN2 network based on SIFT and the logits values ​​of the frozen CNN1 network based on Hu moment, respectively. They are calculated for each c class separately, and the maximum result of the final prediction is selected as the output to achieve the purpose of predicting the fault category.

[0220] like Figure 2 As shown, an embodiment of the present invention provides a dual-path incremental fault diagnosis system integrating an attention mechanism and a class prototype, including:

[0221] The conversion module is used to convert the collected one-dimensional time series monitoring data into a two-dimensional envelope spectrum and a two-dimensional recurrence graph. The backbone networks use the convolutional neural network CNN1 based on Hu moment and the convolutional neural network CNN2 based on scale-invariant feature transform (SIFT).

[0222] The extraction module is used in backbone network 1 to extract global features F1 of time domain features, and backbone network 2 to extract local features F2 of frequency domain features. Through the dual-branch network of time domain, frequency domain, local and global features, and the introduction of attention mechanism, accurate feature extraction is achieved;

[0223] The projection module is used to project the extracted features into a higher-dimensional space through random projection to enhance the linear separability of the features. The obtained projection features are H1 and H2 respectively. The idea of ​​class prototype is introduced to obtain the class prototype matrices P1 and P2.

[0224] The summation module is used to pass the results to the Softmax classifier and obtain the diagnosis result by summing up each element.

[0225] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the dual-path incremental fault diagnosis method that integrates the attention mechanism and the class prototype.

[0226] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the dual-path incremental fault diagnosis method that integrates the attention mechanism and the class prototype.

[0227] Another object of the present invention is to provide an information data processing terminal, which is used to implement the dual-path incremental fault diagnosis system that integrates the attention mechanism and the class prototype.

[0228] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A dual-path incremental fault diagnosis method integrating attention mechanism and class prototype, characterized in that: The steps include: S1, converting the collected one-dimensional time series monitoring data into a two-dimensional envelope spectrum and a two-dimensional recurrence graph respectively; S2, inputting the envelope spectrogram into a convolutional neural network CNN2 based on scale-invariant feature transformation to extract frequency domain local features F2; The recursive graph is input into the Hu moment-based convolutional neural network CNN1 to extract the time domain global feature F1; S3, respectively introduces channel and spatial attention mechanisms to enhance feature representation for F1 and F2; S4, perform random projection transformation on F1 and F2 respectively to obtain high-dimensional features H1 and H2; S5, constructing class prototype matrices P1 and P2 based on the class prototype method; S6. Use the Softmax classifier to perform weighted prediction on the fused features and output the fault diagnosis results.

2. The method according to claim 1, wherein: The two-dimensional envelope spectrogram is obtained by Hilbert transform and Fourier transform of the original time series signal, and the attention mask is generated by SIFT key point detection; the two-dimensional recursive graph is constructed by phase space reconstruction and Euclidean distance matrix, and seven Hu moments are calculated to enhance time domain feature extraction.

3. The method according to claim 1, wherein: The attention mechanism consists of two parts: channel attention and spatial attention. The channel attention uses K-domain one-dimensional convolution to amplify the key channel response, and the spatial attention is based on the concatenation of average pooling and maximum pooling results to generate a weight map to adjust the spatial feature weight distribution.

4. The method according to claim 1, wherein: The F1 and F2 are respectively subjected to high-dimensional mapping through a nonlinear projection function Φ with frozen weights to obtain projection features H1 and H2, which are used to enhance the linear separability between classes.

5. The method according to claim 1, wherein: The class prototypes P1 and P2 are obtained by aggregating samples of each category in the current stage and the historical stage and weighting them according to the sample ratio, and the weight is determined by the number of samples.

6. The method according to claim 1, wherein: In the incremental task process, an LDA variant mechanism of inter-class decorrelation is introduced, and the Gram matrix is ​​used to construct the category outer product and generate the class prototype matrix C. The decorrelation weight matrix P is obtained through ridge regression optimization to optimize feature discriminability.

7. The method according to claim 1, wherein: During the learning process of the newly added task, the parameters of the backbone networks CNN1 and CNN2 are frozen, and only the newly added classification head is trained. During back propagation, only the trainable parameters θh are updated. The freezing is achieved by shielding the gradient.

8. The method according to claim 1, wherein: The final diagnosis prediction is achieved by inputting H1 and H2 into the Softmax function respectively to obtain the logits values ​​L1 and L2, and then performing element-wise summation, and selecting the category corresponding to the maximum value as the prediction result.

9. A dual-path incremental fault diagnosis system integrating attention mechanism and class prototype to implement the method of claim 1, characterized in that: include: A conversion module, used to convert the time series monitoring data into an envelope spectrum and a recurrence graph respectively; Extraction module, used to extract frequency domain local features F2 and time domain global features F1; The projection module is used to perform high-dimensional random projection on F1 and F2 to generate H1 and H2; Prototype module, used to construct prototype-like matrices P1 and P2; Classification module, used to fuse features and output fault diagnosis results.

10. The system according to claim 9, wherein: The projection module uses a random projection mapping structure with fixed frozen parameters; the classification module performs a final summation by combining multiple logits values ​​through a Softmax logistic regression function and outputs a predicted label.

Images