Bearing fault diagnosis method based on fusion of improved capsule network and zero sample learning

Abstract

The invention discloses a bearing fault diagnosis method based on fusion of an improved capsule network and zero sample learning, and relates to the technical field of state monitoring and fault diagnosis of electromechanical equipment, and the method comprises the following steps: collecting a multi-mode signal during the operation of a bearing, employing an improved wavelet threshold denoising algorithm for the multi-mode signal to eliminate environmental noise, and then employing a zero sample learning algorithm for the multi-mode signal; according to the method, the improved wavelet threshold de-noising algorithm and the WPD and VMD fusion decomposition algorithm are adopted to extract the time-frequency domain mixed features as sample data, and the GAN is combined to expand the bearing sample data, so that the data dependence of traditional deep learning is broken through, the time-frequency domain mixed features are extracted through the improved wavelet threshold de-noising algorithm and the WPD and VMD fusion decomposition algorithm, and the time-frequency domain mixed features are extracted through the improved wavelet threshold de-noising algorithm and the WPD and VMD fusion decomposition algorithm. Small sample data learning is realized, and by training a pyramid capsule network and optimizing a cross entropy loss function and combining cross-modal joint optimization and a zero sample inference engine, the diagnosis accuracy of known faults is greatly improved, and unknown fault types can be effectively inferred.

Description

Technical Field

[0001] The present invention relates to the technical field of condition monitoring and fault diagnosis of electromechanical equipment, and particularly relates to a bearing fault diagnosis method based on a fusion of an improved capsule network and zero-shot learning. Background Art

[0002] In the operation and maintenance of modern industrial equipment, the condition monitoring of the bearings of rotating machinery is a core link to ensure production safety and reduce maintenance costs. With the development of intelligent manufacturing, intelligent diagnosis methods based on vibration signal analysis have become the main technical route. There are three significant defects in the current technology:

[0003] 1. Data dependence problem: Traditional deep learning methods such as CNN and RNN both rely on a large number of completely labeled fault samples. However, in actual industrial scenarios, it is costly and very difficult to obtain full-category fault data, or obtaining fault samples requires damaging the equipment, which limits the training of the model and makes it difficult to accurately identify various fault types. Especially for sudden new faults, there are data blank areas;

[0004] 2. Limitation in identifying unknown faults: It is difficult to capture the deep features of bearing faults under complex working conditions, which may not accurately reflect the operating state of the bearings, making the existing supervised learning framework completely ineffective for fault types not included in the training set. When the equipment appears an unforeseen new composite fault mode, traditional methods will produce serious misjudgments;

[0005] 3. Feature representation defect: The spatial hierarchical modeling ability of traditional convolutional neural networks is insufficient, and it is difficult to effectively capture the multi-scale correlation characteristics of fault features under complex working conditions such as variable rotational speed and strong noise environments.

[0006] It is worth noting that existing research has tried to apply capsule networks and zero-shot learning respectively and made some progress, but there are two key technical bottlenecks: one is that the computational complexity of the traditional capsule network dynamic routing mechanism grows exponentially in the industrial big data scenario; the other is that the attribute correlation modeling accuracy of existing zero-shot transfer methods is insufficient in cross-working condition scenarios, resulting in a low generalization recognition rate of unknown faults, a significant decline in performance, and affecting the diagnostic results at the same time. These defects seriously restrict the practical application value of intelligent diagnosis systems in industrial fields.

[0007] The above information disclosed in the background art section is only used to enhance the understanding of the background of the present disclosure, and therefore it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0008] The object of the present invention is to provide a bearing fault diagnosis method based on fusing an improved capsule network and zero-shot learning. The present invention constructs a pyramid capsule network and optimizes the dynamic routing to achieve a fusion model after cross-modal adversarial alignment, so as to solve the problems in the above-mentioned background technology.

[0009] To achieve the above object, the present invention provides the following technical solutions: A bearing fault diagnosis method based on fusing an improved capsule network and zero-shot learning, comprising the following steps:

[0010] S1. Collect multi-modal signals during the operation of the bearing. First, use an improved wavelet threshold denoising algorithm to eliminate environmental noise from the multi-modal signals, and then use a fusion decomposition algorithm of wavelet packet decomposition (WPD) and variational mode decomposition (VMD) to extract time-frequency domain hybrid features as sample data, and construct an adversarial generative network (GAN) to expand the training sample data;

[0011] S2. Perform dynamic modeling according to the sample data of the time-frequency domain hybrid features, use a single-pass feedforward attention mechanism to optimize the dynamic routing algorithm, design a pyramid capsule network structure to compress features, and introduce a bidirectional gated recurrent unit (Bi-GRU) to achieve cross-level feature interaction;

[0012] S3. Construct a fault attribute knowledge graph by defining the association rules between the physical attributes and fault types of the bearing, use a graph convolutional network (GCN) graph embedding algorithm to generate transferable semantic vectors, design a JS divergence constraint for a two-channel mapping network, and achieve cross-modal adversarial alignment by minimizing the modal difference through an adversarial loss function;

[0013] S4. First, use the known fault samples of the bearing to train the pyramid capsule network, optimize the cross-entropy loss function, freeze the bottom layer and then introduce a zero-shot loss term for cross-modal joint optimization to generate a fusion model for fault diagnosis. Then, start to fine-tune the network with 5% of the parameter quantity and combine the elastic weight consolidation (EWC) algorithm to update the lightweight model. Among them, the gradient clipping method is used to prevent gradient explosion during the modal alignment process;

[0014] S5. The fusion model performs online time-frequency analysis on the multi-modal signals input by the bearing. The updated lightweight capsule network directly outputs a classification result to diagnose a known fault. Start the zero-shot inference engine, infer the fault type as a position fault by calculating the cosine similarity between the visual features and semantic attributes, generate a fault evolution heat map for the bearing, and use visualization evaluation technology to display the bearing damage position and severity.

[0015] Optionally, the acquisition and processing logic steps of the multi-modal signals are as follows:

[0016] For the built-in acceleration sensors, temperature sensors, and acoustic emission sensors on electromechanical equipment, multi-modal signals including vibration signals, acoustic emission signals, and temperature signals of the bearings are synchronously collected under different working conditions and calibrated as Mms(t);

[0017] For the multi-modal signal Mms(t), the improved wavelet threshold denoising algorithm is used to eliminate environmental noise and improve the signal-to-noise ratio. The wavelet decomposition formula is and φ j,k (t) = 2 -j / 2 .φ(2 -j ·t - k), where W j,k represents the wavelet coefficients obtained by continuous wavelet transform of the multi-modal signal Mms(t), φ j,k (t) represents the wavelet basis function after scale translation, Mms(t) represents the multi-modal signal at the input time step t, j represents the decomposition level and j = 5, and k represents the translation coefficient;

[0018] Combining noise estimation and signal energy, the adaptive improved threshold calculation formula is where λ j represents the threshold corresponding to the j-th layer of wavelet decomposition, σ j represents the noise standard deviation of the j-th layer, N represents the total number of sampling points in the signal length, and E j represents the signal energy of the j-th layer;

[0019] After threshold processing using the improved non-linear soft threshold function, the wavelet is reconstructed. The improved soft threshold function calculation formula is where represents retaining the sign of the original wavelet coefficient W j,k , sign represents the sign function, 0.05 represents the attenuation coefficient to suppress pseudo Gibbs oscillations, represents the exponential decay term;

[0020] The calculation formula for the reconstructed wavelet is where represents the new signal of the approximate original signal after threshold processing and reconstruction of the multi-modal signal Mms(t).

[0021] Optionally, the steps for extracting and augmenting samples of the time-frequency domain hybrid features are as follows:

[0022] For the new signal of the denoised signal Perform i-layer wavelet packet decomposition using WPD to generate 8 sub-bands. The calculation formula for WPD is where WPD n,iDenoted as the sub - band coefficient with node number n at the i - th level of decomposition, where n represents the node number, i represents the decomposition level, and i = 3, ψ n Denoted as the wavelet packet basis function, and k0 represents the parameter of the translation of the wavelet packet basis function;

[0023] For each sub - band of WPD, variational mode decomposition (VMD) is used to solve the constraints, and then the alternating direction multiplier method (ADMM) algorithm is used for iterative solution to obtain the IMF components. By setting the number of modes and the penalty factor, the calculation formula of VMD is And the constraint condition is In the formula, Denoted as the partial derivative with respect to time t, and δ(t) is denoted as the delta function, which is used for convolution operation with the mode function u K (t), and u K (t) is denoted as the K - th mode component, and K = 6, Denoted as the complex exponential function, Denoted as the objective function of the constrained variation, u K Denoted as the sequence of mode functions, w K Denoted as the center frequency sequence corresponding to the mode function u K (t), Denoted as the kernel function of the Hilbert transform, which is used to perform the Hilbert transform on the mode function u K (t), ||||2 2 Denoted as the square of the two - norm. Among them, the IMF component is the mode component u K (t);

[0024] For each IMF component u K (t), perform the Hilbert transform to generate the analytic signal z K (t), and calculate the instantaneous energy spectrum E K (t, f). Then the calculation formula of the analytic signal is And (j0) 2 = 1. In the formula, z K (t) is denoted as the analytic signal corresponding to the K - th mode component u K (t), Denoted as the result of performing the Hilbert transform on the K - th mode component u K (t), and j0 represents the imaginary unit;

[0025] The calculation formula of the instantaneous energy spectrum is In the formula, E K (t, f) is denoted as the instantaneous energy spectrum of the K - th mode component at time t and frequency f, and STFT(z K (t)) is denoted as the short - time Fourier transform of the analytic signal z K(t) is the result of performing the short-time Fourier transform, where df and dt represent differential elements of time and frequency respectively;

[0026] Fuse all IMF components u K (t) of the energy spectrum E K (t,f), generate the matrix E of the time-frequency domain hybrid feature TFJD map with 128×128 pixels K {t,f};

[0027] Perform data augmentation on the extracted time-frequency domain hybrid feature TFJD by using the generative adversarial network GAN. Input the bearing working conditions through the generator, perform spectral normalization constraints on the discriminator, and define the loss functions for adversarial and physical constraints, thereby realizing sample expansion.

[0028] Optionally, the optimization steps of the dynamic routing algorithm are as follows:

[0029] Input the time-frequency domain hybrid feature E K {t,f} into the capsule network for feature mapping as And perform a linear transformation on the output of the bottom-layer capsules. The calculation formula of the linear transformation is And In the formula, Represents the Kth capsule vector after linear transformation through the query matrix W q , with a dimension of d in , W q Represents a learnable query matrix, and its dimension is d out ×d in , map the input capsule from the d in -dimensional space to the d out -dimensional space for linear transformation. R represents the set of real numbers, and d out Represents the dimension of the output capsule vector after linear transformation, and d in Represents the dimension of the input capsule vector , Represents the Kth input capsule vector;

[0030] Generate routing coefficients through key-value pair attention, that is, calculate the attention weights. The calculation formula of the attention weights is And In the formula, α K,L Represents the routing coefficient from the Kth transformed capsule to the Lth high-level capsule. Softmax represents the Softmax function, and W K , W L respectively represent the transformed capsules A key matrix and a value matrix mapped to a key space and a value space, both with a dimension of d out ×d out , Denoted as the L-th transformed capsule Denoted as the scaling factor;

[0031] The single-pass feedforward attention mechanism uses single-pass feedforward aggregation to replace iterative routing. The calculation formula for the capsule output is and In the formula, E L Denoted as the output of the L-th high-level capsule, with a dimension of d out , Denoted as for all N transformed input capsules According to the attention weight α K,L Perform weighted summation to obtain a d out dimensional vector Denoted as the 2-norm of the vector after weighted summation Denoted as the square of the modulus of the vector after weighted summation

[0032] Optionally, the design steps of the pyramid capsule network structure are as follows:

[0033] Constructing the pyramid capsule network structure includes a bottom layer, a middle layer, and a top layer. Among them, the bottom layer structure is an 8×8 grid, which is used to let each capsule cover a 2×2 area of the signal time-frequency diagram. The bottom layer captures the bearing local micro-defect features according to the extracted time-frequency domain mixed features. The calculation formula for the bearing local micro-defect features is And the value ranges of x1 and y1 are 0-7. In the formula, Denoted as the capsule vector at the position (x1, y1) in the bottom layer capsule network. x1 and y1 represent the position coordinates in the bottom layer capsule network. F caps Denoted as the capsule function, E K {t, f} represents the regional data of the TFJD diagram, and [2x1:2x1+2, 2y1:2y1+2] represents the index for slicing the regional data E K {t, f} of the TFJD diagram;

[0034] The middle layer structure is a 4×4 grid, which is used to aggregate the 2×2 areas of 4 capsules in the bottom layer and model the bearing component-level association features. The calculation formula for the association features is And the value ranges of x2 and y2 are 0-3. In the formula, Denoted as the capsule vector at the position (x2, y2) in the middle layer capsule network. Routing represents the routing algorithm and Denoted as the capsule vectors in the bottom layer participating in the aggregation;

[0035] The high-level structure is global pooling, which is used to integrate all capsules in the middle layer and model the system-level fault evolution pattern. The calculation formula of is In the formula, E top represents the output vector of the top-level capsule network, which contains the global feature information of the entire time-frequency map data and is used for the final fault diagnosis of the bearing. Globalrouting represents the global routing algorithm, represents all capsule vectors in the middle layer;

[0036] The bidirectional gated recurrent unit Bi-GRU is used to achieve cross-level feature interaction. The Bi-GRU mechanism includes forward propagation, backward propagation, and feature fusion. Among them, forward propagation is the feature transfer from the middle layer capsule to the top layer of the pyramid capsule network; backward propagation is the gradient backpropagation from the top layer capsule to the bottom layer of the pyramid capsule network; feature fusion is to adjust the capsule weights after splicing the bidirectional states of the pyramid capsule network using the fusion matrix.

[0037] Optionally, the construction logic of the fault attribute knowledge graph is as follows:

[0038] Record the interpretable association rules of the relationship between the physical attributes and fault types of the bearing based on historical fault records;

[0039] Use the physical attributes and fault types of the bearing as nodes and the association rules as edges to construct a fault attribute knowledge graph;

[0040] Use the graph convolutional network GCN graph embedding algorithm to encode the discrete graph relationship into a continuous vector space;

[0041] Take the fault type node features output by the last layer of GCN as semantic vectors.

[0042] Optionally, the steps of the cross-modal adversarial alignment are as follows:

[0043] Design two mapping networks to map different modal data of the semantic vector of the fault attribute knowledge graph and visual features into a common feature space respectively. Among them, the probability distributions of the semantic vector and visual features are calibrated as P v 、P s ;

[0044] Use the JS divergence to measure the distribution difference between the two modalities of semantic P v and visual P s in the common feature space, and make the two distributions as close as possible through constraints. The expression of the JS divergence constraint is In the formula, Denoted as the JS divergence constraint loss, KL is denoted as the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions;

[0045] An adversarial loss function is introduced to achieve cross-modal adversarial alignment by minimizing the modality difference. Among them, the expression of the adversarial loss function is In the formula, Denoted as the adversarial loss, E v 、E s Denote the expectations of the visual feature samples v and the semantic vector samples s respectively, and D is denoted as the discriminator. Denote the feature vectors obtained after the visual feature v and the semantic vector s go through the projection operation respectively;

[0046] Add a data reconstruction loss term to ensure the integrity of information. Among them, the expression of the reconstruction loss term is In the formula, Denoted as the reconstruction loss, Denote the original visual feature vector and semantic vector respectively. Dec denotes the decoder composed of 2 deconvolution layers. Denoted as the square of the 2-norm, Denote the visual feature and semantic connection recovered by the decoder respectively;

[0047] The JS divergence constraint loss, the adversarial loss, and the reconstruction loss are weighted and summed according to certain weights to obtain the total loss function, and minimizing the total loss function is performed to align the visual features and semantic vectors in the shared space while maintaining modality specificity and information integrity. Among them, the calculation formula of the total loss function is And λ1 + λ2 + λ3 = 1. In the formula, Denoted as the total loss function, λ1, λ2, and λ3 denote the corresponding JS divergence constraint loss Adversarial loss And reconstruction loss Weights.

[0048] Optionally, the bearing fault diagnosis system based on the fusion improved capsule network and zero-shot learning includes a multi-modal signal collection and processing module: using a variety of sensors to collect multi-modal data of the bearing in different operating states and performing preprocessing to extract time-frequency domain hybrid features;

[0049] Lightweight capsule network design module: perform dynamic modeling using the single-routing attention mechanism according to the augmented samples and perform feature compression through the pyramid capsule network structure;

[0050] Zero-shot learning module: Establish a fault attribute knowledge graph based on the association rules between bearing physical attributes and fault types, use the GCN graph embedding algorithm to generate transferable semantic vectors, and design a dual-channel adversarial mapping network to align the capsule network features and semantic vectors;

[0051] Fusion model training module: Train and fuse the lightweight capsule network and zero-shot learning. By defining a loss function and supporting incremental learning, update the classification loss of the lightweight capsule network and combine the attribute association of the zero-shot learning module to achieve cross-modal joint optimization;

[0052] Fault diagnosis and evaluation module: According to the cross-modal joint optimization fault diagnosis model, use the start zero-shot inference engine and lightweight capsule network technology to diagnose known and unknown faults of bearings in real time, and use visualization evaluation technology to generate a fault evolution heat map for bearing faults, showing the damage location and severity.

[0053] A computer device, comprising: a memory and a processor; the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning are implemented.

[0054] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning are implemented.

[0055] In the above technical solution, the technical effects and advantages provided by the present invention:

[0056] The present invention extracts time-frequency domain hybrid features by adopting an improved wavelet threshold denoising algorithm and a WPD and VMD fusion decomposition algorithm, and combines the generative adversarial network GAN to expand bearing sample data, breaking through the data dependence of traditional deep learning, realizing small sample data learning, and by training the pyramid capsule network and optimizing the cross-entropy loss function, combining cross-modal joint optimization and zero-shot inference engine, greatly improving the diagnostic accuracy of known faults, and can effectively infer unknown fault types, improving the comprehensiveness of diagnosis, and adopting a 5% parameter fine-tuning network combined with the elastic weight consolidation EWC algorithm to update the lightweight model, which not only ensures the diagnostic effect but also reduces the number of parameters, realizing efficient online real-time diagnosis; at the same time, the gradient clipping method is used in modal alignment to prevent gradient explosion, ensuring the stability of model training, and finally, by generating a fault evolution heat map and performing visualization evaluation, it can intuitively display the bearing damage location and severity, providing clear fault information for maintenance personnel, facilitating timely maintenance measures, and reducing maintenance costs and downtime. Brief Description of the Drawings

[0057] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments described in the present invention. For those of ordinary skill in the art, other accompanying drawings can also be obtained based on these drawings.

[0058] Figure 1 This is a flowchart of the selective maintenance method for multiple states of mechanical equipment based on deep learning in the present invention.

[0059] Figure 2 This is a schematic diagram of the modules of the selective maintenance system for multiple states of mechanical equipment based on deep learning in the present invention. Detailed implementation manners

[0060] Now, the exemplary embodiments will be described more comprehensively with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein. On the contrary, these exemplary embodiments are provided so that the present disclosure will be more complete and comprehensive, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art.

[0061] Embodiment 1

[0062] The present invention provides a Figure 1 bearing fault diagnosis method based on a fusion-improved capsule network and zero-shot learning as shown below, including the following steps:

[0063] S1. Collect multi-modal signals during the operation of the bearing. First, use an improved wavelet threshold denoising algorithm to eliminate environmental noise from the multi-modal signals, and then use a wavelet packet decomposition WPD and variational mode decomposition VMD fusion decomposition algorithm to extract time-frequency domain hybrid features as sample data, and construct an adversarial generative network GAN to expand the training sample data;

[0064] Specifically, the acquisition and processing logic steps of the multi-modal signals are as follows:

[0065] For the acceleration sensor, temperature sensor, and acoustic emission sensor built into the bearing on the electromechanical equipment, synchronously collect multi-modal signals including vibration signals, acoustic emission signals, and temperature signals of the bearing under different working conditions, and calibrate them as Mms(t);

[0066] Use an improved wavelet threshold denoising algorithm to eliminate environmental noise from the multi-modal signal Mms(t) and improve the signal-to-noise ratio. Then the wavelet decomposition formula is and φ j,k (t) = 2 -j / 2 . φ(2 -j ·t - k), where W j,k represents the wavelet coefficient decomposed after the continuous wavelet transform of the multi-modal signal Mms(t), and φj,k (t) represents the wavelet basis function after scale translation, Mms(t) represents the multi-modal signal at the input time step t, j represents the decomposition level, and j = 5, k represents the translation coefficient;

[0067] Combining noise estimation and signal energy, the adaptive improved threshold calculation formula is In the formula, λ j represents the threshold corresponding to the j-th layer wavelet decomposition, σ j represents the noise standard deviation of the j-th layer, N represents the total number of sampling points in the signal length, E j represents the signal energy of the j-th layer;

[0068] After threshold processing using the improved non-linear soft threshold function to reconstruct the wavelet, the improved soft threshold function calculation formula is In the formula, represents retaining the sign of the original wavelet coefficient W j,k of, sign represents the sign function, 0.05 represents the attenuation coefficient for suppressing pseudo Gibbs oscillations, represents the exponential decay term;

[0069] The calculation formula for reconstructing the wavelet is In the formula, represents the new signal of the approximate original signal after threshold processing and reconstruction of the multi-modal signal Mms(t).

[0070] Specifically, the steps for extracting and expanding samples of the time-frequency domain hybrid features are as follows:

[0071] For the new signal of the denoised signal Use WPD to perform i-layer wavelet packet decomposition to generate 8 sub-bands, then the calculation formula of WPD is In the formula, WPD n,i represents the sub-band coefficient of the node number n during the i-th layer decomposition, n represents the node number, i represents the decomposition level, and i = 3, ψ n represents the wavelet packet basis function, k0 represents the parameter for translating the wavelet packet basis function;

[0072] For each sub-band of WPD, use VMD to solve the constraint through variational mode decomposition, and then iteratively solve it through the alternating direction multiplier ADMM algorithm to obtain the IMF components, and set the number of modes and the penalty factor, then the calculation formula of VMD is And, the constraint condition is In the formula, represents the partial derivative with respect to time t, δ(t) represents the delta function, which is used for convolution operation with the mode function u K (t) in VMD, uK (t) represents the K-th modal component, where K = 6, is represented as a complex exponential function, is represented as the objective function of constrained variation, u K is represented as a sequence of modal functions, w K is represented as related to the modal function u K (t) corresponding central frequency sequence, is represented as the kernel function of the Hilbert transform for performing the Hilbert transform on the modal function u K (t), ||||2 2 is represented as the square of the two-norm, where the IMF component is the modal component u K (t);

[0073] For each IMF component u K (t), perform the Hilbert transform to generate the analytic signal z K (t), and calculate the instantaneous energy spectrum E K (t,f), then the calculation formula for the analytic signal is and (j0) 2 = 1, where z K (t) represents the analytic signal corresponding to the K-th modal component u K (t), represents the result of performing the Hilbert transform on the K-th modal component u K (t), j0 represents the imaginary unit;

[0074] The calculation formula for the instantaneous energy spectrum is where E K (t,f) represents the instantaneous energy spectrum of the K-th modal component at time t and frequency f, STFT(z K (t)) represents the result of performing the short-time Fourier transform on the analytic signal z K (t), df and dt respectively represent the differential elements of time and frequency;

[0075] Fuse the energy spectra E K (t) of all IMF components u K (t,f) to generate the matrix E of the time-frequency domain hybrid feature TFJD map with 128×128 pixels K {t,f};

[0076] Perform data augmentation processing on the extracted time-frequency domain hybrid feature TFJD by using the generative adversarial network GAN. The generator inputs the bearing working conditions, the discriminator performs spectral normalization constraints, and the loss functions of adversarial and physical constraints are defined to realize sample expansion.

[0077] Furthermore, the use of the improved wavelet threshold denoising algorithm is an important basis for solving the problem of feature blurring caused by data noise, which further improves the signal-to-noise ratio. The denoised signal and comprehensive time-frequency domain hybrid features enable the model to more accurately identify the fault types and states of bearings, improving the accuracy of fault diagnosis. The WPD-VMD fusion decomposition can cover all features and accurately extract fault feature frequencies. Using GAN to generate and expand samples allows the model to learn more fault features and patterns, making up for the phenomenon of model overfitting caused by small samples, enhancing the generalization ability of the model under different working conditions and fault types, and improving the generalization ability across working conditions. Therefore, accurate fault diagnosis can timely detect potential bearing faults, take corresponding maintenance measures, improve the reliability and safety of bearing operation, and reduce equipment fault risks and maintenance costs in industrial production.

[0078] S2. Perform dynamic modeling based on the sample data of the time-frequency domain hybrid features, use the single-pass feedforward attention mechanism to optimize the dynamic routing algorithm, design a pyramid capsule network structure to compress features, and introduce a bidirectional gated recurrent unit Bi-GRU to achieve cross-level feature interaction;

[0079] Specifically, the optimization steps of the dynamic routing algorithm are as follows:

[0080] Input the time-frequency domain hybrid feature E K {t,f} into the capsule network for feature mapping as And perform a linear transformation on the output of the bottom-layer capsules. The calculation formula for the linear transformation is And In the formula, Represents the Kth capsule vector after linear transformation through the query matrix W q With a dimension of d in , W q Represents a learnable query matrix with a dimension of d out ×d in , Map the input capsule From the d in -dimensional space to the d out -dimensional space for linear transformation. R represents the set of real numbers, and d out Represents the dimension of the output capsule vector after linear transformation, and d in Represents the dimension of the input capsule vector , Represents the Kth input capsule vector;

[0081] Generate routing coefficients through key-value pair attention, that is, calculate the attention weights. The calculation formula for the attention weights is And In the formula, α K,LDenoted as the K-th transformed capsule The routing coefficient from the K-th transformed capsule to the L-th high-level capsule, where Softmax represents the Softmax function, and W K 、W L Denote the key matrix and value matrix that map the transformed capsules to the key space and value space respectively, and both have dimensions of d out ×d out , Denoted as the L-th transformed capsule, Denoted as the scaling factor, which is used to prevent the dot product result from being too large and avoid the problem of gradient disappearance;

[0082] The single-feedforward attention mechanism uses single-feedforward aggregation to replace iterative routing. Then, the calculation formula for the capsule output is And In the formula, E L Denoted as the output of the L-th high-level capsule, with a dimension of d out , Denoted as the weighted sum of all N transformed input capsules according to the attention weight α K,L to obtain a vector with a dimension of d out dimension, Denoted as the L2 norm of the vector after weighted summation, Denoted as the square of the modulus of the vector after weighted summation.

[0083] Specifically, the design steps of the pyramid capsule network structure are as follows:

[0084] Constructing the pyramid capsule network structure includes the bottom layer, the middle layer, and the top layer. Among them, the bottom layer structure is an 8×8 grid, which is used to let each capsule cover a 2×2 area of the signal time-frequency diagram. The bottom layer captures the local micro-defect features of the bearing according to the extracted time-frequency domain mixed features. Then, the calculation formula for the local micro-defect features of the bearing is And the value ranges of x1 and y1 are 0 - 7. In the formula, Denoted as the capsule vector at the position (x1, y1) in the bottom layer capsule network, which is used to capture the micro-defect features of the corresponding local area of the signal time-frequency diagram. x1 and y1 represent the position coordinates in the bottom layer capsule network, and F caps Denoted as the capsule function, which is used to convert the local area data of the input time-frequency diagram into a capsule vector, and E K {t, f} denotes the regional data of the TFJD diagram, and [2x1:2x1 + 2, 2y1:2y1 + 2] denotes the index for slicing the regional data E K {t, f} of the TFJD diagram, which is used to select a 2×2 local area from the time-frequency diagram. This area is the coverage range of each capsule in the bottom layer;

[0085] The middle layer structure is a 4×4 grid, which is used to aggregate the 2×2 area of the 4 bottom capsules and model the bearing component-level correlation features. The calculation formula for the correlation features is and the value ranges of x2 and y2 are 0 - 3. In the formula, represents the capsule vector at the position (x2, y2) in the middle layer capsule network, and Routing represents the routing algorithm, and represents the bottom layer capsule vectors participating in the aggregation;

[0086] The top layer structure is global pooling, which is used to integrate all the capsules in the middle layer and model the system-level fault evolution pattern. The calculation formula for is In the formula, E top represents the output vector of the top layer capsule network, which contains the global feature information of the entire time-frequency map data and is used for the final fault diagnosis of the bearing. Globalrouting represents the global routing algorithm, represents all the capsule vectors in the middle layer;

[0087] The bidirectional gated recurrent unit Bi-GRU is used to achieve cross-layer feature interaction. The Bi-GRU mechanism includes forward propagation, backward propagation, and feature fusion. Among them, forward propagation is the feature transfer from the middle layer capsules to the top layer in the pyramid capsule network; backward propagation is the gradient backpropagation from the top layer capsules to the bottom layer in the pyramid capsule network; feature fusion is to use the fusion matrix to adjust the capsule weights after splicing the bidirectional states of the pyramid capsule network.

[0088] Furthermore, through extracting the time-frequency domain hybrid features for dynamic modeling, it can comprehensively capture the features of the sensor signals inside the bearing and accurately describe the complex characteristics of the signals. In fault diagnosis, it can more accurately identify different types of fault features, improve the accuracy of fault diagnosis. Using the single-pass feedforward attention mechanism in the model can reduce the number of routing iterations, determine the connection weights between capsules through a single feedforward process, reduce the computational complexity. And the pyramid capsule network structure gradually compresses the feature dimension from the bottom layer to the top layer through hierarchical aggregation, reduces the redundant information of the features, and reduces the complexity of the model. In addition, applying the Bi-GRU mechanism strengthens the information exchange and fusion between different layers, enables the model to more comprehensively utilize the feature information of each layer, improves the feature expression ability, significantly improves the training and inference efficiency of the model, reduces the risk of overfitting, improves the generalization ability of the model, and reduces the consumption of computing resources while obtaining the bearing diagnosis results faster.

[0089] S3. Construct a fault attribute knowledge graph by defining the association rules between bearing physical attributes and fault types, adopt the graph embedding algorithm of the graph convolutional network GCN to generate transferable semantic vectors, design the JS divergence constraint of the dual-channel mapping network, and achieve cross-modal adversarial alignment by minimizing the modal difference through the adversarial loss function;

[0090] Specifically, the construction logic of the fault attribute knowledge graph is as follows:

[0091] Record the interpretable association rules of the relationship between the physical attributes and fault types of the bearing based on historical fault records. Among them, the physical attributes of the bearing include rotational speed, temperature, vibration frequency, impact period, energy entropy value, crack length, spalling area, load, etc., and the fault types include inner ring fault, outer ring fault, and rolling element fault;

[0092] Use the physical attributes and fault types of the bearing as nodes and the association rules as edges to construct a fault attribute knowledge graph. Among them, each node and edge can have corresponding attributes to describe the characteristic relationship between the physical attributes and fault types of the bearing;

[0093] Use the graph embedding algorithm of the graph convolutional network GCN to encode the discrete graph relationship into a continuous vector space;

[0094] Take the fault type node features output by the last layer of GCN as semantic vectors.

[0095] Specifically, the steps of cross-modal adversarial alignment are as follows:

[0096] Design two mapping networks to map the semantic vectors of the fault attribute knowledge graph and different modal data of visual features into a common feature space respectively. Among them, the probability distributions of the semantic vectors and visual features are calibrated as P v 、P s ;

[0097] Use JS divergence to measure the distribution difference between the two modalities of semantic P v and visual P s in the common feature space, and make the two distributions as close as possible through constraints. The expression of the JS divergence constraint is In the formula, represents the JS divergence constraint loss, and KL represents the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions;

[0098] Introduce an adversarial loss function to achieve cross-modal adversarial alignment by minimizing the modal difference. Among them, the expression of the adversarial loss function is In the formula, represents the adversarial loss, E v 、Es Denoted as the expectations of the visual feature samples v and the semantic vector samples s respectively, and D is denoted as the discriminator. Denoted as the feature vectors obtained after the visual feature v and the semantic vector s go through the projection operation respectively.

[0099] Adding the data reconstruction loss term ensures the integrity of information, which is used to ensure that after the visual feature and the semantic vector are projected into the shared space and go through a series of processes, the original visual feature and semantic vector can still be recovered by the decoder. Among them, the expression of the reconstruction loss term is In the formula, Denoted as the reconstruction loss. Denoted as the original visual feature vector and semantic vector respectively, and Dec is denoted as the decoder composed of 2 deconvolution layers. Denoted as the square of the 2-norm. Denoted as the visual feature and semantic connection recovered by the decoder respectively.

[0100] The JS divergence constraint loss, the adversarial loss and the reconstruction loss are weighted and summed according to certain weights to obtain the total loss function, and minimizing the total loss function is performed to realize the alignment of the visual feature and the semantic vector in the shared space, while maintaining the modality specificity and information integrity. Among them, the calculation formula of the total loss function is And λ1 + λ2 + λ3 = 1. In the formula, Denoted as the total loss function, and λ1, λ2, λ3 are denoted as the corresponding weights of the JS divergence constraint loss Adversarial loss And reconstruction loss respectively.

[0101] Furthermore, it should be noted that through the automatic graph embedding of the GCN graph embedding algorithm to learn the association rules between the bearing fault attributes (such as rotational speed, temperature, vibration frequency, impact period, energy entropy value, crack length, spalling area and load, etc.) and the fault types (such as inner race fault, outer race fault and rolling element fault), replacing the artificial subjective definition. Through the joint optimization of the JS divergence constraint and the adversarial loss, the modality difference between the visual feature and the semantic vector output by the capsule network is effectively reduced, breaking through the cross-modal mapping bottleneck, and through the decoder reconstruction loss to ensure the information integrity of the feature projection process, avoiding the loss of key diagnostic features, thereby solving the core problems of strong dependence on artificial experience and low recognition rate of unknown faults caused by the modality gap in the traditional method.

[0102] S4. First, train the Pyramid Capsule Network with known fault samples of the bearing, optimize the cross-entropy loss function, freeze the bottom layer, introduce the zero-shot loss term for cross-modal joint optimization to generate the fusion model for fault diagnosis, and then start fine-tuning the network with 5% of the parameter quantity and update the lightweight model by combining with the Elastic Weight Consolidation (EWC) algorithm. During the modal alignment process, the gradient clipping method is used to prevent gradient explosion.

[0103] Specifically, the steps for training the Pyramid Capsule Network are as follows:

[0104] Input the time-frequency feature matrix E K {t, f} into the Pyramid Capsule Network, and successively pass through the bottom capsules of the 8×8 grid, the middle capsules of the 4×4 grid, and the global pooling top capsules.

[0105] Use the cross-entropy loss function with class weights for loss calculation.

[0106] Use the Adam optimizer to update the network parameters, define the learning rate, and apply gradient clipping.

[0107] Freeze the parameters of the bottom layer of the Pyramid Capsule Network, introduce the total loss function, and perform cross-modal joint optimization to generate the fusion model for fault diagnosis, so as to align the visual features and semantic vectors and improve the zero-shot ability.

[0108] Specifically, the steps for updating the lightweight model are as follows:

[0109] When the fusion model diagnoses and detects a new type of fault, start fine-tuning the network with 5% of the parameter quantity to further optimize the performance of the fusion model.

[0110] Use the EWC algorithm to update the parameters of the lightweight model to prevent catastrophic forgetting during the fine-tuning process of the fusion model.

[0111] Combine the loss of the new task cross-entropy for joint optimization, and automatically update the fault knowledge graph embedded in the GCN to create new fault nodes and establish associated edges with attributes.

[0112] Furthermore, by using 5% of the parameter quantity to fine-tune the network instead of full model update and introducing the Elastic Weight Consolidation (EWC) algorithm, it not only meets the real-time update requirements of edge devices but also ensures the continuous stability of the bearing diagnosis process. By using the pre-training of the Pyramid Capsule Network combined with the joint optimization method of cross-modal alignment, the compatibility of the fusion model is improved, and the generalization ability of known fault and location fault recognition is further enhanced, providing an efficient, stable, and sustainable autonomous evolution paradigm for the intelligent operation and maintenance of industrial equipment.

[0113] S5. The fusion model performs online time-frequency analysis on the multi-modal signals input by the bearing. The updated lightweight capsule network directly outputs the classification result as a known fault, activates the zero-shot inference engine, infers the fault type as a position fault by calculating the cosine similarity between the visual features and semantic attributes, generates a fault evolution heat map for the bearing, and uses visualization evaluation technology to display the bearing damage position and severity.

[0114] Specifically, the generation logic of the fault evolution heat map is as follows:

[0115] By using the middle-layer capsule output of the 4×4 grid of the pyramid capsule network to obtain the feature map of the bearing in the time-frequency domain, and each capsule corresponding to a specific area of the time-frequency map, the activation vector is used to locate the fault-related area;

[0116] Calculate the fault category weights through gradient backpropagation to determine the importance of different regions for fault classification;

[0117] Use bilinear interpolation to upsample to 128×128 pixels, and calculate the heat matrix value after matching the size of the original time-frequency map;

[0118] Combine the heat matrix value with the feature map to generate a heat map, showing the damage position and severity.

[0119] Furthermore, by generating a fault evolution heat map for the bearing, that is, using the capsule network feature activation positioning and gradient backpropagation technology and visualization evaluation technology, it can intuitively display the spatial distribution and severity of bearing damage, provide intuitive fault information for operators, facilitate timely maintenance measures, reduce maintenance costs and downtime, and realize the functions of real-time and efficient diagnosis of bearings, accurate inference of unknown faults, and visual and accurate positioning, and can guide maintenance personnel to give priority to dealing with high-risk parts.

[0120] Embodiment 2

[0121] The present invention provides a Figure 2 bearing fault diagnosis system based on a fusion improved capsule network and zero-shot learning as shown, including a multi-modal signal collection and processing module: using a variety of sensors to collect multi-modal data of the bearing in different operating states, and performing preprocessing to extract time-frequency domain mixed features, which are used to eliminate environmental noise while extracting time-frequency domain mixed features, and then generating augmented samples according to the extracted time-frequency domain mixed features;

[0122] Lightweight capsule network design module: According to the augmented samples, use the single-routing attention mechanism for dynamic modeling, and perform feature compression through the pyramid capsule network structure, which is used to simplify the capsule network by adjusting the number of layers and capsules of the capsule network, thereby enhancing the feature transfer ability of the network and improving the training speed and performance of the lightweight capsule network model;

[0123] Zero-shot learning module: Establish a fault attribute knowledge graph according to the association rules between bearing physical attributes and fault types, generate transferable semantic vectors using the GCN graph embedding algorithm, and design a dual-channel adversarial mapping network to align the capsule network features and semantic vectors, for identifying and classifying unseen fault types by learning the semantic information of samples, thereby improving the generalization ability of the model and solving the problem of low generalization recognition rate of unknown faults in the cross-condition scenario for existing zero-shot transfer methods;

[0124] Fusion model training module: Train and fuse the lightweight capsule network and zero-shot learning. By defining the loss function and supporting incremental learning, update the classification loss of the lightweight capsule network and combine the attribute association of the zero-shot learning module to achieve cross-modal joint optimization, for enabling the two modules to cooperate with each other to jointly complete the fault diagnosis task and construct a unified fault diagnosis model;

[0125] Fault diagnosis and evaluation module: According to the cross-modal jointly optimized fault diagnosis model, use the zero-shot inference engine and lightweight capsule network technology to diagnose known and unknown faults of the bearing in real time, and use visualization evaluation technology to generate a fault evolution heat map for the bearing fault to display the damage location and severity, for performing degree evaluation while diagnosing the bearing fault in real time.

[0126] The above formulas are all dimensionless and take their numerical calculations. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the real situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0127] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0128] It should be understood that in various embodiments of the present application, the sequence numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0129] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0130] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all such changes or substitutions should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A bearing fault diagnosis method based on a fusion-improved capsule network and zero-shot learning, characterized in that It includes the following steps: S1. Collect multi-modal signals during the operation of the bearing. First, use an improved wavelet threshold denoising algorithm to eliminate environmental noise from the multi-modal signals, and then use a fusion decomposition algorithm of wavelet packet decomposition (WPD) and variational mode decomposition (VMD) to extract time-frequency domain hybrid features as sample data, and construct a generative adversarial network (GAN) to expand the training sample data; S2. Perform dynamic modeling based on the sample data of time-frequency domain hybrid features, use a single-pass feed-forward attention mechanism to optimize the dynamic routing algorithm, design a pyramid capsule network structure to compress features, and introduce a bidirectional gated recurrent unit (Bi-GRU) to achieve cross-level feature interaction; S3. Construct a fault attribute knowledge graph by defining the association rules between the physical attributes and fault types of the bearing, use a graph convolutional network (GCN) graph embedding algorithm to generate transferable semantic vectors, design a JS divergence constraint for a two-channel mapping network, and achieve cross-modal adversarial alignment by minimizing the modal difference through an adversarial loss function; S4. First, train the pyramid capsule network with known fault samples of the bearing, optimize the cross-entropy loss function, freeze the bottom layer and then introduce a zero-shot loss term for cross-modal joint optimization to generate a fusion model for fault diagnosis. Then, start fine-tuning the network with 5% of the number of parameters and combine the elastic weight consolidation (EWC) algorithm to update the lightweight model. Among them, the gradient clipping method is used to prevent gradient explosion during the modal alignment process; S5. The fusion model performs online time-frequency analysis on the multi-modal signals input by the bearing. The updated lightweight capsule network directly outputs the classification result as a known fault. Start the zero-shot inference engine, infer the fault type as a position fault by calculating the cosine similarity between the visual features and semantic attributes, generate a fault evolution heat map for the bearing, and use visualization evaluation technology to display the bearing damage position and severity.

2. The bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to claim 1, wherein, The acquisition and processing logic steps of the multi-modal signals are as follows: For the bearings on the electromechanical equipment, install acceleration sensors, temperature sensors, and acoustic emission sensors, and synchronously collect multi-modal signals including vibration signals, acoustic emission signals, and temperature signals of the bearings under different working conditions, and calibrate them as Mms(t); Use the improved wavelet threshold denoising algorithm to eliminate environmental noise from the multimodal signal Mms(t) and improve the signal-to-noise ratio. The wavelet decomposition formula is and φ j,k (t) = 2 -j / 2 . φ(2 -j ·t - k), where W j,k represents the wavelet coefficients obtained by continuously wavelet-transforming the multimodal signal Mms(t), φ j,k (t) represents the wavelet basis function after scale translation, Mms(t) represents the multimodal signal at the input time step t, j represents the decomposition level and j = 5, and k represents the translation coefficient; Combined with noise estimation and signal energy, the adaptive improved threshold calculation formula is In the formula, λ j represents the threshold corresponding to the j-th layer wavelet decomposition, σ j represents the noise standard deviation of the j-th layer, N represents the total number of sampling points in the signal length, and E j represents the signal energy of the j-th layer; After threshold processing with an improved non-linear soft threshold function and then reconstructing the wavelet, the calculation formula for the improved soft threshold function is In the formula,[[]]END]] denotes retaining the sign of the original wavelet coefficient W j,k , sign denotes the sign function, and 0.05 denotes the attenuation coefficient for suppressing pseudo Gibbs oscillations denotes the exponential decay term; The calculation formula for the reconstructed wavelet is In the formula, represents the new signal that is approximately the original signal after threshold processing and reconstruction of the multimodal signal Mms(t).

3. The bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to claim 2, characterized in that, The steps for extracting and expanding samples of the time-frequency domain hybrid features are as follows: New signal for the denoised signal Perform i - layer wavelet packet decomposition using WPD to generate 8 sub - bands. Then the calculation formula of WPD is In the formula, WPD n,i represents the sub - band coefficient of the node numbered n during the i - th layer decomposition. n represents the node number, i represents the decomposition layer number, and i = 3, ψ n represents the wavelet packet basis function, and k0 represents the parameter of the translation of the wavelet packet basis function; Solve the constraints of variational mode decomposition (VMD) for each sub-band of WPD, and then iteratively solve it through the alternating direction method of multipliers (ADMM) algorithm to obtain the IMF components. Set the number of modes and the penalty factor. Then, the calculation formula of VMD is And the constraint condition is In the formula,[[]] denotes the partial derivative with respect to time t, and δ(t) denotes the delta function, which is used to convolve with the mode function u K (t), and u K (t) denotes the Kth mode component, and K = 6 denotes the complex exponential function denotes the objective function of the constrained variation, and u K denotes the sequence of mode functions, and w K denotes the sequence of center frequencies corresponding to the mode function u K (t) denotes the kernel function of the Hilbert transform, which is used to perform the Hilbert transform on the mode function u K (t), and ||||2 2 denotes the square of the two-norm. Among them, the IMF component is the mode component u K (t); For each IMF component u K (t), perform the Hilbert transform to generate the analytic signal z K (t), and calculate the instantaneous energy spectrum E K (t,f). Then the calculation formula for the analytic signal is and (j0) 2 = 1. In the formula, z K (t) is expressed as the analytic signal corresponding to the K-th modal component u K (t), is expressed as the result of performing the Hilbert transform on the K-th modal component u K (t), and j0 represents the imaginary unit; The calculation formula for the instantaneous energy spectrum is where E K (t,f) represents the instantaneous energy spectrum of the K-th modal component at time t and frequency f, and STFT(z K (t)) represents the result of performing a short-time Fourier transform on the analytic signal z K (t), and df and dt respectively represent the differential elements of time and frequency; Fuse all IMF components u K (t) of the energy spectrum E K (t, f), generate the matrix E of the time-frequency domain hybrid feature TFJD map with 128×128 pixels K {t, f}; Perform data augmentation on the extracted time-frequency domain hybrid features (TFJD) by using a generative adversarial network (GAN). Input the bearing working condition conditions into the generator, perform spectral normalization constraints on the discriminator, and define the loss functions for adversarial and physical constraints, thereby realizing sample expansion.

4. The bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to claim 3, wherein The optimization steps of the dynamic routing algorithm are as follows: Mix the time-frequency domain features E K {t,f} and input them into the capsule network for feature mapping as Then perform a linear transformation on the output of the bottom-layer capsules. The calculation formula for the linear transformation is And In the formula represents the K-th capsule vector after linear transformation through the query matrix W q with a dimension of d in , W q represents a learnable query matrix with a dimension of d out ×d in The input capsule is mapped from a d in -dimensional space to a d out -dimensional space for linear transformation. R represents the set of real numbers, and d out represents the dimension of the output capsule vector after linear transformation, and d in represents the dimension of the input capsule vector , and represents the K-th input capsule vector; Generate routing coefficients through key-value pair attention, that is, calculate attention weights. The calculation formula for the attention weights is And In the formula, α K,L Represents the routing coefficient from the K-th transformed capsule To the L-th high-level capsule. Softmax represents the Softmax function, and W K , W L Respectively represent the key matrix and value matrix that map the transformed capsule To the key space and value space, and both have dimensions of d out ×d out , Represents the L-th transformed capsule, Represents the scaling factor; The single-pass feedforward attention mechanism uses single-pass feedforward aggregation instead of iterative routing. The calculation formula for the capsule output is And In the formula, E L represents the output of the L-th high-level capsule, with a dimension of d out , represents the weighted sum of all N transformed input capsules according to the attention weight α K,L to obtain a d out -dimensional vector, represents the 2-norm of the vector after weighted summation, represents the square of the modulus of the vector after weighted summation.

5. The bearing fault diagnosis method based on the improved capsule network fused with zero-shot learning according to claim 4, wherein, The design steps of the pyramid capsule network structure are as follows: The construction of the pyramid capsule network structure includes the bottom layer, the middle layer, and the top layer. Among them, the bottom layer structure is an 8×8 grid, which is used to make each capsule cover a 2×2 area of the signal time-frequency diagram. The bottom layer captures the local micro-defect features of the bearing according to the extracted time-frequency domain mixed features. The calculation formula for the local micro-defect features of the bearing is and the value ranges of x1 and y1 are 0 - 7. In the formula, represents the capsule vector at the coordinate (x1, y1) in the bottom capsule network. x1 and y1 represent the position coordinates in the bottom capsule network, and F caps represents the capsule function, and E K {t, f} represents the regional data of the TFJD diagram, and [2x1:2x1 + 2, 2y1:2y1 + 2] represents the index for slicing the regional data E K {t, f} of the TFJD diagram; The middle layer structure is a 4×4 grid, which is used to aggregate the 2×2 areas of 4 bottom-layer capsules, and model the bearing component-level correlation features. The calculation formula for the correlation features is and the value ranges of x2 and y2 are 0 - 3. In the formula, represents the capsule vector at the position (x2, y2) in the middle-layer capsule network, and Routing represents the routing algorithm, and represent the bottom-layer capsule vectors participating in the aggregation; The high-level structure is global pooling, which is used to integrate all the capsules in the middle layer and model the system-level fault evolution pattern. The calculation formula of is In the formula, E top represents the output vector of the top-layer capsule network, which contains the global feature information of the entire time-frequency map data and is used for the final fault diagnosis of the bearing. Globalrouting represents the global routing algorithm. represents all the capsule vectors in the middle layer; Use a bidirectional gated recurrent unit (Bi-GRU) to achieve cross-level feature interaction. The Bi-GRU mechanism includes forward propagation, backward propagation, and feature fusion. Among them, forward propagation is the feature transfer from the middle capsules to the top layer in the pyramid capsule network; backward propagation is the gradient backpropagation from the top layer capsules to the bottom layer in the pyramid capsule network; feature fusion is to adjust the capsule weights after splicing the bidirectional states of the pyramid capsule network using a fusion matrix.

6. The bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to claim 5, wherein The construction logic of the fault attribute knowledge graph is as follows: An interpretable association rule for recording the relationship between the physical properties and fault types of bearings based on historical fault records; Construct a fault attribute knowledge graph with the physical properties and fault types of bearings as nodes and the association rules as edges; Use the graph convolutional network GCN graph embedding algorithm to encode the discrete graph relationship into a continuous vector space; Take the fault type node features output by the last layer of GCN as semantic vectors.

7. The bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to claim 6, wherein The steps of the cross-modal adversarial alignment are as follows: Design two mapping networks to map the semantic vectors of the fault attribute knowledge graph and different modal data of visual features into a common feature space respectively, where the probability distributions of the semantic vectors and visual features are calibrated as P v , P s ; Use JS divergence to measure semantic P v and visual P s The distribution difference between these two modalities in the common feature space, and through constraints, make the two distributions as close as possible. The expression of the JS divergence constraint is In the formula,[[]]END]] is expressed as the JS divergence constraint loss, and KL is expressed as the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions; Introduce an adversarial loss function to achieve cross-modal adversarial alignment by minimizing the modal difference. Among them, the expression of the adversarial loss function is In the formula,[[]]END]] denotes the adversarial loss, and E v and E s denote the expectations of the visual feature samples v and the semantic vector samples s respectively, and D denotes the discriminator.[[]]END]] denote the feature vectors obtained after the visual feature v and the semantic vector s undergo projection operations respectively.[[]]END]] Adding a data reconstruction loss term ensures the integrity of information, where the expression of the reconstruction loss term is In the formula is expressed as the reconstruction loss respectively represent the original visual feature vector and semantic vector, and Dec represents a decoder composed of two deconvolution layers is expressed as the square of the 2-norm respectively represent the visual features and semantic connections recovered by the decoder The JS divergence constraint loss, adversarial loss, and reconstruction loss are weighted and summed according to certain weights to obtain the total loss function, and minimizing the total loss function is performed for the alignment of the visual features and semantic vectors in the shared space while maintaining modality specificity and information integrity. The calculation formula of the total loss function is and λ1 + λ2 + λ3 = 1. In the formula, is denoted as the total loss function, and λ1, λ2, and λ3 are respectively denoted as the weights corresponding to the JS divergence constraint loss adversarial loss and reconstruction loss respectively.

8. The bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to claim 7, wherein, The bearing fault diagnosis system based on the fusion of an improved capsule network and zero-shot learning includes a multi-modal signal collection and processing module: using multiple sensors to collect multi-modal data of the bearing in different operating states, and preprocessing to extract time-frequency domain hybrid features; A lightweight capsule network design module: perform dynamic modeling using the single-routing attention mechanism according to the augmented samples, and perform feature compression through the pyramid capsule network structure; A zero-shot learning module: establish a fault attribute knowledge graph according to the association rules between the physical properties and fault types of bearings, generate transferable semantic vectors using the GCN graph embedding algorithm, and design a dual-channel adversarial mapping network to align the capsule network features and semantic vectors; A fusion model training module: train and fuse the lightweight capsule network and zero-shot learning, update the classification loss of the lightweight capsule network combined with the attribute association of the zero-shot learning module by defining a loss function and supporting incremental learning, and achieve cross-modal joint optimization; A fault diagnosis and evaluation module: according to the cross-modal jointly optimized fault diagnosis model, use the start zero-shot inference engine and lightweight capsule network technology to diagnose known and unknown faults of the bearing in real time, and use visualization evaluation technology to generate a fault evolution heat map for the bearing fault to display the damage location and severity.

9. A computer device, comprising: A memory and a processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the bearing fault diagnosis method based on the fusion of an improved capsule network and zero-shot learning according to any one of claims 1 to 8 are implemented.

Images