Mechanical fault diagnosis method and system based on spiking neural network

Abstract

The invention relates to the technical field of mechanical fault diagnosis, in particular to a mechanical fault diagnosis method and system based on a pulse neural network. According to the invention, a fault diagnosis network is constructed, and the fault diagnosis network comprises a pulse time-frequency repair module, a pulse space-time attention module and a pulse residual network; wherein the pulse time-frequency repairing module comprises a time-frequency convolution layer and a pulse repairing sub-module; time-frequency distribution characteristics of the collected mechanical vibration signals are extracted by using a time-frequency convolution kernel, the time-frequency distribution characteristics are input into a pulse repairing sub-module, and a time-frequency characteristic group of a pulse residual network time step length number is output; inputting each time-frequency feature group into a pulse space-time attention module, and extracting key time-frequency features of each time-frequency feature group; and taking the key time-frequency characteristics of each time-frequency characteristic group as the input of each time step length of the pulse residual network, and outputting a prediction fault category label. According to the invention, the mechanical fault diagnosis precision is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mechanical fault diagnosis, and in particular to a mechanical fault diagnosis method and system based on a spiking neural network. Background Art

[0002] Mechanical system components such as rolling bearings and gearboxes are commonly used in systems such as ships, aviation, and wind turbines, and play an important role in the normal operation of mechanical systems. However, in an increasingly complex working environment, these mechanical components often suffer from different forms of faults, seriously affecting the economic benefits of enterprises and threatening the lives of operators. Therefore, it is crucial to accurately detect the occurrence of faults and diagnose the specific fault types. In recent years, fault diagnosis for mechanical system components has received extensive attention, giving rise to the proposal and extension of many theories. Thanks to big data technology, most current fault diagnosis methods are data-driven. Generally speaking, data-driven includes traditional signal processing methods and deep learning methods. The former has high physical interpretability, while deep learning methods mainly based on artificial neural networks (ANNs) have the advantages of feature learning and high accuracy. Nowadays, emerging fault diagnosis methods are not just simple signal processing methods. More often, traditional signal processing methods are used as preprocessing methods to extract features from raw data and then input them into deep learning models, which not only improves the diagnostic accuracy of deep learning models but also increases a certain degree of physical interpretability. However, even so, its interpretability is limited outside the network model, and there are two further directions for thinking about interpretability: one is how to embed signal processing methods into the network; the other is whether there is a network that is more interpretable than the ANN itself. For the former, there have been relevant studies on combining time-frequency transforms with convolutional neural networks (CNNs), and for the latter, undoubtedly, the spiking neural network (SNN) is the first model to consider.

[0003] Inspired by the information transmission of biological neurons through spike timing, spiking neural networks, known as the "third-generation neural networks", have been proposed. The structure of spiking neural networks originates from biological neurons, transmits discrete spike timing information through synaptic connections, and classifies according to the membrane potential threshold, so it has strong biological interpretability. In addition, the discrete binary characteristic also enables it to greatly save energy consumption when deployed on hardware. This has made spiking neural networks receive more and more attention in recent years. In the initial stage of spiking neural networks, many studies focused on how to solve the non-differentiable training problem of spiking neural networks. Many scholars have conducted in-depth research on this problem and proposed three main training methods: bionic training method, ANN-to-spiking neural network conversion method, and surrogate gradient (SG) method. The surrogate gradient method uses a surrogate gradient to approximate the gradient of the non-differentiable spike function in backpropagation, enabling the direct training of spiking neural networks, so it is widely used. The existing research content on spiking neural networks mainly includes encoding methods, network structures, and spiking neurons. Others also cover related research such as lightweight network design of spiking neural networks, improvement of adversarial robustness of spiking neural networks, and reversible design of spiking neural networks. In terms of applications, spiking neural networks are widely used in image recognition and target detection, while the research on mechanical fault diagnosis methods based on spiking neural networks is less.

[0004] As an important signal processing method, the time-frequency transformation method has high physical interpretability and can extract the time-frequency characteristics of mechanical component faults. Therefore, the time-frequency transformation method is commonly used as a feature extraction means for fault diagnosis to further improve physical interpretability and the diagnostic accuracy of the network. Currently, most intelligent diagnosis methods combining time-frequency transformation and spiking neural networks use time-frequency transformation as a preprocessing method, perform time-frequency transformation on one-dimensional vibration signals outside the model, and then input the time-frequency characteristics into the spiking neural network for training.

[0005] However, in mechanical fault diagnosis, the operating conditions of equipment are complex and changeable. Different types of faults may exhibit different vibration characteristics, and the same fault may also have different manifestations under different operating conditions. For example, under high-speed and heavy-load conditions, the wear fault of gears may generate high-frequency and high-amplitude vibration signals, while under low-speed and light-load conditions, the characteristics of the same wear fault may not be so obvious in the vibration signal. The time-frequency transformation is based on a fixed mathematical algorithm to transform the collected mechanical vibration signals and cannot adaptively adjust the transformation method according to the specific fault situation and operating conditions, making it difficult to accurately extract the most representative fault characteristics. As a result, the mechanical fault diagnosis model is prone to misjudging other non-fault characteristics or interference signals as fault characteristics, or missing the true fault signals, leading to misdiagnosis or missed diagnosis of the mechanical fault diagnosis model and thus affecting the diagnostic accuracy of mechanical fault diagnosis. Summary of the Invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the fact that the time-frequency transformation cannot be adaptively adjusted according to the fault conditions and operating conditions, resulting in a large amount of redundant information in the time-frequency characteristics, while the key information that can truly characterize the fault is weakened, reducing the diagnostic accuracy of the mechanical fault diagnosis model.

[0007] To solve the above technical problem, the present invention provides a mechanical fault diagnosis method based on a spiking neural network, comprising the following steps:

[0008] Construct a fault diagnosis network, the fault diagnosis network comprising: a spiking time-frequency repair module, a spiking spatio-temporal attention module, and a spiking residual network; wherein, the spiking time-frequency repair module comprises: a time-frequency convolutional layer, a spiking repair sub-module;

[0009] Input the collected mechanical vibration signal into the time-frequency convolutional layer, and output the time-frequency distribution characteristics;

[0010] Input the time-frequency distribution characteristics into the spiking repair sub-module, and output T time-frequency feature groups, including:

[0011] According to the number of time steps T of the spiking residual network, evenly divide the feature length of each channel in the time-frequency distribution characteristics in the time dimension into T equal parts, take the length W of each equal part as the feature width input for each time step, and combine each equal part with the frequency dimension H of the channel to form a time-frequency feature group with a dimension of H×W, obtaining T time-frequency feature groups with a dimension of H×W;

[0012] Input each time-frequency feature group into the spiking spatio-temporal attention module, and extract the key time-frequency features of each time-frequency feature group;

[0013] Take the key time-frequency features of each time-frequency feature group as the input for each time step of the spiking residual network, and output the predicted fault category label.

[0014] Preferably, inputting the collected mechanical vibration signal into the time-frequency convolutional layer and outputting the time-frequency distribution characteristics includes:

[0015] Pass the collected mechanical vibration signal through the real part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolutional layer, and perform convolution along the length direction to obtain the real part features corresponding to each channel;

[0016] Pass the collected mechanical vibration signal through the imaginary part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolutional layer, and perform convolution along the length direction to obtain the imaginary part features corresponding to each channel;

[0017] Based on the modulus of the real part feature and the modulus of the imaginary part feature corresponding to each channel, a feature sequence corresponding to each channel is obtained. The feature sequence corresponding to each channel is used as the time dimension of each channel, and the number of channels in the time-frequency convolutional layer is used as the width dimension of each channel. The time dimension and width dimension of each channel are combined as the feature representation of each channel of the time-frequency distribution feature, and the time-frequency distribution feature is obtained.

[0018] Preferably, the mechanical vibration signal collected is convolved along the length direction through the real part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolutional layer to obtain the real part feature corresponding to each channel. The formula is:

[0019]

[0020] The mechanical vibration signal collected is convolved along the length direction through the imaginary part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolutional layer to obtain the imaginary part feature corresponding to each channel. The formula is:

[0021]

[0022] The feature sequence corresponding to each channel is obtained based on the modulus of the real part feature and the modulus of the imaginary part feature corresponding to each channel. The formula is:

[0023]

[0024] where h k is the feature sequence corresponding to the k-th channel. The feature sequence corresponding to each channel is used as the time dimension of each channel, and the number of channels in the time-frequency convolutional layer is used as the width dimension of each channel. The time dimension and width dimension of each channel are combined to obtain the time-frequency distribution feature h, h = [h 1 , h 2 … h K , K is the number of channels in the time-frequency convolutional layer, h k,real is the real part feature corresponding to the k-th channel, h k,imag is the imaginary part feature corresponding to the k-th channel, is the real part kernel of the time-frequency convolution kernel corresponding to the k-th channel in the time-frequency convolutional layer, is the imaginary part kernel of the time-frequency convolution kernel corresponding to the k-th channel in the time-frequency convolutional layer, x is the collected mechanical vibration signal, * is the convolution operation, and θ is the trainable control parameter of the kernel function.

[0025] Preferably, the pulse spatio-temporal attention module includes: a time attention unit, a spatial attention unit, and a ReLU function connected in sequence; the time attention unit includes: a max pooling layer, an average pooling layer, a shared multi-layer perceptron, and a Sigmoid activation function; the spatial attention unit includes: a max pooling layer, an average pooling layer, a 7×7 convolution, and a Sigmoid activation function.

[0026] Preferably, the step of inputting each time-frequency feature group into the pulse spatio-temporal attention module to extract the key time-frequency features of each time-frequency feature group includes:

[0027] Inputting each time-frequency feature group into the time attention unit to extract the time attention features of each time-frequency feature group, including:

[0028] Passing each time-frequency feature group through a max pooling layer and a shared multi-layer perceptron in sequence to obtain the time-frequency max pooling perception feature of each time-frequency feature group;

[0029] Passing each time-frequency feature group through an average pooling layer and a shared multi-layer perceptron in sequence to obtain the time-frequency average pooling perception feature of each time-frequency feature group;

[0030] Adding the time-frequency max pooling perception feature and the time-frequency average pooling perception feature of each time-frequency feature group, and then passing through a Sigmoid activation function to obtain the time weight of each time-frequency feature group;

[0031] Element-wise multiplying the time weight of each time-frequency feature group by the time-frequency feature group to obtain the time attention feature of each time-frequency feature group;

[0032] Inputting the time attention feature of each time-frequency feature group into the spatial attention unit to extract the spatial attention feature of each time-frequency feature group, including:

[0033] Passing the time attention feature of each time-frequency feature group through a max pooling layer and a shared multi-layer perceptron in sequence to obtain the time attention max pooling perception feature of each time-frequency feature group;

[0034] Passing the time attention feature of each time-frequency feature group through an average pooling layer and a shared multi-layer perceptron in sequence to obtain the time attention average pooling perception feature of each time-frequency feature group;

[0035] Concatenating the time attention max pooling perception feature and the time attention average pooling perception feature of each time-frequency feature group, and then passing through a 7×7 convolution and a Sigmoid activation function in sequence to obtain the spatial weight of each time-frequency feature group;

[0036] Element-wise multiplying the spatial weight of each time-frequency feature group by the time attention feature;

[0037] Obtain the spatial attention features of each time-frequency feature group;

[0038] Pass the spatial attention features of each time-frequency feature group through the ReLU activation function to obtain the key time-frequency features of each time-frequency feature group.

[0039] Preferably, the pulsed residual network includes: a downsampling layer, a first basic layer, a second basic layer, a third basic layer, a fourth basic layer, and a fully connected layer connected in sequence;

[0040] Among them, the downsampling layer includes: a convolutional layer, a batch normalization layer, a pulsed neuron, and a max pooling layer connected in sequence. Each basic layer is composed of two basic blocks with the same number of channels. Each basic block includes: a 3×3 convolution, a batch normalization layer, a pulsed neuron, a 3×3 convolution, a batch normalization layer, and a pulsed neuron connected in sequence.

[0041] Preferably, the pulsed residual network is any one of SEW-ResNet18, MS-ResNet18, and SEW-ResNet50.

[0042] Preferably, the pulsed neuron in the pulsed residual network is any one of LIF neurons, PSN neurons, and PLIF neurons.

[0043] Preferably, the dynamics formula of the PLIF neuron is:

[0044] H t =V t-1 +k(a)(-(V t-1 -V reset )+X t )

[0045] S t =Θ(H t -V th )

[0046] V t =H t (1-S t )+V reset S t ,

[0047] Among them, H t is the membrane potential at time step t after neuron dynamics, V t is the membrane potential at time step t after pulse triggering, V t-1 is the membrane potential at time step t-1 after pulse triggering, V reset is the reset membrane potential, X t is the input of the PLIF neuron, S tis the output pulse at time step t, Θ(.) is the Heaviside step function, and V th is the membrane potential threshold, k(a) is the sigmoid activation function, exp(.) is the exponential function with the natural constant as the base, and a is the trainable parameter.

[0048] The present invention also provides a mechanical fault diagnosis system based on a spiking neural network, including:

[0049] A model construction module for constructing a fault diagnosis network, where the fault diagnosis network includes: a spiking time-frequency repair module, a spiking spatio-temporal attention module, and a spiking residual network; among them, the spiking time-frequency repair module includes: a time-frequency convolutional layer and a spiking repair sub-module;

[0050] A time-frequency distribution feature extraction module for inputting the collected mechanical vibration signal into the time-frequency convolutional layer and outputting the time-frequency distribution feature;

[0051] A time-frequency feature group acquisition module for inputting the time-frequency distribution feature into the spiking repair sub-module and outputting T time-frequency feature groups, including:

[0052] According to the number T of time steps of the spiking residual network, the feature length of each channel in the time-frequency distribution feature is evenly divided into T equal parts in the time dimension, and the length W of each equal part is used as the feature width input for each time step. Each equal part and the frequency dimension H of the channel form a time-frequency feature group with dimensions H×W, and T time-frequency feature groups with dimensions H×W are obtained;

[0053] A key time-frequency feature extraction module for inputting each time-frequency feature group into the spiking spatio-temporal attention module and extracting the key time-frequency features of each time-frequency feature group;

[0054] A prediction module for using the key time-frequency features of each time-frequency feature group as the input for each time step of the spiking residual network and outputting the predicted fault category label.

[0055] The above technical solutions of the present invention have the following beneficial effects compared with the prior art:

[0056] A mechanical fault diagnosis method and system based on a spiking neural network according to the present invention proposes a spiking time-frequency patching module. By inputting the collected mechanical vibration signal into the time-frequency convolutional layer, the time-frequency convolutional layer is used to extract the time-frequency distribution characteristics. The time-frequency distribution characteristics are input into the spiking patching sub-module, and T time-frequency feature groups are output. This module embeds the time-frequency transformation method into the network, enabling the one-dimensional vibration signal to complete the time-frequency transformation with a time dimension inside the network. Different from the time-frequency transformation of traditional fixed algorithms, this module can adaptively adjust the time-frequency transformation method according to the data feedback in the learning process of the fault diagnosis model. When facing the complex and changeable operating conditions in mechanical fault diagnosis, it can dynamically adjust the parameters for the vibration signal characteristics of different fault types, more accurately extract the features that can best reflect the essence of the fault, and avoid the feature extraction deviation caused by fixed algorithms, thus significantly improving the accuracy and pertinence of feature extraction. In addition, embedding the time-frequency transformation method into the network also improves the physical interpretability of the spiking neural network, simplifies the fault diagnosis process, reduces manual intervention, and improves the real-time performance and accuracy of diagnosis.

[0057] Since the representation method of the time dimension of the time-frequency distribution characteristics does not match the time step of the spiking neural network and cannot meet the time dimension requirements of the spiking neural network, the spiking neural network cannot directly process the time-frequency distribution characteristics. Therefore, the present invention uses the spiking patching sub-module to discretize the continuous time-frequency distribution characteristics according to the time step required by the spiking neural network. According to the number T of time steps of the spiking residual network, the feature length of each channel in the time-frequency distribution characteristics in the time dimension is evenly divided into T equal parts, and the length W of each equal part is used as the feature width input for each time step. Each equal part and the frequency dimension H of the channel form a time-frequency feature group of H×W, and T time-frequency feature groups of H×W are obtained, so that the time-frequency distribution characteristics can match the time dimension of the spiking neural network to meet the requirements of the spiking neural network for time information processing, and an end-to-end mechanical fault diagnosis process can be realized. The end-to-end diagnosis process does not require manual intervention and processing in the intermediate links, reduces the errors generated in the intermediate process, and improves the efficiency and accuracy of the mechanical fault diagnosis by the fault diagnosis network.

[0058] By inputting each time-frequency feature group into the pulse spatio-temporal attention module, the key time-frequency features of each time-frequency feature group are extracted. The time attention unit calculates the time-frequency maximum pooling perception feature and the time-frequency average pooling perception feature respectively, and after adding them and passing through the Sigmoid activation function, the time weight is obtained. This method can adaptively assign weights to different time steps of each time-frequency feature group, improving the feature representation ability of the spiking neural network in the time dimension. The spatial attention unit processes the time attention features by combining maximum pooling and average pooling, and then after concatenating the obtained features and passing through convolution and the Sigmoid activation function, the spatial weight is obtained. By assigning the spatial weight, the feature information of the important region is highlighted, improving the capture ability of the spiking neural network for the features in the time dimension and spatial dimension of the collected mechanical vibration signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to make the content of the present invention easier to be clearly understood, the following further describes the present invention in detail according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0060] Figure 1 is a schematic flow chart of a mechanical fault diagnosis method of a spiking neural network according to the present invention.

[0061] Figure 2 is a structural diagram of a pulse time-frequency repair module.

[0062] Figure 3 is a schematic diagram of the training of a fault diagnosis network.

[0063] Figure 4 is a schematic diagram of the testing of a fault diagnosis network.

[0064] Figure 5 is a t-SNE dimensionality reduction visualization graph of the classification results of a variable working condition gearbox fault vibration data set.

[0065] Figure 6 is a confusion matrix graph of the classification results of a variable working condition gearbox fault vibration data set. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] The following further describes the present invention in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the examples given are not intended to limit the present invention.

[0067] Referring to Figure 1 as shown, Embodiment 1 of the present invention provides a mechanical fault diagnosis method of a spiking neural network, including the following steps:

[0068] Step S1: Construct a fault diagnosis network, where the fault diagnosis network includes: a pulse time-frequency repair module, a pulse spatio-temporal attention module, and a pulse residual network; among them, the pulse time-frequency repair module includes: a time-frequency convolution layer and a pulse repair sub-module;

[0069] As Figure 2 shown, Figure 2 is the structural diagram of the pulse time-frequency repair module.

[0070] Step S2: Input the collected mechanical vibration signal into the time-frequency convolution layer and output the time-frequency distribution feature;

[0071] In this embodiment, specifically, inputting the collected mechanical vibration signal into the time-frequency convolution layer and outputting the time-frequency distribution feature includes:

[0072] Step S21: Pass the collected mechanical vibration signal through the real part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolution layer, and perform convolution along the length direction to obtain the real part feature corresponding to each channel. The formula is:

[0073]

[0074] Step S22: Pass the collected mechanical vibration signal through the imaginary part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolution layer, and perform convolution along the length direction to obtain the imaginary part feature corresponding to each channel. The formula is:

[0075]

[0076] Step S23: Based on the modulus of the real part feature and the modulus of the imaginary part feature corresponding to each channel, obtain the feature sequence corresponding to each channel. Take the feature sequence corresponding to each channel as the time dimension of each channel, and the number of channels in the time-frequency convolution layer as the width dimension of each channel. Combine the time dimension and width dimension of each channel as the feature representation of each channel of the time-frequency distribution feature to obtain the time-frequency distribution feature;

[0077] In this embodiment, specifically, the formula for obtaining the feature sequence corresponding to each channel based on the modulus of the real part feature and the modulus of the imaginary part feature corresponding to each channel is:

[0078]

[0079] Among them, h k is the feature sequence corresponding to the kth channel. Take the feature sequence corresponding to each channel as the time dimension of each channel, and the number of channels in the time-frequency convolution layer as the width dimension of each channel. Combine the time dimension and width dimension of each channel to obtain the time-frequency distribution feature h, h = [h 1 , h 2 … h K, where K is the number of channels of the time-frequency convolutional layer, and h k,real is the real part feature corresponding to the k-th channel, and h k,imag is the imaginary part feature corresponding to the k-th channel, is the real part kernel of the time-frequency convolutional kernel corresponding to the k-th channel in the time-frequency convolutional layer, is the imaginary part kernel of the time-frequency convolutional kernel corresponding to the k-th channel in the time-frequency convolutional layer, x is the collected mechanical vibration signal, * is the convolution operation, and θ is the trainable control parameter of the kernel function.

[0080] Step S3: Input the time-frequency distribution feature into the pulse patching sub-module, and output a time-frequency feature group, including:

[0081] Step S31: According to the number T of time steps of the pulse residual network, evenly divide the feature length L of each channel in the time-frequency distribution feature h k in the time dimension into T equal parts, and the formula is h k,τ is the τ-th equal part of h k , where τ = 1, 2... T. Take the length W of each equal part as the feature width input for each time step, Take each equal part h k,τ and the frequency dimension H of this channel to form a time-frequency feature group with dimensions H×W, and obtain T time-frequency feature groups with dimensions H×W;

[0082] The present invention proposes a pulse time-frequency patching module. By embedding the time-frequency transformation method into the network, the one-dimensional vibration signal input completes the time-frequency transformation with a time dimension inside the network, improving the physical interpretability of the pulse neural network and achieving the end-to-end fault diagnosis goal; the proposed pulse patching step enables obtaining the time dimension without mechanically repeating the input as in the general training of pulse neural networks. Instead, the time part corresponding to the time-frequency feature is used as the input at the corresponding time step, which fully aligns the time-frequency feature and the time dimension of the pulse neural network, adapts to the time dynamic characteristics of the pulse neural network while reducing information repetition, thereby improving the processing efficiency.

[0083] Step S4: Input each time-frequency feature group into the pulse spatio-temporal attention module to extract the key time-frequency features of each time-frequency feature group;

[0084] In this embodiment, preferably, Step S41: Input each time-frequency feature group into the time attention unit to extract the time attention features of each time-frequency feature group, including:

[0085] Step S411: Pass each time-frequency feature group through the max pooling layer and the shared multi-layer perceptron in sequence to obtain the time-frequency max pooling perception feature of each time-frequency feature group;

[0086] Step S412: Pass each time-frequency feature group through an average pooling layer and a shared multi-layer perceptron in sequence to obtain the time-frequency average pooling perception features of each time-frequency feature group;

[0087] Step S413: After adding the time-frequency max pooling perception features and the time-frequency average pooling perception features of each time-frequency feature group, pass them through the Sigmoid activation function to obtain the time weights of each time-frequency feature group;

[0088] Step S414: Multiply the time weights of each time-frequency feature group element-wise with the time-frequency feature group to obtain the time attention features of each time-frequency feature group;

[0089] By calculating the time-frequency max pooling perception features and the time-frequency average pooling perception features separately, and adding them and then passing through the Sigmoid activation function to obtain the time weights, this method can adaptively assign weights to different time steps of each time-frequency feature group. Different time steps may have different importance for the final classification. The time attention unit can automatically learn these importance differences, highlight the feature information of the key time steps, and suppress the irrelevant or interfering time step information, thereby improving the model's feature representation ability in the time dimension.

[0090] After completing the feature processing in the time dimension, in order to comprehensively capture the important information of the time-frequency features, further analysis needs to be carried out from the spatial dimension next.

[0091] Step S42: Input the time attention features of each time-frequency feature group into the spatial attention unit to extract the spatial attention features of each time-frequency feature group, including:

[0092] Step S421: Pass the time attention features of each time-frequency feature group through a max pooling layer and a shared multi-layer perceptron in sequence to obtain the time attention max pooling perception features of each time-frequency feature group;

[0093] Step S422: Pass the time attention features of each time-frequency feature group through an average pooling layer and a shared multi-layer perceptron in sequence to obtain the time attention average pooling perception features of each time-frequency feature group;

[0094] Step S423: After concatenating the time attention max pooling perception features and the time attention average pooling perception features of each time-frequency feature group, pass them through a 7×7 convolution and the Sigmoid activation function in sequence to obtain the spatial weights of each time-frequency feature group;

[0095] Step S424: Multiply the spatial weights of each time-frequency feature group element-wise with the time attention features to obtain the spatial attention features of each time-frequency feature group;

[0096] Step S43: Pass the spatial attention features of each time-frequency feature group through the ReLU activation function to obtain the key time-frequency features of each time-frequency feature group.

[0097] Similarly, the time attention features are processed by combining max pooling and average pooling. Then, the obtained features are concatenated and passed through a convolution and the Sigmoid activation function to obtain the spatial weights. This operation helps the model accurately capture the important regions of the features in the spatial dimension. In time-frequency features, different spatial positions may contribute differently to the final classification result. The spatial attention unit can learn the importance of these spatial positions, and by assigning spatial weights, highlight the feature information in the important regions, improving the model's sensitivity to spatial features. In the process of obtaining the spatial weights, a 7×7 convolution and the Sigmoid activation function are used. The convolution operation can perform a non-linear transformation on the concatenated features to extract higher-level feature representations; the Sigmoid activation function maps the output value to the interval [0,1], enabling the spatial weights to intuitively represent the importance of each spatial position. This non-linear mapping method enhances the discriminability of the features, helping the model better distinguish different fault categories.

[0098] Step S5: Use the key time-frequency features of each time-frequency feature group as the input for each time step of the pulse residual network, and output the predicted fault class label.

[0099] Using the time attention unit and the spatial attention unit to process the time-frequency feature groups in stages can mine the information of the time-frequency features from different perspectives. The time attention unit focuses on the importance of each time step in the time dimension, and the spatial attention unit pays attention to the important regions of the features in the spatial dimension. This multi-stage and multi-dimensional feature extraction method can express the time-frequency features more comprehensively and deeply, improving the model's ability to capture data features.

[0100] In this embodiment, specifically, the pulse residual network includes: a downsampling layer, a first basic layer, a second basic layer, a third basic layer, a fourth basic layer, and a fully connected layer connected in sequence;

[0101] Among them, the downsampling layer includes: a convolution layer, a batch normalization layer, a pulse neuron, and a max pooling layer connected in sequence. Each basic layer is composed of two basic blocks with the same number of channels. Each basic block includes: a 3×3 convolution, a batch normalization layer, a pulse neuron, a 3×3 convolution, a batch normalization layer, and a pulse neuron connected in sequence.

[0102] In this embodiment, specifically, the pulse residual network is any one of SEW-ResNet18, MS-ResNet18, and SEW-ResNet50.

[0103] In this embodiment, specifically, the pulsed neuron in the pulsed residual network is any one of a LIF neuron, a PSN neuron, and a PLIF neuron.

[0104] In this embodiment, preferably, the dynamics formula of the PLIF neuron is:

[0105] H t =V t-1 +k(a)(-(V t-1 -V reset )+X t )

[0106] S t =Θ(H t -V th )

[0107] V t =H t (1 - S t )+V reset S t ,

[0108] where, H t is the membrane potential at time step t after neuron dynamics, V t is the membrane potential at time step t after pulse triggering, V t-1 is the membrane potential at time step t - 1 after pulse triggering, V reset is the reset membrane potential, X t is the input of the PLIF neuron, S t is the output pulse at time step t, Θ(.) is the Heaviside step function, V th is the membrane potential threshold, k(a) is the sigmoid activation function, exp(.) is the exponential function with the natural constant as the base, and a is a trainable parameter.

[0109] As Figure 3 shown, Figure 3 is the training schematic diagram of the fault diagnosis network. The training set is input into the fault diagnosis network to obtain the predicted fault class labels of each test sample, and the fault diagnosis network is trained using an optimization algorithm to obtain a trained fault diagnosis network;

[0110] As Figure 4 shown, Figure 4 is the test schematic diagram of the fault diagnosis network. The performance of the fault diagnosis network is evaluated through the test set to obtain the target fault diagnosis network;

[0111] In this embodiment, specifically, the training process of the fault diagnosis network is:

[0112] Obtain a mechanical fault dataset and preprocess it, and divide the mechanical fault dataset into a training set and a test set; wherein, the mechanical fault dataset includes multiple collected mechanical vibration signals and their true labels;

[0113] In this embodiment, specifically, preprocessing the mechanical fault dataset includes: intercepting multiple collected mechanical vibration signals into data samples, unifying the sample length, and normalizing the sample amplitude to the range of [0, 1].

[0114] Input the training set into the pulse time-frequency repair module to obtain T time-frequency feature groups corresponding to each training sample; input the T time-frequency feature groups corresponding to each training sample into the pulse spatio-temporal attention module respectively to focus on important features, and classify and train the extracted key time-frequency features through the pulse residual network. Optimize the model training by minimizing the mean square error between the predicted output class label and the actual label of the fault diagnosis network through an optimization algorithm.

[0115] The purpose of the optimized training is to enable the fault diagnosis network to classify the input samples into the correct fault class labels. The test set is used to verify the accuracy of the predicted class labels of the model, and the predicted sample labels correspond to the health status classes of each data sample.

[0116] In this embodiment, specifically, the optimization algorithm is any one of the root mean square propagation algorithm (RMSprop), the stochastic gradient descent method (SGD), and the adaptive moment estimation algorithm (Adam).

[0117] In this embodiment, specifically, the optimization algorithm adopts the stochastic gradient descent method (SGD), the learning rate is 0.01, the momentum is 0.9, and the loss function tends to balance after 64 iterations, completing the training of the fault diagnosis network.

[0118] Based on Embodiment 1, Embodiment 2 of the present invention takes a fault vibration dataset of a certain variable-condition gearbox as an example for intelligent fault diagnosis, including:

[0119] The experiment selects a dataset under a time-varying rotational speed condition, where the load is constant at 10 Nm and the rotational speed varies in the range of 0 - 1000 rpm. As shown in Table 1, Table 1 is a schematic table of the used variable-condition gearbox dataset.

[0120] Table 1

[0121]

[0122] The dataset contains fourteen health state categories, including normal state, tooth missing fault, and four single fault states with three different fault degrees. Specifically: the normal state is represented by normal state (N); single faults include tooth missing (MT) without fault degree distinction and gear pitting (P), tooth breakage (TB), gear crack (TC), and gear wear (W) with fault degree distinction. The fault degrees are divided into mild fault (L) of 0.1 mm, moderate fault (M) of 0.3 mm, and severe fault (H) of 0.5 mm. The sampling frequency of the data is 12.8 kHz. It should be noted that the dataset name is marked as health state + fault degree + variable working condition type. For example, P_H_10Nm_1000rpm represents gear pitting with severe fault, and its working condition belongs to the variable speed type, with a constant load of 10 Nm and a speed change range between 0 - 1000 rpm. In the experiment, 614400 points of each type of sample in this dataset are taken and divided into 600 samples according to 1024 points per sample. The training set and test set are divided in a ratio of 5:1, that is, 500 samples of each type are used for training and 100 samples are used for testing.

[0123] As Figure 5 shown, Figure 5 Figure is the t-SNE dimensionality reduction visualization graph of the classification results of the variable working condition gearbox fault vibration dataset. It can be Figure 5 seen that a mechanical fault diagnosis method based on spiking neural network proposed by the present invention can effectively cluster samples of the same category, and there is a relatively obvious boundary between the features of different category samples, proving that the method of the present invention can learn the deep features in the variable working condition gearbox dataset, improve the intra-class clustering and inter-class separability of fault categories, and achieve high-precision end-to-end fault diagnosis.

[0124] As Figure 6 shown, Figure 6 Figure is the confusion matrix graph of the classification results of the variable working condition gearbox fault vibration dataset. It can be Figure 6 seen that the diagnostic accuracy of a mechanical fault diagnosis method based on spiking neural network proposed by the present invention is very high, reaching 99.50%. Only seven classification errors occurred among 1400 test samples of fourteen health states, demonstrating the superior diagnostic performance and generalization ability of the method of the present invention.

[0125] In summary, the present invention establishes a spiking neural network mechanical fault diagnosis model embedded with time-frequency transformation. Through the spiking time-frequency repair module for internal time-frequency conversion of the network, and by the proposed spiking spatio-temporal attention mechanism to focus on important features in the spatio-temporal dimension to assist training, the model can efficiently extract the deep time-frequency features of the variable working condition gearbox fault dataset, and achieve high-precision and highly interpretable end-to-end fault diagnosis.

[0126] Embodiment 2 of the present invention also provides a mechanical fault diagnosis system based on a spiking neural network, including:

[0127] A model construction module for constructing a fault diagnosis network, where the fault diagnosis network includes: a spiking time-frequency repair module, a spiking spatio-temporal attention module, and a spiking residual network; among them, the spiking time-frequency repair module includes: a time-frequency convolutional layer, a spiking repair sub-module;

[0128] A time-frequency distribution feature extraction module for inputting the collected mechanical vibration signal into the time-frequency convolutional layer and outputting the time-frequency distribution feature;

[0129] A time-frequency feature group acquisition module for inputting the time-frequency distribution feature into the spiking repair sub-module and outputting T time-frequency feature groups, including:

[0130] According to the number T of time steps of the spiking residual network, the feature length of each channel in the time-frequency distribution feature in the time dimension is evenly divided into T equal parts, the length W of each equal part is used as the feature width input for each time step, and each equal part and the frequency dimension H of the channel form a time-frequency feature group with dimensions of H×W, obtaining T time-frequency feature groups with dimensions of H×W;

[0131] A key time-frequency feature extraction module for inputting each time-frequency feature group into the spiking spatio-temporal attention module and extracting the key time-frequency features of each time-frequency feature group;

[0132] A prediction module for using the key time-frequency features of each time-frequency feature group as the input for each time step of the spiking residual network and outputting a predicted fault class label.

[0133] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementation in the processFigure 1 one or more processes and / or blocks Figure 1 a device for the functions specified in one or more blocks

[0135] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions in a process Figure 1 one or more processes and / or blocks Figure 1 the functions specified in one or more blocks

[0136] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions in a process Figure 1 one or more processes and / or blocks Figure 1 the functions specified in one or more blocks

[0137] Obviously, the above embodiments are merely examples for clear illustration and are not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A mechanical fault diagnosis method based on pulse neural network, characterized in that: The following steps are involved: Constructing a fault diagnosis network, the fault diagnosis network includes: a pulse time-frequency patching module, a pulse time-space attention module and a pulse residual network; wherein the pulse time-frequency patching module includes: a time-frequency convolution layer and a pulse patching submodule; Input the collected mechanical vibration signal into the time-frequency convolution layer and output the time-frequency distribution features; The time-frequency distribution features are input into the pulse patching submodule, and T time-frequency feature groups are output, including: According to the number of time steps T of the pulse residual network, the feature length of each channel in the time-frequency distribution feature in the time dimension is evenly divided into T equal parts, and the length W of each equal part is used as the feature width of each time step input. Each equal part and the frequency dimension H of the channel form a time-frequency feature group with a dimension of H×W, and T time-frequency feature groups with a dimension of H×W are obtained; Each time-frequency feature group is input into the pulse spatiotemporal attention module to extract the key time-frequency features of each time-frequency feature group; The key time-frequency features of each time-frequency feature group are used as the input of the pulse residual network at each time step, and the predicted fault category label is output.

2. A mechanical fault diagnosis method based on pulse neural network according to claim 1, characterized in that: The collected mechanical vibration signal is input into the time-frequency convolution layer, and the time-frequency distribution features are output, including: The collected mechanical vibration signal is passed through the real part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolution layer, and convolved along the length direction to obtain the real part feature corresponding to each channel; The collected mechanical vibration signal is passed through the imaginary part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolution layer, and convolved along the length direction to obtain the imaginary part features corresponding to each channel; Based on the modulus of the real part features and the modulus of the imaginary part features corresponding to each channel, the feature sequence corresponding to each channel is obtained. The feature sequence corresponding to each channel is used as the time dimension of each channel. The number of channels of the time-frequency convolution layer is used as the wide dimension of each channel. The time dimension and wide dimension of each channel are combined as the feature representation of each channel of the time-frequency distribution feature to obtain the time-frequency distribution feature.

3. A mechanical fault diagnosis method based on pulse neural network according to claim 2, characterized in that: The collected mechanical vibration signal is passed through the real part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolution layer, and convolved along the length direction to obtain the real part feature corresponding to each channel. The formula is: The collected mechanical vibration signal is passed through the imaginary part kernel of the time-frequency convolution kernel corresponding to each channel in the time-frequency convolution layer, and convolved along the length direction to obtain the imaginary part feature corresponding to each channel. The formula is: The feature sequence corresponding to each channel is obtained based on the modulus of the real part feature and the modulus of the imaginary part feature corresponding to each channel, and the formula is: Among them, h k is the feature sequence corresponding to the kth channel, the feature sequence corresponding to each channel is taken as the time dimension of each channel, the number of channels of the time-frequency convolution layer is taken as the width dimension of each channel, and the time dimension and width dimension of each channel are combined to obtain the time-frequency distribution feature h, h = [h1,h2…h K ], K is the number of channels of the time-frequency convolution layer, h k,real is the true part feature corresponding to the kth channel, h k,imag is the imaginary part feature corresponding to the kth channel, is the real partial kernel of the time-frequency convolution kernel corresponding to the kth channel in the time-frequency convolution layer, is the imaginary part kernel of the time-frequency convolution kernel corresponding to the kth channel in the time-frequency convolution layer, x is the collected mechanical vibration signal, * is the convolution operation, and θ is the trainable control parameter of the kernel function.

4. The mechanical fault diagnosis method based on pulse neural network according to claim 1 is characterized in that: The pulse spatiotemporal attention module includes: a temporal attention unit, a spatial attention unit and a ReLU function connected in sequence; the temporal attention unit includes: a maximum pooling layer, an average pooling layer, a shared multi-layer perceptron and a Sigmoid activation function; the spatial attention unit includes: a maximum pooling layer, an average pooling layer, a 7×7 convolution and a Sigmoid activation function.

5. The mechanical fault diagnosis method based on pulse neural network according to claim 1 is characterized in that: The step of inputting each time-frequency feature group into the pulse spatiotemporal attention module and extracting the key time-frequency features of each time-frequency feature group includes: Each time-frequency feature group is input into the temporal attention unit, and the temporal attention features of each time-frequency feature group are extracted, including: Each time-frequency feature group is passed through the maximum pooling layer and the shared multi-layer perceptron in turn to obtain the time-frequency maximum pooling perceptual features of each time-frequency feature group; Each time-frequency feature group is passed through the average pooling layer and the shared multi-layer perceptron in turn to obtain the time-frequency average pooling perceptual features of each time-frequency feature group; After adding the time-frequency maximum pooling perception features and the time-frequency average pooling perception features of each time-frequency feature group, the time weight of each time-frequency feature group is obtained through the Sigmoid activation function; Multiply the time weight of each time-frequency feature group by the time-frequency feature group element by element to obtain the time attention feature of each time-frequency feature group; The temporal attention features of each time-frequency feature group are input into the spatial attention unit, and the spatial attention features of each time-frequency feature group are extracted, including: The temporal attention features of each time-frequency feature group are sequentially passed through the maximum pooling layer and the shared multi-layer perceptron to obtain the temporal attention maximum pooling perceptual features of each time-frequency feature group; The temporal attention features of each time-frequency feature group are sequentially passed through the average pooling layer and the shared multi-layer perceptron to obtain the temporal attention average pooling perceptual features of each time-frequency feature group; After concatenating the temporal attention maximum pooling perception features and the temporal attention average pooling perception features of each time-frequency feature group, they are sequentially passed through 7×7 convolution and Sigmoid activation function to obtain the spatial weight of each time-frequency feature group; Multiply the spatial weight of each time-frequency feature group by the temporal attention feature element by element to obtain the spatial attention feature of each time-frequency feature group; The spatial attention features of each time-frequency feature group are passed through the ReLU activation function to obtain the key time-frequency features of each time-frequency feature group.

6. The mechanical fault diagnosis method based on pulse neural network according to claim 1 is characterized in that: The pulse residual network includes: a downsampling layer, a first basic layer, a second basic layer, a third basic layer, a fourth basic layer and a fully connected layer connected in sequence; Among them, the downsampling layer includes: a convolutional layer, a batch normalization layer, a pulse neuron, and a maximum pooling layer connected in sequence. Each basic layer is composed of two basic blocks with the same number of channels. Each basic block includes: a 3×3 convolution, a batch normalization layer, a pulse neuron, a 3×3 convolution, a batch normalization layer and a pulse neuron connected in sequence.

7. A mechanical fault diagnosis method based on pulse neural network according to claim 6, characterized in that: The pulse residual network is any one of SEW-ResNet18, MS-ResNet18, and SEW-ResNet50.

8. The mechanical fault diagnosis method based on pulse neural network according to claim 6 is characterized in that: The pulse neuron in the pulse residual network is any one of a LIF neuron, a PSN neuron and a PLIF neuron.

9. A mechanical fault diagnosis method based on pulse neural network according to claim 8, characterized in that: The dynamics of the PLIF neuron are: H t =V t-1 +k(a)(-(V t-1 -V reset )+X t ) S t =Θ(H t -V th ) V t =H t (1-S t )+V reset S t , Among them, H t is the membrane potential at time step t after neuronal dynamics, V t is the membrane potential at time step t after the pulse is triggered, V t-1 is the membrane potential at time step t-1 after the pulse is triggered, V reset To reset the membrane potential, X t is the input of PLIF neurons, S t is the output pulse at time step t, Θ(.) is the Heaviside step function, V th is the membrane potential threshold, k(a) is the sigmoid activation function, exp(.) is an exponential function with a natural constant as the base, and a is a trainable parameter.

10. A mechanical fault diagnosis system based on pulse neural network, characterized in that: include: A model building module is used to build a fault diagnosis network, wherein the fault diagnosis network includes: a pulse time-frequency patching module, a pulse time-space attention module and a pulse residual network; wherein the pulse time-frequency patching module includes: a time-frequency convolution layer and a pulse patching submodule; The time-frequency distribution feature extraction module is used to input the collected mechanical vibration signal into the time-frequency convolution layer and output the time-frequency distribution feature; The time-frequency feature group acquisition module is used to input the time-frequency distribution features into the pulse repair submodule and output T time-frequency feature groups, including: According to the number of time steps T of the pulse residual network, the feature length of each channel in the time-frequency distribution feature in the time dimension is evenly divided into T equal parts, and the length W of each equal part is used as the feature width of each time step input. Each equal part and the frequency dimension H of the channel form a time-frequency feature group with a dimension of H×W, and T time-frequency feature groups with a dimension of H×W are obtained; A key time-frequency feature extraction module is used to input each time-frequency feature group into the pulse spatiotemporal attention module to extract the key time-frequency features of each time-frequency feature group; The prediction module is used to take the key time-frequency features of each time-frequency feature group as the input of the pulse residual network at each time step and output the predicted fault category label.

Images