Method and apparatus for fault diagnosis

Abstract

The specification provides a fault diagnosis method and device. The method comprises: acquiring a plurality of sensor signals collected by a plurality of sensors, the sensor signals representing the running state of an autonomous unmanned system; inputting the plurality of sensor signals into a convolutional coding layer in a fault diagnosis model, encoding the plurality of sensor signals into a pulse space-time sequence by the convolutional coding layer according to the collection time of the sensor signals, extracting sequence features from the pulse space-time sequence by a first convolutional layer in a feature extraction layer, calculating attention weights corresponding to the sequence features by a self-attention layer connected to the first convolutional layer, further extracting prediction features related to fault detection from the sequence features by a second convolutional layer in an output layer based on the attention weights corresponding to the sequence features, and performing fault detection on the autonomous unmanned system based on the prediction features by a fully connected layer.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computers, and in particular to a fault diagnosis method and apparatus. Background Technology

[0002] With the continuous development of artificial intelligence technology, autonomous unmanned systems developed based on artificial intelligence technology are expanding their application scope due to their significant autonomy and adaptability, such as drones, autonomous vehicles, deep space probes, and autonomous underwater vehicles.

[0003] Taking autonomous underwater vehicles (AUVs) as an example, due to their autonomous operation capabilities in deep-sea environments and their long-distance and high-precision operational performance, they have been widely adopted in marine applications, playing an important role in areas such as offshore oil and gas exploration, maritime rescue operations, and marine environmental surveys. However, due to the complex and unpredictable environment, the use of AUVs also carries certain risks, mainly concentrated in mission failures caused by equipment or component malfunctions, and may even lead to the loss of the AUV.

[0004] Therefore, a solution is needed that can autonomously diagnose faults to ensure the safety and reliability of such autonomous unmanned systems. Summary of the Invention

[0005] This specification provides a fault diagnosis method and apparatus in its embodiments.

[0006] According to a first aspect of the embodiments of this specification, a fault diagnosis method is provided, applied to an autonomous unmanned system, the autonomous unmanned system being equipped with multiple sensors and a fault diagnosis model constructed using a spiking neural network based on a self-attention mechanism; the spiking neural network includes a convolutional coding layer, a feature extraction layer, and an output layer, the feature extraction layer including a first type of convolutional layer for extracting features and a self-attention layer connected to the first type of convolutional layer, the output layer including a second type of convolutional layer and a fully connected layer connected to the second type of convolutional layer; the method includes:

[0007] Acquire multi-sensor signals characterizing the operating state of the autonomous unmanned system collected by the multiple sensors; wherein the signals carry the acquisition time.

[0008] The multi-sensor signals are input into the convolutional coding layer of the fault diagnosis model. The convolutional coding layer encodes the multi-sensor signals into a pulse spatiotemporal sequence according to the acquisition time of the sensor signals, and outputs the pulse spatiotemporal sequence to the feature extraction layer. The first type of convolutional layer in the feature extraction layer extracts sequence features from the pulse spatiotemporal sequence, and the self-attention layer connected to the first type of convolutional layer calculates the attention weights corresponding to the sequence features. The sequence features and the attention weights corresponding to the sequence features are further output to the output layer. The second type of convolutional layer in the output layer further extracts predictive features related to fault detection from the sequence features based on the attention weights corresponding to the sequence features, and outputs the predictive features to the fully connected layer. The fully connected layer performs fault detection for the autonomous unmanned system based on the predictive features.

[0009] Optionally, the feature extraction layer includes at least two sets of first-type convolutional layers stacked in a serial order for feature extraction, and a self-attention layer connected to the first-type convolutional layers.

[0010] Optionally, both the first type of convolutional layer and the second type of convolutional layer include a neuron model for simulating neurons in the human brain, the neuron model including:

[0011]

[0012] Where, τ m V represents the membrane time constant, and V(t) represents the membrane potential of the neuron. rest R represents the resting potential of a neuron. m Let I(t) represent the leakage resistance of the neuron, and let I(t) represent the input current of the neuron.

[0013] Optionally, the first and second convolutional layers further include an internal recurrent feature learning algorithm to make them recurrent convolutional layers; wherein the recurrent feature learning algorithm includes:

[0014]

[0015] in, The recurrent convolutional layer represents the potential change of the neuron simulated by the recurrent convolutional layer. S[t] represents the input feature of the recurrent convolutional layer at time t, used to simulate the input potential of the neuron at time t; V[t-1] represents the modulus potential of the neuron at time t-1; H[t] represents the membrane potential of the neuron at time t; 1 / τ m This represents the update gate used to preserve important features in S[t], 1-1 / τ mThis represents a forget gate used to discard unimportant features in V[t-1]; the important features include features that are highly relevant to fault detection, and the unimportant features include features that are low in relation to fault detection.

[0016] Optionally, the neuron will fire when its membrane potential exceeds the firing threshold, and the mode potential will be reset.

[0017] The discharge includes iteratively processing the important features in S[t] retained by the update gate as new input potentials to the neuron that has reset the membrane potential.

[0018] Optionally, the step of calculating the attention weights corresponding to the sequence features by the self-attention layer connected to the convolutional layer includes:

[0019] Each type of feature in the sequence features is encoded into a first positional code and a second positional code according to the time step by a self-attention layer connected to the convolutional layer.

[0020] Matrix multiplication is performed on the first positional code and the second positional code at each time step to obtain an attention score representing the similarity between the first positional code and the second positional code;

[0021] The attention scores are normalized to obtain the attention weights corresponding to the sequence features.

[0022] Optionally, the fault diagnosis model is trained using a gradient substitution algorithm to optimize the model parameters through gradient backpropagation.

[0023] Optionally, the gradient substitution algorithm includes a Gaussian error function;

[0024] The optimization of model parameters using the gradient substitution algorithm with gradient backpropagation includes:

[0025] The gradient generated by the Gaussian error function replaces the gradient required for the transition function in the gradient backpropagation, and the replaced gradient is used to optimize the model parameters through backpropagation.

[0026] Optionally, the loss function of the fault diagnosis model includes:

[0027]

[0028] Where N represents the number of training samples, The coded value representing the fault type, This represents the predicted feature of the nth training sample.

[0029] Optionally, the autonomous unmanned system includes an autonomous underwater vehicle.

[0030] According to a second aspect of the embodiments of this specification, a fault diagnosis device is provided, applied to an autonomous unmanned system. The autonomous unmanned system is equipped with multiple sensors and a fault diagnosis model constructed using a spiking neural network based on a self-attention mechanism. The spiking neural network includes a convolutional coding layer, a feature extraction layer, and an output layer. The feature extraction layer includes a first type of convolutional layer for extracting features and a self-attention layer connected to the first type of convolutional layer. The output layer includes a second type of convolutional layer and a fully connected layer connected to the second type of convolutional layer. The device includes:

[0031] The acquisition unit acquires multi-sensor signals representing the operating state of the autonomous unmanned system collected by the multiple sensors; wherein the signals carry the acquisition time.

[0032] The diagnostic unit inputs the multi-sensor signals into the convolutional coding layer of the fault diagnosis model. The convolutional coding layer encodes the multi-sensor signals into a pulse spatiotemporal sequence according to the acquisition time of the sensor signals, and outputs the pulse spatiotemporal sequence to the feature extraction layer. The first type of convolutional layer in the feature extraction layer extracts sequence features from the pulse spatiotemporal sequence, and the self-attention layer connected to the first type of convolutional layer calculates the attention weights corresponding to the sequence features. The sequence features and the attention weights corresponding to the sequence features are further output to the output layer. The second type of convolutional layer in the output layer further extracts predictive features related to fault detection from the sequence features based on the attention weights corresponding to the sequence features, and outputs the predictive features to the fully connected layer. The fully connected layer performs fault detection for the autonomous unmanned system based on the predictive features.

[0033] According to a third aspect of the embodiments of this specification, an autonomous underwater vehicle is provided, wherein an autonomous unmanned system is deployed on the autonomous underwater vehicle, and the autonomous unmanned system implements the method described in any of the above.

[0034] This specification provides a fault diagnosis scheme through its embodiments. A self-attention encoding mechanism (connecting a self-attention layer to the first type of convolutional layer used for feature extraction) is introduced into a fault diagnosis model built on a spiking neural network, thereby constructing a fault diagnosis model based on the self-attention mechanism. In this way, the attention weights for the sequence features extracted by the first type of convolutional layer are calculated through the self-attention encoding layer, allowing the second type of convolutional layer in the output layer to extract important features, i.e., features relevant to fault diagnosis, from the sequence features based on these attention weights. On the one hand, since the second type of convolutional layer ignores unimportant features, the accuracy of the diagnostic results can be avoided due to unimportant features, thus improving the accuracy of the diagnostic results; on the other hand, reducing unimportant features can reduce the power consumption of the fully connected layers, further reducing the power consumption of fault detection. Attached Figure Description

[0035] Figure 1 This is a flowchart of a fault diagnosis method provided in one embodiment of this specification;

[0036] Figure 2 This is a schematic diagram of the architecture of a spiking neural network provided in one embodiment of this specification;

[0037] Figure 3 A schematic diagram of a fault diagnosis model constructed based on a self-attention mechanism spiking neural network, provided as an embodiment of this specification;

[0038] Figure 4 A schematic diagram of a learnable recursive feature learning mechanism provided in one embodiment of this specification;

[0039] Figure 5 A schematic diagram of a self-attention layer provided in one embodiment of this specification;

[0040] Figure 6 A schematic diagram illustrating the components and common faults of an autonomous underwater vehicle provided in one embodiment of this specification;

[0041] Figure 7 This is a hardware structure diagram of a fault diagnosis device provided in one embodiment of this specification;

[0042] Figure 8 This is a module of the fault diagnosis device provided in one embodiment of this specification. Detailed Implementation

[0043] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0044] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0045] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0046] With the continuous development of artificial intelligence technology, autonomous unmanned systems (also known as intelligent unmanned systems) developed based on artificial intelligence technology are expanding their application scope due to their significant autonomy and adaptability. Examples include drones, autonomous vehicles, deep space probes, and autonomous underwater vehicles (AUVs).

[0047] Taking autonomous underwater vehicles as an example, due to their autonomous operation capabilities in the deep-sea environment and their long-distance and high-precision operation performance, they have been widely used in marine applications, such as playing an important role in offshore oil and gas exploration, maritime rescue operations, and marine environmental surveys.

[0048] However, due to the complex and unpredictable environment, the use of autonomous underwater vehicles also carries certain risks, mainly focusing on mission failures caused by equipment or component malfunctions, which may even lead to the loss of the autonomous underwater vehicle.

[0049] Therefore, a solution capable of autonomous fault diagnosis is needed, which is particularly important for ensuring the safety and reliability of such autonomous unmanned systems.

[0050] In related technologies, the development of fault diagnosis technology for autonomous unmanned systems has gone through the following stages:

[0051] The first stage involves data-driven fault diagnosis technology. With advancements in sensor and operational monitoring technologies, a vast amount of operational data from autonomous unmanned systems has been accumulated. The approach of using data-driven fault diagnosis algorithms to extract useful data features from these accumulated historical datasets and establishing a nonlinear mapping from data characteristics to fault types has gradually matured.

[0052] However, data-driven algorithms primarily rely on experienced experts manually selecting meaningful features from collected historical datasets, and then using these features to perform fault diagnosis using machine learning-based classifiers. However, the number of manually selected features is limited, and the presence of subjective human influence results in limited non-linear representation capabilities, making it difficult to accurately diagnose faults in autonomous unmanned systems.

[0053] In the second stage, with the advancements in Artificial Neural Networks (ANNs) across various fields, applying ANNs to fault diagnosis in autonomous unmanned systems became possible. Among the various variants of ANNs, Convolutional Neural Networks (CNNs) have gradually become the mainstream solution for fault diagnosis in autonomous unmanned systems due to their excellent performance in image analysis and signal processing. However, although ANNs are inspired by the human brain, they lack the dynamic mechanisms within neurons in the human brain and cannot accurately simulate the complex neural operations of the human brain. This limitation results in existing ANN models having poor interpretability, limited generalization performance, and high resource consumption.

[0054] In the third stage, given the limitations of artificial neural networks, researchers constructed a neural network model with stronger cognitive abilities by deeply studying the characteristics of the human brain, namely Spiking Neural Networks (SNNs). SNNs mimic the information transmission mechanism and dynamic characteristics of neurons in the brain, possessing unique spatiotemporal dynamics, diverse encoders, and advantages in temporal processing, and are hailed as the third generation of neural network models.

[0055] However, there are still some technical challenges to overcome in the research and application of SNNs, especially in fault identification of autonomous unmanned systems. For example, autonomous unmanned systems are highly sensitive to energy consumption due to the need for long-term autonomous operation. Although SNNs themselves are characterized by low energy consumption, they still cannot meet the needs of such energy-sensitive autonomous unmanned systems in fault diagnosis. That is, how to further reduce energy consumption without compromising fault diagnosis performance.

[0056] Therefore, this invention proposes a novel fault diagnosis scheme. By introducing a self-attention encoding mechanism into a fault diagnosis model built on a spiking neural network, a self-attention layer is connected to the first type of convolutional layer used for feature extraction in the spiking neural network. This allows the construction of a fault diagnosis model based on a spiking neural network with a self-attention mechanism. The self-attention encoding layer calculates the attention weights for the sequence features extracted by the first type of convolutional layer, allowing the second type of convolutional layer in the output layer to extract important features (i.e., predictive features related to fault detection) from the sequence features based on these attention weights, while ignoring unimportant features. On the one hand, this avoids reducing the accuracy of the diagnostic results due to unimportant features (i.e., the implementation can improve the accuracy of fault diagnosis); on the other hand, reducing unimportant features can reduce the power consumption of the fully connected layer, further reducing the power consumption of fault detection.

[0057] Please refer to the following: Figure 1 The diagram illustrates a fault diagnosis method that can be applied to autonomous unmanned systems. The autonomous unmanned system is equipped with multiple sensors and a pre-trained fault diagnosis model constructed using a self-attention-based spiking neural network (SRA-SNN). The self-attention-based spiking neural network comprises a convolutional coding layer, a feature extraction layer, and an output layer. The feature extraction layer includes a first type of convolutional layer for feature extraction and a self-attention layer connected to the first type of convolutional layer. The output layer includes a second type of convolutional layer and a fully connected layer connected to the second type of convolutional layer. The method may include the following steps:

[0058] Step 110: Acquire multi-sensor signals representing the operating state of the autonomous unmanned system collected by the multiple sensors; wherein the signals carry the acquisition time.

[0059] Multiple sensors are installed on the autonomous unmanned system. These multiple sensors can collect multi-sensor signals from the autonomous unmanned system during operation, and then use the collected multi-sensor signals to detect faults in the autonomous unmanned system.

[0060] The multi-sensor signals collected in this embodiment may include, but are not limited to: acceleration along the XYZ axes, angular velocity along the XYZ axes, latitude and longitude of the autonomous unmanned system, voltage and current of the power equipment, etc. How to use these sensor signals will be described in subsequent embodiments.

[0061] Since sensors continuously or periodically acquire signals, the sensor signal acquired by a single sensor can be considered a data stream; and this data stream can be represented as a time-series vector. in, This could be the number of data points in the data stream represented by the time-series vector Z. Assuming there are y sensors, the data stream consisting of the signals from y sensors can be represented as...

[0062] Furthermore, since sensor signals acquired by different sensors can have different dimensions, preprocessing can be performed on the multi-sensor signals to obtain standardized sensor signals in order to eliminate the influence of dimensions. For example, the Z-score normalization algorithm can be used to normalize multi-sensor signals.

[0063] Step 120: The multi-sensor signals are input into the convolutional coding layer of the fault diagnosis model. The convolutional coding layer encodes the multi-sensor signals into a pulse spatiotemporal sequence according to the acquisition time of the sensor signals, and outputs the pulse spatiotemporal sequence to the feature extraction layer. The first type of convolutional layer in the feature extraction layer extracts sequence features from the pulse spatiotemporal sequence, and the self-attention layer connected to the first type of convolutional layer calculates the attention weights corresponding to the sequence features. The sequence features and the attention weights corresponding to the sequence features are further output to the output layer. The second type of convolutional layer in the output layer further extracts predictive features related to fault detection from the sequence features based on the attention weights corresponding to the sequence features, and outputs the predictive features to the fully connected layer. The fully connected layer performs fault detection for the autonomous unmanned system based on the predictive features.

[0064] The fault diagnosis model based on the self-attention mechanism of the spiking neural network mentioned in this specification can refer to a fault diagnosis model that incorporates the self-attention mechanism into a spiking neural network (SNN) model.

[0065] To better understand the fault diagnosis model based on a self-attention mechanism spiking neural network provided in this invention, the spiking neural network will be introduced first.

[0066] Please refer to Figure 2 The diagram shows a spiking neural network. Figure 2 As shown in the upper part, the spiking neural network model can simulate the working mechanism of neurons in the human brain. A typical neuron consists of dendrites, axons, and a cell body. Dendrites receive pulse signals from pre-neurons and transmit them to the cell body. When the neuron's membrane potential exceeds the firing threshold, the neuron generates an output pulse signal, which propagates through the axon to post-neurons. The connection between two neurons is called a synapse. The goal of the spiking neural network model is to leverage neuroscience and machine learning techniques to achieve efficient intelligent computing with biologically reliable mechanisms.

[0067] This manual is based on the idea of ​​a spiking neural network model, introducing the concept of convolution into the spiking neural network model, and using the ability of convolution to process high-dimensional data to simulate neurons in the human brain. It is worth mentioning that the first type of convolutional layer and the second type of convolutional layer mentioned above are named as such for easy distinction because of their different locations, but their convolutional layers are the same.

[0068] In an exemplary embodiment, both the first type of convolutional layer and the second type of convolutional layer include a neuron model for simulating neurons in the human brain, the neuron model including:

[0069]

[0070] Where, τ m V represents the membrane time constant, and V(t) represents the membrane potential of the neuron. rest R represents the resting potential of a neuron. m Let I(t) represent the leakage resistance of the neuron, and let I(t) represent the input current of the neuron.

[0071] Furthermore, this neuron also possesses a pulse firing mechanism, specifically meaning that the neuron will fire and the membrane potential will be reset when the membrane potential exceeds the firing threshold.

[0072]

[0073] Where θ represents the discharge threshold, V r This indicates a reset potential.

[0074] like Figure 2 The lower half of the diagram shows a schematic of the neuron model, where the post-neuron receives pulse signals from n pre-neurons, along with the weights w1, w2, ..., wn of these n pulse signals. Correspondingly, the neurodynamic mechanism of the neuron model in the convolutional layer can be represented as:

[0075]

[0076] O(t f )=1,V(t)=V r if V(t)≥θ

[0077] O(t) = 0, if V(t) ≤ θ

[0078] Among them, S i O(t) represents the action potential output by the i-th preneuron, and O(t) represents the action potential output by the postneuron.

[0079] Unlike artificial neural network models, spiking neural network models use pulse signals as the medium for information transmission, and each neuron can exhibit different dynamic properties. Because spiking neural network models possess the aforementioned pulse emission mechanism and the dynamic change of membrane potential based on the discharge threshold, they are endowed with good spatiotemporal representation capabilities while maintaining biological realism.

[0080] The following section further introduces the spiking neural network model based on the self-attention mechanism, which is an improvement upon the spiking neural network model provided in this specification.

[0081] like Figure 3 The schematic diagram shown is of a spiking neural network model based on a self-attention mechanism. It can be composed of a convolutional coding layer, a feature extraction layer, and an output layer. The feature extraction layer includes a first type of convolutional layer for extracting features and a self-attention layer connected to the first type of convolutional layer. The output layer includes a second type of convolutional layer and a fully connected layer connected to the second type of convolutional layer.

[0082] In this design, a first-type convolutional layer and a self-attention layer connected to it can be considered as a group. To obtain more accurate predicted features, the feature extraction layer can stack at least two groups of first-type convolutional layers for feature extraction, along with self-attention layers connected to the first-type convolutional layers, in a sequential order. More precise attention weights can be calculated using the features from these at least two groups, thereby improving the accuracy of the predicted features extracted by the second-type convolutional layer in the output layer.

[0083] This invention introduces the concept of convolution into the spiking neural network model to achieve synaptic weight sharing and hierarchical spiking representation learning. Assume that at time t, the input feature map is... If its length is l, then U t (v) represents the output of the preneuron located at position v-th on the feature map.

[0084] The size of the neuron synaptic weights is set to 2k+1 to construct the convolution kernel W(i), -k≤i≤k. Assume the feature map U... t If we perform a convolution with kernel W(i) centered at position v, then we can obtain the output O of the convolutional layer at position v. t (v):

[0085]

[0086] Taking the above neuron model as an example, it can be expressed as:

[0087]

[0088] O t (l, h) = 1, V(t) = Vr if V(t)≥θ

[0089] O t (l, h) = 0, if V(t) ≤ θ

[0090] When the postneuron at position v receives an output pulse signal from the preneuron at position 2k+1, it will output a pulse signal if the membrane potential of the postneuron reaches a preset firing threshold; otherwise, it will remain silent.

[0091] Based on the aforementioned neuron model, this specification introduces a learnable recursive feature learning mechanism; by setting a recurrent feature learning algorithm inside the first type of convolutional layer and the second type of convolutional layer, the first type of convolutional layer and the second type of convolutional layer become recurrent convolutional layers.

[0092] Such recurrent convolutional layers can not only achieve spatial feature learning similar to convolutional neural networks through convolutional topology, but also have... Figure 4 The learnable recursive feature learning mechanism shown allows the neurodynamic mechanism of the recurrent convolutional layer to be discretized into the following recursive formula:

[0093] First formula:

[0094] The second formula:

[0095] The third formula:

[0096] The fourth formula: V[t] = H[t](1-O[t]) + V rest O[t].

[0097] The second formula describes the changing state of neuronal potential. The functions represent the potential changes of neurons simulated by the recurrent convolutional layer. S[t] represents the input feature of the recurrent convolutional layer at time t, used to simulate the input potential of the neuron at time t; V[t-1] represents the modulus potential of the neuron at time t-1; and H[t] represents the membrane potential of the neuron at time t. Θ(·) is the Heaviside step function. 9 represents the synaptic threshold. O[t] represents the action potential output at time t.

[0098] The third formula describes the firing process of neurons, and the fourth formula illustrates the resetting process of neurons. rest It is usually set to 0.

[0099] The second formula above can be transformed into a recurrent feature learning algorithm:

[0100]

[0101] 1 / τ m This represents the update gate used to preserve important features in S[t], 1-1 / τ m This represents a forget gate used to discard unimportant features in V[t-1]; the important features include features that are highly relevant to fault detection, and the unimportant features include features that are low in relation to fault detection.

[0102] As mentioned earlier, when the membrane potential of a neuron exceeds the firing threshold, it will fire and reset the modulus potential. In conjunction with this cyclic feature learning algorithm, it can be seen that the firing may include passing important features in S[t] retained by the update gate as new input potentials to the neuron that has reset the membrane potential for iterative processing.

[0103] After introducing the first type of convolutional layer and the second type of convolutional layer (which can also be a recurrent convolutional layer), we will now introduce the self-attention layer (also known as a self-attention module) that works in conjunction with them.

[0104] like Figure 5 The diagram shown illustrates a self-attention layer. If the input to the self-attention layer is... s l [i] represents the feature of layer 1 at time step i, where C and L are the number of channels and the feature length, respectively.

[0105] For each type of feature in the feature sequence, it is encoded into a first positional code Q according to the time step. i Second position code K i and the third position code V i ;

[0106] Q i =W Q *S l [i]+b Q , 1≤i≤N

[0107] K i =W K *S i [i]+b K , 1≤i≤N

[0108] V i =W V *S l [i]+b V , 1≤i≤N

[0109] Then, matrix multiplication is performed on the first positional code and the second positional code at each time step to obtain an attention score representing the similarity between the first positional code and the second positional code.

[0110] For example, the obtained attention scores can be normalized using a classifier (such as Softmax) to obtain the attention weights corresponding to the sequence features.

[0111]

[0112] Furthermore, attention weights can be... and the third position encoding V i Matrix multiplication is performed to obtain a feature sequence optimized by attention weights, which helps the second type of convolutional layer in the output layer to further extract predictive features related to fault detection from this optimized feature sequence.

[0113] After introducing the architecture of the fault diagnosis model, the training of the fault diagnosis model will be described below:

[0114] Collect training samples for model training; the training samples include multi-sensor signals labeled with fault detection results;

[0115] The fault diagnosis model can be trained in a supervised manner using the training samples, and the model parameters can be optimized by using a gradient substitution algorithm with gradient backpropagation.

[0116] Because the transition function Θ(x) used in traditional backpropagation is nondifferentiable, it is difficult to perform gradient propagation and model parameter optimization in spiking neural network models.

[0117] To this end, this patent employs a gate function σ(x) that is similar to but differentiable than the transition function Θ(x) for backpropagation. Specifically, during forward propagation, Θ(x) is used, and neurons transmit information via a spatiotemporal sequence of pulses and membrane dynamics; while during backpropagation, the gradient of the transition function Θ(x) is replaced by the gradient of the gradient substitution algorithm, Θ′(x)≈σ′(x), to achieve gradient propagation and parameter optimization adjustment.

[0118] In one exemplary embodiment, the gradient substitution algorithm may include a Gaussian error function;

[0119] The optimization of model parameters using the gradient substitution algorithm with gradient backpropagation includes:

[0120] The gradient generated by the Gaussian error function replaces the gradient required for the transition function in the gradient backpropagation, and the replaced gradient is used to optimize the model parameters through backpropagation.

[0121] For example, the above replacement algorithm can be expressed as:

[0122]

[0123] Through the above embodiments, the gradient substitution algorithm can solve the problem that gradient propagation and model parameter optimization are difficult to perform in spiking neural network models due to the non-differentiability of transition functions.

[0124] Assume the number of fault types in the autonomous unmanned system is Then the number of neurons in the output layer is also... The output layer with T time steps is accumulated. The output pulses of each neuron are used to obtain the total number of impulse releases for each neuron. Then, the total number of impulse releases is divided by the time step to obtain the impulse release frequency.

[0125]

[0126] Ideally, when the category of the input data is i, the i-th neuron in the output layer of a spiking neural network model based on the self-attention mechanism should have the maximum activation level, while other neurons remain inactive.

[0127] Therefore, the loss function can use the cross-entropy function to measure the distance between the fault type output by the fully connected layer in the output layer and the actual labeled fault type, which can be expressed as:

[0128]

[0129] Where N represents the number of training data, The coded value representing the fault type, This represents the predicted feature of the nth training sample. This encoded value can be obtained using ne-hot encoding. It's worth noting that, since the spiking neural network model is a memory model, the model's state needs to be reset when new data is input.

[0130] After introducing the training of the fault diagnosis model, let's revisit... Figure 1 The illustrated embodiment of the fault diagnosis method, combined with Figure 3 The lower half of the diagram shows a schematic of the model architecture of a self-attention-based spiking neural network (SRA-SNN). Figure 3 As shown, the SRA-SNN model can consist of a convolutional coding layer, a feature extraction layer, and an output layer. The feature extraction layer includes a first type of convolutional layer for feature extraction and a self-attention layer connected thereto. The output layer includes a second type of convolutional layer and a fully connected layer connected thereto.

[0131] When an autonomous unmanned system acquires signals from multiple sensors, these signals can be input into an SRA-SNN model. First, a convolutional coding layer encodes the input multi-sensory sensor signals into a pulse spatiotemporal sequence. Then, a first type of convolutional layer in the feature extraction layer extracts sequence features from the pulse spatiotemporal sequence, and a self-attention layer connected to the first type of convolutional layer calculates the attention weights corresponding to the sequence features. Next, a second type of convolutional layer in the output layer further extracts predictive features related to fault detection from the sequence features based on the attention weights corresponding to the sequence features. Finally, the fully connected layer acts as a classifier to accurately map the predictive features to the corresponding fault types.

[0132] The following is based on Figure 6 The diagram shown is an example of an autonomous underwater vehicle. Figure 6 As shown, this autonomous underwater vehicle belongs to a small autonomous unmanned system that is sensitive to energy consumption. The main body of the autonomous underwater vehicle is the control cabin and four engines (each engine can be composed of a motor and a propeller) deployed around the control cabin. In addition, the autonomous underwater vehicle is also equipped with multiple sensors (the depth sensor shown in the figure, satellite positioning equipment, inertial measurement unit, etc. are not shown in the figure).

[0133] Multiple sensors can be used to collect multi-sensor signals. Please refer to Table 1 for examples of 16 different sensor data, such as motor control signals, battery voltage, depth, angular velocity, acceleration, and attitude information.

[0134] Table 1

[0135]

[0136] like Figure 6 As shown, there are five types of faults that can be identified in autonomous underwater vehicles: normal state (F1), abnormal load (F2), depth sensor fault (F3), serious propeller fault (F4), and minor propeller fault (F5).

[0137] During training sample collection, the AUV was subjected to each fault type and run multiple times to acquire multi-sensor signals for each fault type. Each run lasted approximately 10 to 20 seconds to obtain sufficient diverse sensor data. Ultimately, approximately 250 training samples were obtained for each fault type, for a total of 1220 training samples.

[0138] To test the model's performance, the 1220 training samples were randomly divided into a training set and a test set at a 4:1 ratio, with the training set containing 975 training samples and the test set containing 245 training samples. Subsequently, each training sample was Z-score standardized in the time direction to remove the influence of dimensions. At this point, the training sample collection and preprocessing process was completed.

[0139] The following comparison between the self-attention-based spiking neural network model SRA-SNN and the artificial neural network model ANN demonstrates that the SRA-SNN provided in this invention has significant fault detection performance. The evaluation metrics are: accuracy (ACC), F1-score, and AUC.

[0140] In the comparative experiments, ten ANN-based fault diagnosis models (hereinafter referred to as experimental models) were implemented on the AUV dataset, namely Multi-ANN, Wen-CNN, Sheng-CNN, AD-CNN, Multi-GRU, Multi-LSTM, Multi-RNN, MFCNN, Gu-1DCNN and Conv-Trans.

[0141] Among them, Multi-ANN consists of multiple fully connected neural architectures. Wen-CNN, Sheng-CNN, and AD-CNN use 2D convolutional layers to extract valuable features and FC-Layers for fault diagnosis. Multi-GRU, Multi-LSTM, and Multi-RNN use different recurrent neural units to learn temporal characteristics and FC-Layers for fault diagnosis. MFCNN is a convolutional network model for multi-fault classification of rolling bearings, using average summation instead of maximum summation. Gu-1DCNN is a convolutional model for multi-sensor information fusion, achieving excellent motor fault diagnosis performance. Finally, following the ViT architecture approach, CNN and Transformer are combined to construct Conv-Trans.

[0142] The comparative experiment was divided into three parts: (1) no noise was added; (2) Gaussian white noise was added respectively; (3) impulse noise was added respectively.

[0143] (1) No noise was added. The experimental results comparing the models under noise-free conditions are shown in Table 2:

[0144] Table 2

[0145]

[0146] (2) Add Gaussian white noise respectively:

[0147] To further verify the noise robustness of SRA-SNN, experiments were conducted on the aforementioned experimental models under noisy conditions. To simulate noise interference in the AUV dataset, Gaussian white noise with different signal-to-noise ratios (SNRs) was added. Specifically, noise with SNRs of 8 dB and 4 dB was added to the dataset. The experimental models were then trained and tested on the noisy dataset to demonstrate their noise resistance performance. The experimental results for each model under the two Gaussian noise conditions (8 dB and 4 dB) are shown in Table 3 below. The bold text in the table indicates the best diagnostic performance or the smallest standard deviation.

[0148] Experimental results show that the performance of all experimental models significantly degraded under different Gaussian noise conditions. For example, when the SNR was 8dB, the F1-Score of SRA-SNN dropped from 97.32% to 89.72%; the F1-Score of Wen-CNN dropped from 95.27% to 84.73%; the F1-Score of Multi-LSTM dropped from 91.85% to 79.04%; and the F1-Score of Gu-1DCNN dropped from 95.62% to 83.80%. However, the SRA-SNN provided by this invention still achieved the best fault diagnosis performance, and improved by at least 3% compared to Wen-CNN. This indicates that SRA-SNN has good noise resistance, mainly due to its impulse-based information transmission mechanism. The impulse-based information transmission mechanism transmits information only through impulse spatiotemporal sequences, thus suppressing the influence of noise signals on the model. However, ANNs use floating-point numbers to transmit information, and noise signals affect the entire model along with the information transmission, so their noise resistance is poor. When the SNR is 4dB, the performance of these models degrades even more severely. However, the SRA-SNN proposed in this invention still has better diagnostic performance than Wen-CNN, MFCNN, and Conv-Trans, further demonstrating that SRA-SNN has good noise resistance in noisy environments.

[0149] Table 3

[0150]

[0151] (3) Add impulse noise separately:

[0152] This section further evaluates the performance of SRA-SNN and ten experimental models under impulse noise conditions. Impulse noise is characterized by its intermittent nature, consisting of irregular pulses or noise spikes with short durations and large amplitudes. Considering the intermittent nature of impulse noise, its impact on the signal is quantified by calculating the percentage of signal values ​​affected by impulses and the total number of signal values. For example, a 10% impulse noise level is assumed to mean that 10% of the signal values ​​are represented by impulses. The experimental results for each model under two different impulse noise conditions (10% and 20%) are shown in Table 4 below.

[0153] Experimental results reveal the significant impact of impulse noise on SRA-SNN and other ANN-based experimental models. Specifically, when 10% impulse noise is present in the signal, the diagnostic accuracy of SRA-SNN drops from 97.55% to 93.57%. Similarly, the diagnostic accuracy of Wen-CNN drops from 95.76% to 88.98%, Multi-GRU from 93.61% to 86.73%, and Conv-Trans from 96.73% to 91.33%. Clearly, SRA-SNN exhibits a relatively small decrease and achieves the best fault diagnosis performance. This observation highlights the high noise robustness of SNN models under impulse noise conditions. Even with a noise level increase to 20%, SRA-SNN achieves an accuracy of 88.31% and an F1-score of 87.45%, surpassing the performance of other experimental models. The experimental results provide strong evidence that the proposed SRA-SNN achieves noise robustness across various noise types and levels.

[0154] Table 4

[0155]

[0156] Since SRA-SNN can achieve an accuracy of 97.55% on real multi-sensor AUV datasets, it demonstrates that SRA-SNN can serve as an efficient and low-power fault diagnosis solution.

[0157] Corresponding to the aforementioned fault diagnosis method embodiments, this specification also provides embodiments of a fault diagnosis device. These device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of its host device reading the corresponding computer program from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 7 The diagram shown is a hardware structure diagram of the device containing the fault diagnosis device described in this manual. (Except for...) Figure 7In addition to the processor, network interface, memory, and non-volatile memory shown, the device in the embodiment may also include other hardware depending on the actual communication function, which will not be described in detail here.

[0158] Please see Figure 8 This is a block diagram of a fault diagnosis device provided in one embodiment of this specification, the device corresponding to... Figure 1 The illustrated embodiment is applied to an autonomous unmanned system. The autonomous unmanned system is equipped with multiple sensors and a fault diagnosis model constructed using a spiking neural network based on a self-attention mechanism. The spiking neural network includes a convolutional coding layer, a feature extraction layer, and an output layer. The feature extraction layer includes a first type of convolutional layer for feature extraction and a self-attention layer connected to the first type of convolutional layer. The output layer includes a second type of convolutional layer and a fully connected layer connected to the second type of convolutional layer. The device includes:

[0159] The acquisition unit 910 acquires multi-sensor signals representing the operating state of the autonomous unmanned system collected by the multiple sensors; wherein the signals carry the acquisition time.

[0160] The diagnostic unit 920 inputs the multi-sensor signals into the convolutional coding layer of the fault diagnosis model. The convolutional coding layer encodes the multi-sensor signals into a pulse spatiotemporal sequence according to the acquisition time of the sensor signals, and outputs the pulse spatiotemporal sequence to the feature extraction layer. The first type of convolutional layer in the feature extraction layer extracts sequence features from the pulse spatiotemporal sequence, and the self-attention layer connected to the first type of convolutional layer calculates the attention weights corresponding to the sequence features. The sequence features and the attention weights corresponding to the sequence features are further output to the output layer. The second type of convolutional layer in the output layer further extracts predictive features related to fault detection from the sequence features based on the attention weights corresponding to the sequence features, and outputs the predictive features to the fully connected layer. The fully connected layer performs fault detection for the autonomous unmanned system based on the predictive features.

[0161] Optionally, the feature extraction layer includes at least two sets of first-type convolutional layers stacked in a serial order for feature extraction, and a self-attention layer connected to the first-type convolutional layers.

[0162] Optionally, both the first type of convolutional layer and the second type of convolutional layer include a neuron model for simulating neurons in the human brain, the neuron model including:

[0163]

[0164] Where, τ mV represents the membrane time constant, and V(t) represents the membrane potential of the neuron. rest R represents the resting potential of a neuron. m Let I(t) represent the leakage resistance of the neuron, and let I(t) represent the input current of the neuron.

[0165] Optionally, the first and second convolutional layers further include an internal recurrent feature learning algorithm to make them recurrent convolutional layers; wherein the recurrent feature learning algorithm includes:

[0166]

[0167] in, The recurrent convolutional layer represents the potential change of the neuron simulated by the recurrent convolutional layer. S[t] represents the input feature of the recurrent convolutional layer at time t, used to simulate the input potential of the neuron at time t; V[t-1] represents the modulus potential of the neuron at time t-1; H[t] represents the membrane potential of the neuron at time t; 1 / τ m This represents the update gate used to preserve important features in S[t], 1-1 / τ m This represents a forget gate used to discard unimportant features in V[t-1]; the important features include features that are highly relevant to fault detection, and the unimportant features include features that are low in relation to fault detection.

[0168] Optionally, the neuron will fire when its membrane potential exceeds a threshold, and the mode potential will be reset.

[0169] The discharge includes iteratively processing the important features in S[t] retained by the update gate as new input potentials to the neuron that has reset the membrane potential.

[0170] Optionally, the calculation of attention weights corresponding to the sequence features by the self-attention layer connected to the convolutional layer in the diagnostic unit 920 includes:

[0171] The self-attention layer connected to the convolutional layer encodes each type of feature in the sequence features into a first positional code and a second positional code according to the time step; matrix multiplication is performed on the first positional code and the second positional code at each time step to obtain an attention score representing the similarity between the first positional code and the second positional code; the attention score is normalized to obtain the attention weight corresponding to the sequence features.

[0172] Optionally, the fault diagnosis model is trained using a gradient substitution algorithm to optimize the model parameters through gradient backpropagation.

[0173] Optionally, the gradient substitution algorithm includes a Gaussian error function;

[0174] The optimization of model parameters using the gradient substitution algorithm with gradient backpropagation includes:

[0175] The gradient generated by the Gaussian error function replaces the gradient required for the transition function in the gradient backpropagation, and the replaced gradient is used to optimize the model parameters through backpropagation.

[0176] Optionally, the loss function of the fault diagnosis model includes:

[0177]

[0178] Where N represents the number of training samples, The coded value representing the fault type, This represents the predicted feature of the nth training sample.

[0179] Optionally, the autonomous unmanned system includes an autonomous underwater vehicle.

[0180] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0181] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0182] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0183] Corresponding to the aforementioned fault diagnosis method embodiments, this specification also provides an autonomous underwater vehicle, on which an autonomous unmanned system is deployed, and the autonomous unmanned system implements the above-mentioned fault diagnosis method.

[0184] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0185] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0186] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

Claims

1. A fault diagnosis method, characterized in that, The method is applied to an autonomous unmanned system equipped with multiple sensors and a fault diagnosis model constructed using a spiking neural network based on a self-attention mechanism. The spiking neural network includes a convolutional coding layer, a feature extraction layer, and an output layer. The feature extraction layer includes a first type of convolutional layer for feature extraction and a self-attention layer connected to the first type of convolutional layer. The output layer includes a second type of convolutional layer and a fully connected layer connected to the second type of convolutional layer. Acquire multi-sensor signals characterizing the operating state of the autonomous unmanned system collected by the multiple sensors; wherein the signals carry the acquisition time. The multi-sensor signals are input into the convolutional coding layer of the fault diagnosis model. The convolutional coding layer encodes the multi-sensor signals into a pulse spatiotemporal sequence according to the acquisition time of the sensor signals, and outputs the pulse spatiotemporal sequence to the feature extraction layer. The first type of convolutional layer in the feature extraction layer extracts sequence features from the pulse spatiotemporal sequence, and the self-attention layer connected to the first type of convolutional layer calculates the attention weights corresponding to the sequence features. The sequence features and the attention weights corresponding to the sequence features are further output to the output layer. The second type of convolutional layer in the output layer further extracts predictive features related to fault detection from the sequence features based on the attention weights corresponding to the sequence features, and outputs the predictive features to the fully connected layer. The fully connected layer performs fault detection for the autonomous unmanned system based on the predictive features. The first and second convolutional layers further include an internal recurrent feature learning algorithm to make them recurrent convolutional layers; wherein the recurrent feature learning algorithm includes: ； in, This represents the potential changes of neurons simulated by a recurrent convolutional layer. This represents the input features of the recurrent convolutional layer at time t, used to simulate the input potential of the neuron at time t; This represents the modal potential of the neuron at time t-1. This represents the membrane potential of the neuron at time t; Indicates for reservation Update gates for important features, Indicates to discard The important features include those highly correlated with fault detection, while the unimportant features include those with low correlation to fault detection. The neuron will fire when its membrane potential exceeds a firing threshold, and the modal potential will be reset. The firing includes retaining the update gate. The key features in the membrane are transmitted as new input potentials to the neurons that reset the membrane potential for iterative processing.

2. The method according to claim 1, characterized in that, The feature extraction layer includes at least two sets of first-type convolutional layers stacked in a serial order for feature extraction, and a self-attention layer connected to the first-type convolutional layers.

3. The method according to claim 1, characterized in that, Both the first and second type of convolutional layers include neuron models for simulating neurons in the human brain, wherein the neuron models include: ； in, Indicates the membrane time constant. Represents the membrane potential of a neuron. This represents the resting potential of a neuron. Indicates the leakage resistance of a neuron. This represents the input current to the neuron.

4. The method according to claim 1, characterized in that, The calculation of attention weights corresponding to the sequence features by the self-attention layer connected to the convolutional layer includes: Each type of feature in the sequence features is encoded into a first positional code and a second positional code according to the time step by a self-attention layer connected to the convolutional layer. Matrix multiplication is performed on the first positional code and the second positional code at each time step to obtain an attention score representing the similarity between the first positional code and the second positional code; The attention scores are normalized to obtain the attention weights corresponding to the sequence features.

5. The method according to claim 1, characterized in that, The fault diagnosis model is trained using a gradient substitution algorithm to optimize model parameters through gradient backpropagation.

6. The method according to claim 5, characterized in that, The gradient substitution algorithm includes a Gaussian error function; The optimization of model parameters using the gradient substitution algorithm with gradient backpropagation includes: The gradient generated by the Gaussian error function replaces the gradient required for the transition function in the gradient backpropagation, and the replaced gradient is used to optimize the model parameters through backpropagation.

7. The method according to claim 1, characterized in that, The loss function of the fault diagnosis model includes: ； Where N represents the number of training samples, The coded value representing the fault type, This represents the predicted feature of the nth training sample.

8. A fault diagnosis device, characterized in that, This device is applied to an autonomous unmanned system, which is equipped with multiple sensors and a fault diagnosis model constructed using a spiking neural network based on a self-attention mechanism. The spiking neural network includes a convolutional coding layer, a feature extraction layer, and an output layer. The feature extraction layer includes a first type of convolutional layer for feature extraction and a self-attention layer connected to the first type of convolutional layer. The output layer includes a second type of convolutional layer and a fully connected layer connected to the second type of convolutional layer. The device includes: The acquisition unit acquires multi-sensor signals representing the operating state of the autonomous unmanned system collected by the multiple sensors; wherein the signals carry the acquisition time. The diagnostic unit inputs the multi-sensor signals into the convolutional coding layer of the fault diagnosis model. The convolutional coding layer encodes the multi-sensor signals into a pulse spatiotemporal sequence according to the acquisition time of the sensor signals, and outputs the pulse spatiotemporal sequence to the feature extraction layer. The first type of convolutional layer in the feature extraction layer extracts sequence features from the pulse spatiotemporal sequence, and the self-attention layer connected to the first type of convolutional layer calculates the attention weights corresponding to the sequence features. The sequence features and the attention weights corresponding to the sequence features are further output to the output layer. The second type of convolutional layer in the output layer further extracts predictive features related to fault detection from the sequence features based on the attention weights corresponding to the sequence features, and outputs the predictive features to the fully connected layer. The fully connected layer performs fault detection for the autonomous unmanned system based on the predictive features. The first and second convolutional layers further include an internal recurrent feature learning algorithm to make them recurrent convolutional layers; wherein the recurrent feature learning algorithm includes: ； in, This represents the potential changes of neurons simulated by a recurrent convolutional layer. This represents the input features of the recurrent convolutional layer at time t, used to simulate the input potential of the neuron at time t; This represents the modal potential of the neuron at time t-1. This represents the membrane potential of the neuron at time t; Indicates for reservation Update gates for important features, Indicates to discard The important features include those highly correlated with fault detection, while the unimportant features include those with low correlation to fault detection. The neuron will fire when its membrane potential exceeds a firing threshold, and the modal potential will be reset. The firing includes retaining the update gate. The key features in the membrane are transmitted as new input potentials to the neurons that reset the membrane potential for iterative processing.

9. An autonomous underwater vehicle, characterized in that, The autonomous underwater vehicle is equipped with an autonomous unmanned system, which implements the method described in any one of claims 1-7.

Images