AUV fault diagnosis method based on hybrid neural network with temporal attention mechanism

Abstract

The present invention discloses an AUV fault diagnosis method based on a hybrid neural network with a time attention mechanism. The method first uses a local feature extraction module to use a one-dimensional CNN network to extract local features on the time axis of AUV sensor monitoring data, and realizes adaptive dimension reduction through a receptive field. Then, a BiLSTM is used to bidirectionally obtain the correlation between AUV faults and variables, and the forward and backward implicit state vectors are coupled, and a time pattern attention mechanism is embedded. The attention weight vector is extracted through the unit state vector and the implicit state vector of the BiLSTM network unit at the previous moment and the next moment, and a new hidden layer state vector containing the degree of information contribution of the previous and next key moments is obtained. Finally, the fault prediction output is obtained through an activation function. Comprehensive evaluation criteria and algorithm comparison show that the proposed method is more suitable for the fault diagnosis of deep-sea autonomous underwater robots, and can detect minor faults in time and perform diagnostic output.

Description

Technical Field

[0001] The present invention relates to the field of computer machine learning, and in particular to an AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network. Background Art

[0002] AUV is an important tool for human beings to understand, explore and develop the ocean, and is a high-tech means to achieve sustainable development. AUV plays an important role in underwater equipment maintenance, seabed survey and evaluation, ocean search, anti-submarine and other fields. When an AUV fails, it may lead to mission failure or loss of the AUV. In order to ensure its safety and avoid major losses, AUV fault diagnosis has become one of the important research issues to ensure the safety of AUV navigation.

[0003] The complexity of the deep-sea environment and the nonlinearity, strong coupling and model uncertainty of AUVs have brought certain difficulties to the research on AUV fault diagnosis. At present, AUV fault diagnosis methods can be basically divided into three categories: rule-based, model-based and data-driven fault diagnosis methods. Bian Xinqian et al. established a fuzzy fault tree for AUV and used operation rules to calculate the fuzzy probability that the AUV cannot work properly. Chu Zhenzhong et al. proposed a terminal sliding mode observer thruster fault reconstruction method, which reconstructed the thruster fault by constructing a sliding mode observer and using the equivalent output error. Shumsky et al. designed a set of nonlinear diagnostic observers as a prediction model for AUV thrusters, and obtained fault residual information by comparing actual behavior with model behavior. When it is difficult to establish an accurate AUV model, data-driven becomes an important solution. With the rapid development of artificial intelligence technology, deep learning technology has achieved remarkable results in many aspects. Nascimento et al. proposed a recurrent neural network underwater thruster diagnosis method based on voltage, speed and other data. Duan Jie et al. proposed an AUV fault predictor based on radial basis function neural network, which achieved higher accuracy. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides an AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network, which achieves a high fault recognition rate without relying on expert experience and establishing a very accurate AUV model. By extracting the spatial and temporal features of monitoring data, some tiny features of the AUV can be captured, effectively solving the problem of low fault diagnosis accuracy and speeding up the diagnosis.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0006] The AUV fault diagnosis method based on the temporal attention mechanism hybrid neural network includes the following steps:

[0007] Establishing AUV fault diagnosis model:

[0008] Step 1: Collect data and preprocess the input data to normalize the data;

[0009] Step 2: Use a one-dimensional CNN network to extract local features on the time axis of the input data, and achieve adaptive dimensionality reduction through the receptive field;

[0010] Step 3: Use the BiLSTM network to bidirectionally obtain the correlation between AUV faults and variables, couple the forward and backward implicit state vectors, and obtain the characteristic state vector;

[0011] Step 4: Use the temporal pattern attention mechanism to extract the attention weight vector and obtain a new hidden layer state vector containing the contribution of the information of the previous and next key moments;

[0012] Step 5: Calculate the mode loss value, back-propagate and update the model parameters to obtain the optimal fault diagnosis model;

[0013] Real-time troubleshooting:

[0014] Step 6: Collect the on-site equipment data of the AUV to be diagnosed in real time, input the data into the optimal fault diagnosis model for prediction after preprocessing, and obtain the AUV fault diagnosis result.

[0015] The input data includes horizontal angular rate, vertical angular rate, rolling angular rate, depth gauge depth value, depth gauge altitude value, PHINS heading angle, PHINS pitch angle, PHINS rolling angle, DVL data quality, DVL forward speed, DVL left speed, DVL vertical speed, and stern left motor speed monitoring variables.

[0016] Step 1 specifically includes the following steps:

[0017] Step 1.1: Use the one-hot encoding method to convert AUV state features into numerical features;

[0018] Step 1.2: Window the numerical feature data: Cut or fill each sample to fix the sequence length so that each sample contains a preset number of fault data points;

[0019] Step 1.3: Normalize the monitoring time series to between (0,1), and the calculation formula is:

[0020]

[0021] Among them, x' is the normalized value.

[0022] The AUV status feature is a working status label of the AUV device, including: whether it is working normally, fault type, and fault level; the numerical feature is each numerical label corresponding to the working status label of the AUV device.

[0023] Step 2 specifically includes the following steps:

[0024] Step 2.1: Input AUV monitoring sequence x t (t=1,2,3,…,N), extract features through convolution operation, generate output through nonlinear activation function, and the convolution layer is calculated as:

[0025] c t = r(w t x t +b t )

[0026] Among them, r(·) is the neural network activation function, w t is the CNN network weight matrix, b t is the bias term;

[0027] Step 2.2: Through the pooling operation, the redundant information in the feature map is removed and the dimension of the feature map is reduced. The pooling calculation is:

[0028] η(c t ,c t-1 )=max(c t ,c t-1 )

[0029] z t =η(c t ,c t-1 )+β t

[0030] Among them, c t is the convolution output feature map, η(·) is the maximum pooling subsampling function, β t is the bias term, max(·) is the maximum value of the specified area in the calculated feature map, highlighting the local key fault information;

[0031] Step 2.3: Obtain high-level features through full connection calculation, the calculation is as follows:

[0032] v t = r(u t z t +ε t )

[0033] Among them, z t represents the output of the pooling layer, u t is the CNN network weight matrix, ε t is the bias term, v tis the output;

[0034] Step 3 specifically includes the following steps:

[0035] Step 3.1: Solve the hidden state vectors for the forward and backward passes and The high-level features obtained in step 2 are fed into the BiLSTM network and obtained through standard LSTM block calculation;

[0036] Step 3.2: Concatenate the hidden states of the forward unit and the backward unit, and calculate it as:

[0037]

[0038] Here, σ(·) is a concatenated function. At this time, the output state of the standard LSTM block is m t .

[0039] Step 4 specifically includes the following steps:

[0040] Step 4.1: Calculate the temporal attention weight of the fault variable at the current moment This weight is affected by the hidden state of the BiLSTM unit. The influence depends on the cell state m of the BiLSTM cell at the previous moment and the next moment. t-1 、m t+1 and the hidden state h t-1 、h t+1 ;

[0041]

[0042] Among them, Q and W are the weight matrices of the multi-layer perceptron in the temporal attention mechanism, K is the bias term, tanh(·) is the hyperbolic tangent activation function, and σ(·) is the concatenation function;

[0043] Step 4.2: Find quantitative features Make the weight coefficient The sum of the weights is 1, and the softmax activation function is used to normalize them, which is calculated as follows:

[0044]

[0045] Where exp(·) is an exponential function, including the characteristics of the previous and next moments The contribution weight of the current moment is calculated through the activation function;

[0046] Step 4.3: Obtain comprehensive information about the state characteristics of the forecast time series The calculation is as follows:

[0047]

[0048] Step 4.4: Get the input of the BiLSTM unit network Fusion of local spatial features of multidimensional correlated variables v t and get;

[0049]

[0050] Step 4.5: Find the hidden layer state at the current moment

[0051]

[0052] Among them, f is a BiLSTM network unit, and its input is a weighted feature containing the weights of the association between AUV monitoring variables and faults.

[0053] Step 5 specifically includes the following steps:

[0054] Step 5.1: Adaptively extract the contribution rate of the monitoring variable features, capture the semantic relationship between the monitoring variables and the fault, and calculate the AUV fault prediction value y:

[0055]

[0056] Among them, F θ is a network model containing model learning parameters θ, and The transformed weights and biases of the entire network;

[0057] Step 5.2: Calculate the loss error between the predicted value and the actual fault, update the model parameters through back propagation and iterative calculation to obtain the optimal model;

[0058]

[0059] Among them, n is the AUV fault state quantity, j is the specific state type, is the state output of the target, and y is the actual diagnosis state output by the fault diagnosis model.

[0060] The AUV fault diagnosis result is a working status label of the AUV equipment.

[0061] The present invention has the following beneficial effects and advantages:

[0062] 1. This method does not rely on expert experience, nor does it need to establish a very accurate AUV model. It fully extracts fault features through a deep network model and obtains deep-level features of slowly changing minor faults to avoid mutations into serious functional faults in the late stage and cause losses.

[0063] 2. Through the hybrid network structure, the amount of calculation is reduced, the convergence speed of the model is improved, and the local optimum is escaped. By adding L2 regularization to limit the size of learning weights, the model cannot arbitrarily fit the random noise in the data, preventing overfitting problems.

[0064] 3. Fully analyze the AUV data and fault characteristics, and deeply consider the correlation between data variables and faults and the temporal characteristics of sequence data. The local spatial characteristics of AUV monitoring data are captured through convolutional neural networks, and then bidirectional long short-term memory networks are used to extract bidirectional feature information between fault variables. A temporal attention mechanism is designed to increase the attention of desired features and obtain minor fault features, resulting in a high fault recognition rate that meets the requirements of practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flow chart of the method of the present invention;

[0066] Figure 2 It is the CNN network;

[0067] Figure 3 It is a BiLSTM network;

[0068] Figure 4 It is a comparison diagram of confusion matrices using the method of the present invention and the traditional method;

[0069] Figure 5 It is a comparison chart of multiple evaluation indicators using the method of the present invention and traditional methods. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0072] like Figure 1 As shown, the present invention provides an AUV fault diagnosis method based on a time attention mechanism hybrid neural network, comprising the following steps:

[0073] Step 1: Select 5 common AUV faults for analysis, preprocess the input data to normalize the data; the input data includes 40 monitoring variables such as horizontal angular rate, vertical angular rate, rolling angular rate, depth gauge depth value, depth gauge altitude value, PHINS navigator heading angle, PHINS navigator pitch angle, PHINS navigator roll angle, DVL (Doppler log) data quality, DVL forward speed, DVL left speed, DVL vertical speed, stern left motor speed, stern right motor speed for fault diagnosis.

[0074] The AUV state features are converted into numerical features using the unique hot encoding method. The AUV state features are the working state labels of the AUV equipment, including: whether it is working normally, fault type, and fault level; the numerical features are the numerical labels corresponding to the working state labels of the AUV equipment. Examples of state features and numerical features are as follows: the value of the normal state is 0, the value of the stern left motor fault state is 1, the value of the stern right motor fault state is 2, the value of the depth sensor fault state is 3, and the value of the battery fault state is 4. The data is windowed, and each sample is cut or padded to fix the sequence length so that each sample contains 256 fault data points, and the monitoring time series is normalized to between (0,1). Where x' is the normalized value.

[0075] Step 2: Use a one-dimensional CNN network to extract local features on the time axis of the input data, and achieve adaptive dimensionality reduction through the receptive field;

[0076] like Figure 2 As shown, through the convolution operation c t = r(w t x t +b t ) extracts features, generates output through nonlinear activation function, and performs pooling operation. t = r(w t x t +b t ), z t =η(c t ,c t-1 )+β t , remove redundant information in the feature map, reduce the dimension of the feature map, and connect v t = r(u t z t +ε t ) calculates and obtains high-level features, where x t (t=1,2,3,…,N) is the input AUV monitoring sequence; w t 、u t is the CNN network weight matrix; b t , βt , ε t is the bias term; c t is the convolution output feature map; r(·) is the neural network activation function; η(·) is the maximum pooling subsampling function; max(·) is the maximum value of the specified area in the feature map, highlighting the local key fault information; z t Represents the output of the pooling layer.

[0077] Step 3: Use BiLSTM to bidirectionally obtain the correlation between AUV faults and variables, couple the forward and backward implicit state vectors, and obtain the state vector;

[0078] like Figure 3 As shown in Figure 1, using the bidirectional extraction of local spatial correlation features of the input, the standard LSTM block calculates the forward and reverse implicit state vectors and Then Concatenate the hidden states of the forward unit and the backward unit. Where σ(·) is the concatenation function. At this time, the output state of the standard LSTM block is m t .

[0079] Step 4: Use the temporal pattern attention mechanism to extract the attention weight vector and obtain a new hidden layer state vector containing the contribution of the information of the previous and next key moments;

[0080] Input the hidden layer state of the BiLSTM unit and the unit state m at the previous moment and the next moment t-1 、m t+1 and the hidden state h t-1 、h t+1 ,pass Calculate the time attention weight of the fault variable at the current moment to obtain the time attention weight of the fault variable at the current moment Where Q and W are the weight matrices of the multi-layer perceptron in the temporal attention mechanism, K is the bias term, and tanh(·) is the hyperbolic tangent activation function step.

[0081] In order to make the sum of the weight coefficients equal to 1, the softmax activation function is used to normalize the calculation. exp(·) is an exponential function, including the characteristics of the previous and next moments The contribution weight at the current moment is calculated through the activation function By adding all the weights The weighted sum of the state of the corresponding hidden layer Get comprehensive information about the state characteristics of the forecasted time series The local spatial features v of the multidimensional associated variables will be extracted t and Fusion as input to BiLSTM unit network in Finally, find the hidden layer state at the current moment

[0082]

[0083] f is a BiLSTM network unit. The input is no longer the original data extraction information, but the weighted features containing the correlation weights between the AUV monitoring variables and the faults.

[0084] Step 5: Calculate the mode loss value, update the model parameters, obtain the optimal model, and predict the AUV failure based on the optimal model;

[0085] Adaptively extract the contribution rate of monitoring variable features, capture the semantic relationship between monitoring variables and faults, and calculate the AUV fault prediction value y. where F θ is a network model containing model learning parameters θ, and Calculate the loss error between the predicted value and the actual fault for the transformed weights and biases of the entire network Where n is the AUV fault state quantity, j is the specific state type, is the state output of the target, and y is the actual diagnosis state output by the fault diagnosis model. The model parameters are updated through iterative calculation to obtain the optimal model for online input prediction.

[0086] Step 6: Collect the on-site equipment data of the AUV to be diagnosed in real time, input the optimal fault diagnosis model for prediction after preprocessing, and obtain the AUV fault diagnosis result. The AUV fault diagnosis result is the working status label of the AUV equipment.

[0087] In order to verify the effectiveness and stability of the proposed method for AUV fault diagnosis, the currently better AUV fault diagnosis algorithms such as RNN, BiLSTM, and CNN-BiLSTM were selected for comparison. Four common faults and one normal state were diagnosed using the above algorithms and the proposed method, respectively. The obtained diagnostic results are shown in the confusion matrix. Figure 4Among (1) to (5), (1) RNN method, (2) BiLSTM method, (3) CNN method, (4) CNN-BiLSTM method, and (5) the method of the present invention. The method of the present invention fully learns the coupling characteristic information between fault monitoring variables and can diagnose implicit faults. The correct rate in predicting the right stern motor fault and the right stern motor fault both reached 1, indicating that the model fully learned the time information of related variables such as motor voltage and speed. At the same time, the prediction of depth sensor fault and battery fault also achieved the best results compared with other models, proving that the method of the present invention can learn the time correlation between faults and time series data. Figure 5 Among the evaluation indicators of recall rate, precision rate, F1 and accuracy, the method of the present invention achieves the best effect among the five compared models, and can achieve a faster speed, has better engineering practice significance, and realizes better AUV fault diagnosis.

[0088] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. AUV fault diagnosis method based on temporal attention mechanism hybrid neural network, characterized in that: The following steps are involved: Establishing AUV fault diagnosis model: Step 1: Collect data and preprocess the input data to normalize the data; Step 2: Use a one-dimensional CNN network to extract local features on the time axis of the input data, and achieve adaptive dimensionality reduction through the receptive field; Step 3: Use the BiLSTM network to bidirectionally obtain the correlation between AUV faults and variables, couple the forward and backward implicit state vectors, and obtain the characteristic state vector; Step 4: Use the temporal pattern attention mechanism to extract the attention weight vector and obtain a new hidden layer state vector containing the contribution of the information of the previous and next key moments; Step 5: Calculate the mode loss value, back-propagate and update the model parameters to obtain the optimal fault diagnosis model; Real-time troubleshooting: Step 6: Collect the on-site equipment data of the AUV to be diagnosed in real time, input the data into the optimal fault diagnosis model for prediction after preprocessing, and obtain the AUV fault diagnosis result.

2. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 1 is characterized in that: The input data includes horizontal angular rate, vertical angular rate, rolling angular rate, depth gauge depth value, depth gauge altitude value, PHINS heading angle, PHINS pitch angle, PHINS rolling angle, DVL data quality, DVL forward speed, DVL left speed, DVL vertical speed, and stern left motor speed monitoring variables.

3. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 1 is characterized in that: Step 1 specifically includes the following steps: Step 1.1: Use the one-hot encoding method to convert AUV state features into numerical features; Step 1.2: Window the numerical feature data: Cut or fill each sample to fix the sequence length so that each sample contains a preset number of fault data points; Step 1.3: Normalize the monitoring time series to between (0,1), and the calculation formula is: Among them, x' is the normalized value.

4. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 3 is characterized in that: The AUV status feature is a working status label of the AUV device, including: whether it is working normally, fault type, and fault level; the numerical feature is each numerical label corresponding to the working status label of the AUV device.

5. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 1, characterized in that: Step 2 specifically includes the following steps: Step 2.1: Input AUV monitoring sequence x t (t=1,2,3,…,N), extract features through convolution operation, generate output through nonlinear activation function, and the convolution layer is calculated as: c t =r(w t x t +b t ) Among them, r(·) is the neural network activation function, w t is the CNN network weight matrix, b t is the bias term; Step 2.2: Through the pooling operation, the redundant information in the feature map is removed and the dimension of the feature map is reduced. The pooling calculation is: the(c t ,c t-1 )=max(c t ,c t-1 ) z t =η(c t ,c t-1 )+b t Among them, c t is the convolution output feature map, η(·) is the maximum pooling subsampling function, β t is the bias term, max(·) is the maximum value of the specified area in the calculated feature map, highlighting the local key fault information; Step 2.3: Obtain high-level features through full connection calculation, the calculation is as follows: v t =r(u t z t +ε t ) Among them, z t represents the output of the pooling layer, u t is the CNN network weight matrix, ε t is the bias term, v t is the output.

6. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 1 is characterized in that: Step 3 specifically includes the following steps: Step 3.1: Solve the hidden state vectors for the forward and backward passes and The high-level features obtained in step 2 are fed into the BiLSTM network and obtained through standard LSTM block calculation; Step 3.2: Concatenate the hidden states of the forward unit and the backward unit, and calculate as: Among them, σ(·) is a series function. At this time, the output state of the standard LSTM block is m t .

7. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 1 is characterized in that: Step 4 specifically includes the following steps: Step 4.1: Calculate the temporal attention weight of the fault variable at the current moment This weight is affected by the hidden state of the BiLSTM unit. The influence depends on the cell state m of the BiLSTM cell at the previous moment and the next moment. t-1 、m t+1 and the hidden state h t-1 、h t+1 ; Among them, Q and W are the weight matrices of the multi-layer perceptron in the temporal attention mechanism, K is the bias term, tanh(·) is the hyperbolic tangent activation function, and σ(·) is the concatenation function; Step 4.2: Find quantitative features Make the weight coefficient The sum of the weights is 1, and the softmax activation function is used to normalize them, which is calculated as follows: Where exp(·) is an exponential function, including the characteristics of the previous and next moments The contribution weight of the current moment is calculated through the activation function; Step 4.3: Obtain comprehensive information about the state characteristics of the forecast time series The calculation is as follows: Step 4.4: Get the input of the BiLSTM unit network Fusion of local spatial features of multidimensional correlated variables v t and get; Step 4.5: Find the hidden layer state at the current moment Among them, f is a BiLSTM network unit, and its input is a weighted feature containing the weights of the association between AUV monitoring variables and faults.

8. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 1, characterized in that: Step 5 specifically includes the following steps: Step 5.1: Adaptively extract the contribution rate of the monitoring variable features, capture the semantic relationship between the monitoring variables and the fault, and calculate the AUV fault prediction value y: Among them, F θ is a network model containing model learning parameters θ, and The transformed weights and biases of the entire network; Step 5.2: Calculate the loss error between the predicted value and the actual fault, update the model parameters through back propagation and iterative calculation to obtain the optimal model; Among them, n is the AUV fault state quantity, j is the specific state type, is the state output of the target, and y is the actual diagnosis state output by the fault diagnosis model.

9. The AUV fault diagnosis method based on a temporal attention mechanism hybrid neural network according to claim 1, characterized in that: The AUV fault diagnosis result is a working status label of the AUV equipment.

Images