Rotating assembly fault early warning method and device based on multi-modal data fusion

Abstract

The invention discloses a rotating component fault early warning method and device based on multi-modal data fusion. The method and device are used for solving the technical problem that an existing rotating component fault early warning method is poor in early warning effect. The method comprises the following steps: acquiring multi-modal signal data, and performing preprocessing according to the multi-modal signal data by adopting a band-pass filtering algorithm to generate a multi-modal feature vector set; performing feature fusion on the multi-modal feature vector set by adopting a recurrent neural network based on an attention mechanism, and outputting a target health index; based on a preset optimization search algorithm, adopting an initial hidden Markov model to generate a target hidden state sequence according to the target health index; and adopting a preset nonlinear Wiener degradation model to generate a rotating assembly fault early warning result according to the target hidden state sequence and the multi-mode signal data.

Description

Technical Field

[0001] The present invention relates to the technical field of rotating machinery, and in particular to a rotating component fault early warning method and device with multi-modal data fusion. Background Art

[0002] Rotating parts of rotating machinery such as injection molding machines and flexible winding machines play an irreplaceable role in ensuring the stable operation of the equipment. Once damaged during the production process, it may lead to product defects or even catastrophic equipment failure. According to authoritative statistics, about 30% of equipment failures are directly attributed to damage to rotating parts, which further highlights the importance of the early warning mechanism for rotating parts failure of injection molding machines.

[0003] At present, data samples that reflect the operating status of mechanical rotating parts are mostly collected by vibration acceleration sensors. However, the data samples collected in actual work are also mixed with various vibration source signals such as other equipment and components, as well as inevitable environmental background noise. This makes the signals that can reflect the effective fault information of mechanical rotating parts seriously interfered or even completely submerged in the noise, greatly increasing the difficulty of fault warning.

[0004] Most of the existing rotating component fault warning methods are based on machine learning methods to calculate data distribution under the information amount of a single sensor, thereby predicting the failure time of rotating components. However, the amount of information from a single sensor is not comprehensive enough, and the reflection of the degree of damage is not intuitive enough, resulting in poor warning effect. Summary of the invention

[0005] The present invention provides a rotating component fault early warning method and device based on multi-modal data fusion, which are used to solve the technical problem that the existing rotating component fault early warning method leads to poor early warning effect.

[0006] A first aspect of the present invention provides a rotating component fault early warning method using multi-modal data fusion, comprising:

[0007] Acquire multimodal signal data, and use a bandpass filtering algorithm to preprocess the multimodal signal data to generate a multimodal feature vector set;

[0008] Using a recurrent neural network based on an attention mechanism to perform feature fusion on the multimodal feature vector set and output a target health indicator;

[0009] Based on a preset optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health indicator;

[0010] A preset nonlinear Wiener degradation model is used to generate a rotating component fault warning result according to the target latent state sequence and the multimodal signal data.

[0011] Optionally, the multimodal signal data includes initial acoustic emission signal data and initial vibration signal data; the multimodal feature vector set includes a vibration feature vector set and an acoustic emission feature vector set; and the preprocessing of the multimodal signal data using a bandpass filtering algorithm to generate a multimodal feature vector set includes:

[0012] Performing data cleaning on the initial acoustic emission signal data and the initial vibration signal data respectively to generate intermediate acoustic emission signal data and intermediate vibration signal data;

[0013] Time-aligning the intermediate acoustic emission signal data and the intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data;

[0014] Based on the bandpass filtering algorithm, high-frequency signal extraction is performed on multiple time-step acoustic emission sub-signals in the target acoustic emission signal data, and multiple acoustic emission high-frequency signals are output;

[0015] Based on the bandpass filtering algorithm, extract low-frequency signals from multiple time-step vibration sub-signals in the target vibration signal data, and output multiple vibration low-frequency signals;

[0016] Feature extraction is performed on the plurality of acoustic emission high-frequency signals and the plurality of vibration low-frequency signals respectively to generate a vibration feature vector set and an acoustic emission feature vector set.

[0017] Optionally, the attention mechanism-based recurrent neural network includes a recurrent neural network and an attention mechanism network; the use of the attention mechanism-based recurrent neural network to perform feature fusion on the multimodal feature vector set and output a target health indicator includes:

[0018] Normalizing the vibration feature vector set and the acoustic emission feature vector set respectively, and outputting a normalized vibration feature vector set and a normalized acoustic emission feature vector set;

[0019] Performing time alignment on the normalized vibration feature vector set and the normalized acoustic emission feature vector set, and outputting a target vibration feature vector set and a target acoustic emission feature vector set;

[0020] The target vibration feature vector set and the target acoustic emission feature vector set are respectively used as inputs of a recurrent neural network, and a vibration hidden state vector corresponding to the target vibration feature vector set and an acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set are output;

[0021] Using an attention mechanism network to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector according to the vibration hidden state vector and the acoustic emission hidden state vector respectively;

[0022] Splicing the vibration context vector and the acoustic emission context vector to generate a splicing vector;

[0023] The concatenated vector is mapped and a target health indicator is output.

[0024] Optionally, the adopting of the attention mechanism network to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector respectively according to the vibration hidden state vector and the acoustic emission hidden state vector, comprises:

[0025] Performing linear transformation on the vibration hidden state vector and the acoustic emission hidden state vector respectively to generate a plurality of vibration attention scores and a plurality of acoustic emission attention scores;

[0026] Nonlinearly mapping the plurality of vibration attention scores and the plurality of acoustic emission attention scores is performed respectively, and outputting a plurality of time-step vibration attention weights and a plurality of time-step acoustic emission attention weights;

[0027] Weighting the vibration attention weights of the multiple time steps and the vibration features of the multiple time steps in the vibration hidden state vector to generate a vibration context vector;

[0028] The acoustic emission attention weights of the multiple time steps and the acoustic emission features of the multiple time steps in the acoustic emission hidden state vector are weighted to generate an acoustic emission context vector.

[0029] Optionally, the preset optimization search algorithm includes a particle swarm optimization algorithm and a Viterbi algorithm; the method based on the preset optimization search algorithm, using an initial hidden Markov model to generate a target hidden state sequence according to the target health indicator, includes:

[0030] The particle swarm optimization algorithm is used to train the initial hidden Markov model and determine the target hidden Markov model;

[0031] Taking the target health indicator as an observation sequence and inputting it into the target hidden Markov model, outputting the target observation sequence probability;

[0032] The target hidden state sequence is calculated using the Viterbi algorithm according to the target observation sequence probability.

[0033] Optionally, the using a preset nonlinear Wiener degradation model to generate a rotating component fault warning result according to the target latent state sequence and the multimodal signal data includes:

[0034] Dividing the target hidden state sequence to generate a normal state sequence, a fault development state sequence, and a fault failure state sequence;

[0035] Screening the multimodal signal data based on the fault occurrence point corresponding to the normal state sequence to determine the multimodal signal data before degradation;

[0036] The preset nonlinear Wiener degradation model is used to issue an early warning based on the multimodal signal data before degradation, the fault deterioration point threshold corresponding to the fault development state sequence, and the complete failure point threshold corresponding to the fault failure state sequence, thereby generating a rotating component fault early warning result.

[0037] A second aspect of the present invention provides a multi-modal data fusion rotating component fault warning device, comprising:

[0038] An acquisition module, used to acquire multimodal signal data and generate a multimodal feature vector set according to the multimodal signal data using a bandpass filtering algorithm;

[0039] A feature fusion module, used to perform feature fusion on the multimodal feature vector set using a recurrent neural network based on an attention mechanism, and output a target health indicator;

[0040] An output sequence module, for generating a target hidden state sequence according to the target health indicator using an initial hidden Markov model based on a preset optimization search algorithm;

[0041] The output result module is used to generate a rotating component fault warning result according to the target latent state sequence and the multimodal signal data by using a preset nonlinear Wiener degradation model.

[0042] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the rotating component fault warning method of multimodal data fusion as described in any one of the above items.

[0043] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of the rotating component fault warning method using multimodal data fusion as described in any one of the above.

[0044] A fifth aspect of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the steps of the rotating component fault warning method using multimodal data fusion as described in any one of the above items.

[0045] It can be seen from the above technical solutions that the present invention has the following advantages:

[0046] The above technical scheme of the present invention provides a rotating component fault warning method of multimodal data fusion, first, multimodal signal data is obtained, and a bandpass filtering algorithm is used to preprocess the multimodal signal data to generate a multimodal feature vector set; then, a recurrent neural network based on an attention mechanism is used to perform feature fusion on the multimodal feature vector set, and a target health index is output; based on a preset optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health index; finally, a preset nonlinear Wiener degradation model is used to generate a rotating component fault warning result according to the target hidden state sequence and multimodal signal data; based on the above scheme, a recurrent neural network based on an attention mechanism is used to generate a target health index according to a multimodal feature vector set obtained through preprocessing and feature fusion, and then combined with a preset optimization search algorithm, the initial hidden Markov model and the preset nonlinear Wiener degradation model are used to process the target health index to generate a rotating component fault warning result. The present invention uses a recurrent neural network based on an attention mechanism to adaptively fuse multimodal data features to achieve multimodal information complementarity, thereby obtaining richer fault information, thereby improving the warning effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 A flowchart of a method for early warning of a rotating component fault using multi-modal data fusion provided in the first embodiment of the present invention;

[0049] Figure 2 The acoustic emission RMS and vibration RMS diagrams before fusion provided in the first embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a fused health indicator (target health indicator) provided in Embodiment 1 of the present invention;

[0051] Figure 4 A schematic diagram of a process flow of a target health indicator provided in the first embodiment of the present invention;

[0052] Figure 5 A schematic diagram of a flow chart for dividing the whole life of a rotating component into different state stages provided in the first embodiment of the present invention;

[0053] Figure 6 A full life cycle signal and state division diagram of a rotating component provided in the first embodiment of the present invention;

[0054] Figure 7 The incremental distribution diagram of adjacent time steps of the fusion signal provided in the first embodiment of the present invention;

[0055] Figure 8 This is a warning effect diagram of RUL prediction in the fault deterioration stage provided in the first embodiment of the present invention;

[0056] Fig. 9 This is a warning effect diagram of the complete failure RUL prediction provided in the first embodiment of the present invention;

[0057] Fig.10 A schematic diagram of a processing flow of a rotating component fault warning result provided in the first embodiment of the present invention;

[0058] Fig.11 The overall framework diagram of the rotating component fault early warning method of multi-modal data fusion provided in the second embodiment of the present invention;

[0059] Fig.12 This is a structural block diagram of a rotating component fault warning device with multi-modal data fusion provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0060] The embodiments of the present invention provide a rotating component fault early warning method and device based on multi-modal data fusion, which are used to solve the technical problem that the existing rotating component fault early warning method leads to poor early warning effect.

[0061] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1 , Figure 1 A flowchart of the steps of a rotating component fault warning method using multimodal data fusion provided in Example 1 of the present invention.

[0063] The present invention provides a rotating component fault early warning method using multi-modal data fusion, comprising:

[0064] Step 101: Acquire multimodal signal data, and use a bandpass filtering algorithm to preprocess the multimodal signal data to generate a multimodal feature vector set.

[0065] The multimodal signal data includes initial acoustic emission signal data and initial vibration signal data.

[0066] The multimodal feature vector set includes a vibration feature vector set and an acoustic emission feature vector set.

[0067] It should be noted that the multimodal data is preprocessed based on the bandpass filtering algorithm to ensure the validity of the data. The collected multimodal signal data includes the vibration signal and acoustic emission signal of the rotating parts throughout their life cycle (the acoustic emission sampling frequency is 500Khz and the vibration sensor sampling frequency is 20Khz).

[0068] Specifically, the process of preprocessing the multimodal signal data using a bandpass filtering algorithm to generate a multimodal feature vector set can be achieved by executing the following sub-steps S11 to S15:

[0069] Step S11, respectively cleaning the initial acoustic emission signal data and the initial vibration signal data to generate intermediate acoustic emission signal data and intermediate vibration signal data;

[0070] Step S12: Time-aligning the intermediate acoustic emission signal data and the intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data;

[0071] Step S13, based on a bandpass filtering algorithm, extracting high-frequency signals from multiple time-step acoustic emission sub-signals in the target acoustic emission signal data, and outputting multiple acoustic emission high-frequency signals;

[0072] Step S14: Based on a bandpass filtering algorithm, extract low-frequency signals from multiple time-step vibration sub-signals in the target vibration signal data, and output multiple vibration low-frequency signals;

[0073] Step S15: extract features from the multiple acoustic emission high-frequency signals and the multiple vibration low-frequency signals respectively to generate a vibration feature vector set and an acoustic emission feature vector set.

[0074] It should be noted that the data of vibration signal and acoustic emission signal are cleaned separately and aligned in time to ensure the validity of the data. Among them, since the acoustic emission sensor mainly collects high-frequency signals and the vibration sensor mainly focuses on low-frequency information, in order to increase the frequency information contained in the same time part, the vibration signal and the acoustic emission signal are complementary in the frequency domain. After the acoustic emission and vibration are aligned in time, bandpass filtering is used to extract the acoustic emission sub-signal (time step acoustic emission sub-signal) of each time step in the target acoustic emission signal data and the vibration sub-signal (time step vibration sub-signal) of each time step in the target vibration signal data, and output multiple acoustic emission high-frequency signals 10Khz-250Khz and multiple vibration low-frequency signals 0~10Khz. At this point, the two modal data realize frequency domain information complementarity in the frequency domain (0~250Khz).

[0075] Furthermore, feature extraction is performed on the two modal data to construct a vibration feature vector set and an acoustic emission feature vector set. The vibration feature vector set and the acoustic emission feature vector set both correspond to 17-dimensional feature data, as shown in Table 1:

[0076] Table 1 Feature set

[0077]

[0078] In this embodiment, multimodal signal data is acquired, and a bandpass filtering algorithm is used to pre-process the multimodal signal data to generate a multimodal feature vector set.

[0079] Step 102: Use a recurrent neural network based on an attention mechanism to perform feature fusion on the multimodal feature vector set and output the target health indicator.

[0080] It should be noted that the recurrent neural network based on the attention mechanism is an LSTM-AT (Long Short-Term Memory-Attention) neural network, that is, a recurrent neural network (LSTM) with an attention mechanism, which is composed of a recurrent neural network and an attention mechanism network.

[0081] Further, step 102 may include the following sub-steps S21-S26:

[0082] Step S21, normalizing the vibration feature vector set and the acoustic emission feature vector set respectively, and outputting a normalized vibration feature vector set and a normalized acoustic emission feature vector set;

[0083] Step S22, time-aligning the normalized vibration feature vector set and the normalized acoustic emission feature vector set, and outputting a target vibration feature vector set and a target acoustic emission feature vector set;

[0084] It should be noted that, at each time step, the acoustic emission data and vibration data are standardized and time-aligned to obtain acoustic emission data samples and vibration data samples at each time step. Since the acoustic emission data and vibration data of rotating parts often contain positive and negative values, the method of normalizing each extracted feature is used for preprocessing to complete the standardization of each sample to prevent some samples from having too much influence on the training of the model. The normalization calculation is shown in the following formula:

[0085] ;

[0086] Among them, X is the data after normalization; x is the data before normalization; is the minimum value of the sample feature; is the maximum value of the sample feature.

[0087] Step S23, respectively taking the target vibration feature vector set and the target acoustic emission feature vector set as inputs of the recurrent neural network, and outputting the vibration hidden state vector corresponding to the target vibration feature vector set and the acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set;

[0088] It should be noted that the preprocessed target vibration feature vector set and target acoustic emission feature vector set are respectively input into the LSTM+AT network (recurrent neural network based on attention mechanism) to extract the feature vectors of acoustic emission and vibration respectively. AE Take the extraction process as an example.

[0089] The recurrent neural network is used to calculate the acoustic emission hidden state vector H AE The extraction process is as follows:

[0090] (1) Forget Gate : The forget gate determines the memory unit of the current time step How much of it comes from the memory unit of the previous time step? :

[0091] ;

[0092] in, is the forgetting state at time t (time step); is the weight matrix of the forget gate; is the bias of the forget gate; is the sigmoid activation function; is the hidden state at time t-1, indicating the hidden state at the previous moment; is the input for the current time step.

[0093] (2) Input Gate : The input gate determines the input of the current time step How much of it can be written into the current memory cell:

[0094] ;

[0095] in, is the input state at time t; is the sigmoid activation function; is the weight matrix of the input gate; is the hidden state at time t-1, indicating the hidden state at the previous moment; is the current input vector; is the bias of the input gate.

[0096] (3) Candidate memory units : Candidate memory units generate new candidate memories , these memories will be input into the memory cell of the current time step:

[0097] ;

[0098] in, is the candidate memory unit at time t; is the weight matrix of the candidate memory unit; is the bias of the candidate memory unit; is the hidden state at time t-1, represents the hidden state at the previous moment; tanh is the hyperbolic tangent activation function.

[0099] (4) Current memory unit :According to the output of the forget gate and the input gate, update the memory unit of the current time step :

[0100] ;

[0101] in, is the memory unit of the current time step; is the forgetting state at time t; is the memory unit at time t-1; is the input state at time t; is the candidate memory unit at time t.

[0102] (5) Output gate : The output gate determines the memory unit of the current time step Which parts will be output to the features of the current time step:

[0103] ;

[0104] in, is the output state at time t; is the weight matrix of the output gate; is the bias of the output gate; is the sigmoid activation function; is the hidden state at time t-1, indicating the hidden state at the previous moment, that is, the feature of the previous time step.

[0105] (6) Current Features : The characteristics of the current time step is composed of the output gate and the current memory unit Calculated by nonlinear activation function (tanh):

[0106] ;

[0107] in, For the current feature; is the memory unit of the current time step; is the output state at time t; tanh is the hyperbolic tangent activation function.

[0108] (7) Overall formula expression:

[0109] ;

[0110] in, is the input of the current time step; is the feature of the previous time step; is the memory unit of the previous time step; , , , is the weight matrix of each gate; , , , is the bias term of each gate; is the Sigmoid activation function; tanh is the hyperbolic tangent activation function.

[0111] Further, after the above processing, the obtained multiple Construct the acoustic emission hidden state vector H AE The principle of the steps for extracting the vibration hidden state vector is consistent with the principle of the steps for extracting the acoustic emission hidden state vector, and the present invention will not be described in detail.

[0112] Step S24, using an attention mechanism network to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector according to the vibration hidden state vector and the acoustic emission hidden state vector respectively;

[0113] Specifically, step S24 may include the following sub-steps S241-S244:

[0114] Step S241, linearly transform the vibration latent state vector and the acoustic emission latent state vector respectively to generate multiple vibration attention scores and multiple acoustic emission attention scores;

[0115] Step S242, performing nonlinear mapping on the multiple vibration attention scores and the multiple acoustic emission attention scores respectively, and outputting multiple time-step vibration attention weights and multiple time-step acoustic emission attention weights;

[0116] Step S243, weighting the vibration attention weights of multiple time steps and the vibration features of multiple time steps in the vibration hidden state vector to generate a vibration context vector;

[0117] Step S244: weight the acoustic emission attention weights of multiple time steps and the acoustic emission features of multiple time steps in the acoustic emission hidden state vector to generate an acoustic emission context vector.

[0118] Step S25, splicing the vibration context vector and the acoustic emission context vector to generate a splicing vector;

[0119] Step S26: Map the splicing vector and output the target health index.

[0120] It should be noted that the attention mechanism is used to calculate the weighted output of each time step. Specifically, a linear transformation is used to calculate the weight of each time step. Assume that the input is the acoustic emission hidden state vector H AE , N is the feature dimension, T is the total time step, and the score is calculated by linear transformation:

[0121] ;

[0122] Where S is the acoustic emission attention score; V is the first weight matrix; W is the second weight matrix; H AE is the acoustic emission hidden state vector; tanh is the tanh activation function.

[0123] Furthermore, the attention score is nonlinearly mapped through the softmax activation function to obtain the weight of each time step:

[0124] ;

[0125] in, is the time step acoustic emission attention weight, which represents the acoustic emission attention weight of the t-th time step; is the acoustic emission attention score at the t-th time step; softmax is the softmax activation function.

[0126] Furthermore, the features of all time steps are weighted to obtain the final AE context vector:

[0127] ;

[0128] in, is the acoustic emission context vector; is the time step acoustic emission attention weight, which represents the acoustic emission attention weight of the t-th time step; is the time-step acoustic emission characteristic, that is, the acoustic emission characteristic of the tth time step; T is the overall time step.

[0129] It is worth mentioning that the principle of the steps to obtain the acoustic emission context vector is consistent with the principle of the steps to obtain the vibration context vector. By performing the same processing on the target vibration feature vector set, the vibration feature vector H is obtained. V (Vibration hidden state vector), based on the above steps, the vibration context vector is obtained (Vibration context vector).

[0130] Furthermore, the context vectors of the vibration and acoustic emission data are concatenated:

[0131] ;

[0132] in, is the concatenation vector; concat means concatenation.

[0133] For further information, see Figure 2-Figure 3 , the concatenation result (concatenation vector) of the acoustic emission mode and vibration mode context vector obtained above is mapped through the ReLU activation function and the fully connected layer to obtain the final health index (target health index). The fused full life cycle health index diagram, that is, the signal comparison diagram before and after the fusion of acoustic emission and vibration signals, is shown in Figure 2-Figure 3 shown.

[0134] For example, see Figure 4 , each signal is processed independently by a recurrent neural network (LSTM) with an attention mechanism to obtain a feature vector H, and then the score of H is calculated using attention. According to the attention score, the context vector is fused and mapped through the ReLU activation function and the fully connected layer to obtain the final health indicator (target health indicator).

[0135] In this embodiment, a recurrent neural network based on an attention mechanism is used to perform feature fusion on a multimodal feature vector set and output a target health indicator.

[0136] Step 103: Based on a preset optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health indicator.

[0137] Pre-set optimization search algorithms include particle swarm optimization and Viterbi algorithm.

[0138] Specifically, step 103 may include the following sub-steps S31-S33:

[0139] Step S31, using a particle swarm optimization algorithm to train the initial hidden Markov model and determine the target hidden Markov model;

[0140] Step S32, taking the target health indicator as an observation sequence and inputting it into the target hidden Markov model, and outputting the target observation sequence probability;

[0141] It should be noted that the parameters of the initial HMM model (Hidden Markov Model) are optimized by the PSO algorithm (Particle Swarm Optimization) so that the model can maximize the log-likelihood of the observed signal. Specifically, the particle swarm optimization algorithm (PSO) is used to optimize the parameters (transfer matrix, mean, covariance) of the initial HMM to maximize the log-likelihood of the observed signal; where each particle represents a set of initialization parameters of the HMM model:

[0142] ;

[0143] in, is the initialization parameter; transmat is the state transfer matrix, , is the number of states, and the number of states of the present invention is 3, that is (including "normal operation state", "fault development state" and "fault deterioration and failure state"); is the mean value for each state, ; is the variance of each state; is the set of real numbers.

[0144] The speed and position update rules for each particle:

[0145] ;

[0146] Among them, w is the inertia weight, which dynamically adjusts the particle speed; , are the acceleration factors, which control the influence of the particle’s own experience and the group’s experience respectively; is the historical optimal position of particle i; is the global optimal position; is the current position of the particle; is the update speed of the particle at the current moment.

[0147] The fitness function of a particle is:

[0148] ;

[0149] ;

[0150] in, is the log-likelihood value of the signal, the log-likelihood value of the observation sequence X, which is used to optimize the model parameters; For a given parameter The probability of observing sequence X under the condition of For a given parameter The joint probability of the observation sequence X and the hidden state sequence Q under the condition of

[0151] Further, the forward algorithm is used to calculate until the maximum log-likelihood value is found. This process can be divided into three parts:

[0152] 1) Initialization:

[0153] ;

[0154] in, is the probability of being in state i at time t, calculated by the forward algorithm; is the initial state distribution, which indicates the probability of being in state i at the initial moment; is the observed value at the initial time t=1 The probability of being in state i.

[0155] 2) Recursion:

[0156] ;

[0157] in, is the probability of being in state j at time t, calculated by the forward algorithm; is the observed value at time t in state j Probability This represents the transition probability from state i to state j; is the forward probability of being in state i at time t−1.

[0158] 3) Termination:

[0159] ;

[0160] in, For a given parameter The probability of observing the sequence X under the condition of (i.e., the probability of the target observation sequence); is the forward probability of being in state i at time T. This value is obtained by recursive calculation from time 1 to time T.

[0161] Step S33: Calculate the target hidden state sequence according to the target observation sequence probability using the Viterbi algorithm.

[0162] It should be noted that the Viterbi algorithm is used to calculate the optimal state sequence Q*, and the state of each time step t Represents the stage the signal is in. The HMM model uses the Viterbi algorithm to calculate the most likely hidden state sequence:

[0163] ;

[0164] in, is the optimal state sequence (target hidden state sequence); To find the The maximum value of Q is reached.

[0165] ;

[0166] in, For a given parameter The joint probability of the observation sequence X and the hidden state sequence Q under the condition of; is the probability of the hidden state sequence Q, indicating that given the parameters The probability of the hidden state sequence Q appearing under the condition of; is the probability of observing the sequence X, which means that given the parameters The probability of the observation sequence X occurring under the condition of the hidden state sequence Q.

[0167] Specifically, the processing of the target hidden state sequence can be divided into three parts:

[0168] 1) Initialization:

[0169] ;

[0170] in, For a given parameter Under the condition that the first state of the hidden state sequence Q is i, and the first observation value is observed probability; For a given parameter Under the condition of , the probability that the first state i of the hidden state sequence Q is ; For a given parameter When the first state of the hidden state sequence Q is i, the first observation value is observed probability; This means that at time t=1, there is no previous state because this is the beginning of the sequence.

[0171] 2) Recursion:

[0172] ;

[0173] in, is the maximum probability of reaching state j at time point t; To select the maximum value for all possible states i; is the maximum probability of reaching state i at time point t-1; is the transition probability from state i to state j; For a given state Under the condition of probability; is the previous state of the optimal path to state j at time point t; To perform the maximum selection for all possible states i, return the i that maximizes the expression.

[0174] 3) Recursion:

[0175] ;

[0176] in, is the optimal hidden state at time point T, which is a state in the optimal state sequence found by backtracking; To find the value at time point T such that The largest state i; is the maximum probability of reaching state i at time point T, which is calculated through the recursive steps; At time t+1, the optimal state is reached The previous state of It means that the backtracking process starts from time point T-1 and gradually backtracks to time point 1.

[0177] Furthermore, the optimal hidden state sequence Q* is calculated based on the PSO-based HMM model. , which is the target hidden state sequence.

[0178] For example, see Figure 5 , because the mean and variance characteristics of different stages reflect the behavioral characteristics of the signal under different physical states. The present invention uses the obtained health indicators as an observation sequence to input the hidden Markov model (HMM), and adopts the particle swarm optimization (PSO) algorithm (including initialization particles, fitness calculation and other steps) to find the optimal initialization HMM parameters, and then updates the optimal parameters, extracts the corresponding state information from the observed data, and uses the Viterbi algorithm to calculate the most likely hidden state. Finally, by analyzing the signal and outputting the state results, the entire life cycle of the rotating parts is divided into three stages: "normal operation state", "fault development state" and "fault deterioration and failure state". The process is as follows: Figure 5 shown.

[0179] In this embodiment, based on a preset optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health indicator.

[0180] Step 104: using a preset nonlinear Wiener degradation model to generate a rotating component fault warning result according to the target latent state sequence and the multimodal signal data.

[0181] Specifically, step 104 may include the following sub-steps S41-S43:

[0182] Step S41, dividing the target hidden state sequence to generate a normal state sequence, a fault development state sequence, and a fault failure state sequence;

[0183] Please note that Figure 6 , the obtained target hidden state sequence is divided into three stages, which correspond to three hidden states (i.e. normal state sequence, fault development state sequence, and fault failure state sequence). It is assumed that the observation probability obeys Gaussian distribution, and the hidden state is divided into three states: normal, fault development, and fault failure. The particle swarm optimization algorithm (PSO) and hidden Markov model (HMM) are used to realize the adaptive identification of the development stage of rotating parts, enhance the automatic processing capability, effectively filter out irrelevant fault information such as noise, and ensure the retention of key fault degradation information.

[0184] Step S42, screening the multimodal signal data based on the fault occurrence point corresponding to the normal state sequence to determine the multimodal signal data before degradation;

[0185] It should be noted that according to the normal state sequence, fault development state sequence, and fault failure state sequence, three time points of the rotating component can be obtained: the fault occurrence point, the fault deterioration point, and the complete failure point. After the fault occurrence point of the rotating component is monitored, it is determined that the rotating component begins to degrade, and the healthy data before degradation is selected, that is, the acoustic emission sub-signal and the vibration sub-signal of the time step before the fault occurrence point are selected in the multimodal signal data as the multimodal signal data before degradation.

[0186] Step S43, using a preset nonlinear Wiener degradation model to issue an early warning based on the multimodal signal data before degradation, the fault deterioration point threshold corresponding to the fault development state sequence, and the complete failure point threshold corresponding to the fault failure state sequence, to generate a rotating component fault early warning result.

[0187] It should be noted that the Wiener process is a type of random process. Due to its clear physical interpretation and good mathematical properties, it is widely used in degradation modeling and remaining life prediction. It has been verified that the increments of adjacent time steps of the fusion signal meet the normal distribution requirements, N (0, 0.06), such as Figure 7 As shown, nonlinear Wiener process can be used for prediction.

[0188] Furthermore, the present invention adopts a nonlinear Wiener process to characterize the random degradation process of the rotating component. Based on the model assumption, the performance value X(t) of the rotating component can be expressed as:

[0189] ;

[0190] in, is the performance status at time t; is the initial state of the system, which is assumed to be 0 here; is a drift term, which can describe the characteristics of system performance degradation that may change nonlinearly over time. is the drift coefficient, is a nonlinear function of time, a is a constant, describing the rate of change over time; is the diffusion term, which represents the random fluctuation of the system state change. is the diffusion coefficient, describing the fluctuation range of the degradation process, This is a standard Brownian motion.

[0191] Furthermore, in order to estimate the drift coefficient and the diffusion coefficient , the least squares method can be used. The specific steps are as follows:

[0192] First, perform the drift coefficient Calculation:

[0193] ;

[0194] Among them, T is the trend term vector; X is the actual degradation data; Q is the covariance matrix.

[0195] Next, perform the diffusion coefficient Calculation:

[0196] ;

[0197] Where N is the number of samples.

[0198] Furthermore, the remaining life L is defined as the degradation process X(t) from the current state X(t0)=x i The time required to reach the fault threshold D is: X(t0+L)=D, x i is the current degradation time point.

[0199] The extended Wiener process model is:

[0200] ;

[0201] in, For in time The degradation rate when For the current Degradation rate of the point; is a random changing process within L time period.

[0202] At time step t=t0, the state of the degradation process is X(t0)=x iFrom this point on, the remaining life L satisfies:

[0203] ;

[0204] Will The expression is normalized (divided by ):

[0205] ;

[0206] Furthermore, Brownian motion Satisfies normal distribution:

[0207] ;

[0208] Therefore, the standardized random variable is:

[0209] ;

[0210] Among them, Z is a standardized random variable; is a standardized integral distribution.

[0211] Based on the above expressions, we can get:

[0212] ;

[0213] In summary, we can obtain the probability density function of solving L (i.e. presetting the nonlinear Wiener degradation model) f L (l), specifically:

[0214] ;

[0215] in, is the current time point; is the current drift coefficient; is the current diffusion coefficient; is the remaining value of the current degradation distance failure threshold; l is the time step of the remaining life.

[0216] It is worth mentioning that the healthy data before degradation (multimodal signal data before degradation) is used to train the Wiener model (pre-set nonlinear Wiener degradation model), and two thresholds are set to obtain the rotating component fault warning result. The rotating component fault warning result includes the RUL distribution of the stage from the rotating component fault occurrence point to the rotating component fault deterioration point and the RUL distribution of the stage from the rotating component fault occurrence point to the rotating component complete failure point; specifically, the threshold of the fault deterioration point D1 (that is, the fault deterioration point threshold corresponding to the fault development state sequence), and the threshold of the complete failure point D2 (that is, the complete failure point threshold corresponding to the fault failure state sequence). The threshold of the fault deterioration point D1 is input into the above-mentioned RUL probability density function (that is, the trained preset nonlinear Wiener degradation model), and the RUL distribution of the stage from the rotating component fault occurrence point to the rotating component fault deterioration point can be obtained, as shown in FIG. Figure 8 As shown. Inputting the threshold value D2 of the complete failure point into the above RUL probability density function, the RUL distribution (Remaining Useful Life Distribution) from the point where the rotating component failure occurs to the point where the rotating component completely fails can be obtained, as shown in Fig. 9 shown.

[0217] Depend on Figure 8-Figure 9 It can be seen that the RUL distribution predicted by the nonlinear Wiener process model is close to the actual degradation time of the rotating parts, and early warning can be achieved.

[0218] For example, see Fig.10 According to the three fault time points divided, namely, the fault occurrence point, the fault deterioration point, and the complete failure point, the present invention obtains two thresholds, namely, the threshold D1 of the fault deterioration point and the threshold D2 of the complete failure point. The performance data before the fault deterioration point (i.e., the multimodal signal data before degradation) is used for model training to complete the performance degradation modeling, and then parameter estimation and remaining life prediction are performed. That is, the threshold D1 of the fault deterioration point is input into the nonlinear Wiener degradation model to obtain the RUL probability density function of each time step, and the RUL distribution from the bearing fault occurrence point to the bearing fault deterioration point can be obtained. Similarly, the threshold D2 of the complete failure point is input into the degradation model to obtain the RUL distribution from the bearing fault occurrence point to the bearing complete failure point. According to the obtained RUL distribution, the time from the normal state of the bearing to the two threshold points can be known, and the fault early warning can be realized.

[0219] In this embodiment, a preset nonlinear Wiener degradation model is used to generate a rotating component fault warning result according to the target latent state sequence and multimodal signal data.

[0220] As a comparison of technical effects, we can refer to existing technologies. At present, although there are many means of bearing status monitoring, they all have limitations. Temperature change monitoring is difficult to perform online dynamic monitoring due to its delayed response; although vibration signal monitoring is widely used, it is easily interfered by multiple factors, and the accuracy and reliability of early fault diagnosis are greatly reduced; the oil monitoring method has limited sensitivity and requires regular sampling of lubricating oil, which is also difficult to meet the needs of online monitoring. In contrast, the acoustic emission signal monitoring method can keenly capture early damage to bearings, but the traditional method is based on impact data analysis and it is difficult to accurately lock the source and status of the fault.

[0221] With the leap forward in computer processing power, real-time processing of acoustic emission waveform signals has become possible. Therefore, many scholars have devoted themselves to the research of acoustic emission waveform signal processing in order to improve the accuracy of bearing life prediction. At present, life prediction technology is mainly divided into three categories: physical model-based, data-driven and hybrid model. Among them, the data-driven method has made great progress in the field of fault prediction by making full use of historical data.

[0222] Data-driven methods can be further divided into data-driven methods based on machine learning and big data training based on deep learning. Although deep learning methods can achieve extremely high prediction accuracy under massive data training, in actual production environments, obtaining signals throughout the entire life cycle is a daunting challenge. Machine learning-based methods can shine under conditions of few samples. There are many prediction methods based on machine learning, such as LSTM networks based on time series data, and data-driven methods based on statistics and random processes. LSTM neural networks based on time series data capture long-term dependencies and improve prediction accuracy, while prediction methods based on the Wiener process can efficiently calculate data distribution and predict failure time under the condition of few sample drives.

[0223] In addition, the prediction method that relies solely on vibration signals is still insufficient, it is difficult to effectively capture early fault signals, and the reflection of the degree of damage is not intuitive enough. Acoustic emission sensors, with their keen ability to capture early signals, have become a new favorite in the field of fault prediction. Combining the advantages of acoustic emission and vibration signals, early identification and prediction of bearing operating conditions are expected to achieve fault warning and effectively avoid huge losses caused by bearing failure.

[0224] In view of the good trend of vibration signals in reflecting the full life operation status of bearings, and the keen perception and damage characterization capabilities of acoustic emission signals for early faults, the present invention proposes a multimodal data fusion rotating component fault warning method to address the above problems. The present invention combines the advantages of LSTM network and Wiener process, and innovatively combines acoustic emission sensor signals with vibration signals to achieve monitoring and early warning of the entire life cycle of bearings. By fusing acoustic emission and vibration signals through the LSTM+attention mechanism method, while improving the monotonicity of the signal, the stage of acoustic emission and the trend of vibration are retained. Then, the fused health index is used as the observation matrix and input into the hidden Markov model based on particle swarm. The optimal hidden Markov parameters are found through the particle swarm algorithm, and then the fault threshold points of different stages are found by the Viterbi algorithm. Finally, through the nonlinear Wiener process, the arrival time of different stages is predicted in combination with the fault threshold points, and early warnings are issued in time. This innovative achievement provides solid technical support for the maintenance management and life extension of industrial equipment.

[0225] Specifically, the present invention first uses a multimodal sensor to collect the original signal of the whole life of the bearing operation (the present invention uses an acoustic emission sensor and a vibration sensor); then, the original signal is preprocessed, data cleaning is performed, and time domain features and time-frequency domain features are extracted from different types of original signals to form a feature set; the features of the acoustic emission signal and vibration signal feature sets are extracted respectively using the LSTM time series neural network method. The attention score of the acoustic emission feature set and the vibration feature set is calculated using the attention mechanism, and the feature signals of the two modes are fused according to the attention score to construct a health index. The health feature index with the highest monotonicity score is input into the hidden Markov model as the observation matrix, and the whole life cycle of the bearing is divided into three states: normal, fault development, fault deterioration and failure, and three time points are obtained: fault occurrence point, fault deterioration point, and complete failure point. Finally, two thresholds are set, the threshold D1 of the fault deterioration point, and the threshold D2 of the complete failure point. The threshold D1 of the fault deterioration point is input into the nonlinear Wiener degradation model to obtain the RUL probability density function of each time step, and the RUL distribution from the bearing fault occurrence point to the bearing fault deterioration point can be obtained. Similarly, by inputting the threshold value D2 of the complete failure point into the degradation model, the RUL distribution from the point where the bearing fault occurs to the point where the bearing fails completely can be obtained. Based on the obtained RUL distribution, the time it takes for the bearing to go from the normal state to the two threshold points can be known, thus realizing early warning of the fault.

[0226] In an embodiment of the present invention, the present invention provides a rotating component fault warning method of multimodal data fusion, first, multimodal signal data is obtained, and a bandpass filtering algorithm is used to preprocess the multimodal signal data to generate a multimodal feature vector set; then, a recurrent neural network based on an attention mechanism is used to perform feature fusion on the multimodal feature vector set, and a target health index is output; based on a preset optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health index; finally, a preset nonlinear Wiener degradation model is used to generate a rotating component fault warning result according to the target hidden state sequence and multimodal signal data; based on the above scheme, a recurrent neural network based on an attention mechanism is used to generate a target health index according to a multimodal feature vector set obtained through preprocessing and feature fusion, and then combined with a preset optimization search algorithm, the initial hidden Markov model and the preset nonlinear Wiener degradation model are used to process the target health index to generate a rotating component fault warning result. The present invention uses a recurrent neural network based on an attention mechanism to adaptively fuse multimodal data features to achieve multimodal information complementarity, thereby obtaining richer fault information, thereby improving the warning effect.

[0227] For better explanation, refer to Fig.11 , showing the overall framework diagram of the rotating component fault warning method of multimodal data fusion provided by the second embodiment of the present invention. It should be pointed out that this embodiment only briefly describes the general process of the rotating component fault warning method of multimodal data fusion, and the specific implementation process of each step can be understood by referring to the relevant content in the aforementioned embodiment, which will not be described here. It can be understood that the present invention is not limited to this.

[0228] The overall framework of the multimodal data fusion rotating component fault warning method proposed in the present invention can be divided into three parts: feature extraction module, data processing module, and fault warning module; in the feature extraction module: it is used for preprocessing the vibration sensor and the acoustic emission sensor, and 17 eigenvalues ​​are extracted respectively; in the data processing module: the module includes data cleaning and data fusion processes, and adopts LSTM+AT (LSTM+Attention time series neural network) to fuse the acoustic emission and vibration signals. This method cleverly uses the characteristics of the obvious phased signal of the acoustic emission and the good trend of the vibration signal to achieve information complementarity between the acoustic emission and the vibration signal, so as to judge the fault occurrence time in advance; in the fault warning module: the hidden Markov model based on particle swarm optimization is adopted to divide the signal into three states: normal state, fault development state, fault deterioration and failure state. And three time points are obtained: fault occurrence point, fault deterioration point, and complete failure point. Then a nonlinear Wiener process degradation model is established, and the degradation model is trained with the data before the fault occurrence point, and two thresholds are set: the threshold corresponding to the fault deterioration point threshold D1 and the complete failure point threshold D2. The nonlinear Wiener process degradation model is used to predict the remaining life (RUL) distribution of the threshold corresponding to the fault deterioration point D1 and the complete failure point D2, and the remaining life of the fault deterioration point and the complete failure point is obtained in order to achieve fault early warning.

[0229] In the embodiment of the present invention, the present invention uses the LSTM time series neural network with attention mechanism to adaptively fuse multimodal data features, realize multimodal information complementation, thereby obtaining richer fault information, getting rid of the limitation of insufficient information of a single sensor. And the fused data can realize phased warning, effectively reducing the risk of failure of rotating parts.

[0230] See also Fig.12 , Fig.12 This is a structural block diagram of a rotating component fault warning device with multi-modal data fusion provided in Example 3 of the present invention.

[0231] The present invention provides a multi-modal data fusion rotating component fault warning device, comprising:

[0232] An acquisition module 1201 is used to acquire multimodal signal data and generate a multimodal feature vector set according to the multimodal signal data using a bandpass filtering algorithm;

[0233] A feature fusion module 1202 is used to perform feature fusion on a multimodal feature vector set using a recurrent neural network based on an attention mechanism, and output a target health indicator;

[0234] The output sequence module 1203 is used to generate a target hidden state sequence according to the target health indicator using an initial hidden Markov model based on a preset optimization search algorithm;

[0235] The output result module 1204 is used to generate a rotating component fault warning result according to the target latent state sequence and multi-modal signal data using a preset nonlinear Wiener degradation model.

[0236] Further, the multimodal signal data includes initial acoustic emission signal data and initial vibration signal data; the multimodal feature vector set includes a vibration feature vector set and an acoustic emission feature vector set; the acquisition module 1201 is specifically used for:

[0237] Performing data cleaning on the initial acoustic emission signal data and the initial vibration signal data respectively to generate intermediate acoustic emission signal data and intermediate vibration signal data;

[0238] Time-aligning the intermediate acoustic emission signal data and the intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data;

[0239] Based on the bandpass filtering algorithm, high-frequency signals are extracted from multiple time-step acoustic emission sub-signals in the target acoustic emission signal data, and multiple acoustic emission high-frequency signals are output;

[0240] Based on the bandpass filtering algorithm, low-frequency signals are extracted from multiple time-step vibration sub-signals in the target vibration signal data, and multiple vibration low-frequency signals are output;

[0241] Feature extraction is performed on a plurality of acoustic emission high-frequency signals and a plurality of vibration low-frequency signals respectively to generate a vibration feature vector set and an acoustic emission feature vector set.

[0242] Furthermore, the recurrent neural network based on the attention mechanism includes a recurrent neural network and an attention mechanism network; the feature fusion module 1202 includes:

[0243] The first submodule is used to normalize the vibration feature vector set and the acoustic emission feature vector set respectively, and output a normalized vibration feature vector set and a normalized acoustic emission feature vector set;

[0244] The second submodule is used to time-align the normalized vibration feature vector set and the normalized acoustic emission feature vector set, and output a target vibration feature vector set and a target acoustic emission feature vector set;

[0245] The third submodule is used to take the target vibration feature vector set and the target acoustic emission feature vector set as inputs of the recurrent neural network, and output the vibration hidden state vector corresponding to the target vibration feature vector set and the acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set;

[0246] The fourth submodule is used to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector according to the vibration hidden state vector and the acoustic emission hidden state vector respectively by using an attention mechanism network;

[0247] A fifth submodule is used to splice the vibration context vector and the acoustic emission context vector to generate a spliced ​​vector;

[0248] The sixth submodule is used to map the concatenated vector and output the target health indicator.

[0249] Furthermore, the fourth submodule is specifically used for:

[0250] Linearly transforming the vibration hidden state vector and the acoustic emission hidden state vector respectively to generate multiple vibration attention scores and multiple acoustic emission attention scores;

[0251] Nonlinear mapping is performed on multiple vibration attention scores and multiple acoustic emission attention scores respectively, and multiple time-step vibration attention weights and multiple time-step acoustic emission attention weights are output;

[0252] Weighting the vibration attention weights of multiple time steps and the vibration features of multiple time steps in the vibration hidden state vector to generate a vibration context vector;

[0253] The acoustic emission attention weights of multiple time steps and the acoustic emission features of multiple time steps in the acoustic emission hidden state vector are weighted to generate an acoustic emission context vector.

[0254] Furthermore, the preset optimization search algorithm includes a particle swarm optimization algorithm and a Viterbi algorithm; the output sequence module 1203 is specifically used for:

[0255] The particle swarm optimization algorithm is used to train the initial hidden Markov model and determine the target hidden Markov model;

[0256] The target health indicator is used as an observation sequence and input into the target hidden Markov model, and the target observation sequence probability is output;

[0257] The Viterbi algorithm is used to calculate the target hidden state sequence according to the target observation sequence probability.

[0258] Furthermore, the output result module 1204 is specifically used for:

[0259] Divide the target hidden state sequence to generate normal state sequence, fault development state sequence, and fault failure state sequence;

[0260] Screening the multimodal signal data based on the fault occurrence point corresponding to the normal state sequence to determine the multimodal signal data before degradation;

[0261] A preset nonlinear Wiener degradation model is used to generate early warning results for rotating component faults based on the multimodal signal data before degradation, the fault deterioration point threshold corresponding to the fault development state sequence, and the complete failure point threshold corresponding to the fault failure state sequence.

[0262] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0263] An embodiment of the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory; when the computer program is executed by the processor, the processor executes the steps of the rotating component fault warning method of multimodal data fusion as in any of the above embodiments.

[0264] An embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the rotating component fault warning method of multimodal data fusion as in any of the above embodiments are implemented.

[0265] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the rotating component fault warning method of multimodal data fusion as in any of the above embodiments.

[0266] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rotating component fault early warning method based on multi-modal data fusion, characterized in that: include: Acquire multimodal signal data, and use a bandpass filtering algorithm to preprocess the multimodal signal data to generate a multimodal feature vector set; Using a recurrent neural network based on an attention mechanism to perform feature fusion on the multimodal feature vector set and output a target health indicator; Based on a preset optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health indicator; A preset nonlinear Wiener degradation model is used to generate a rotating component fault warning result according to the target latent state sequence and the multimodal signal data.

2. The rotating component fault early warning method based on multi-modal data fusion according to claim 1 is characterized in that: The multimodal signal data includes initial acoustic emission signal data and initial vibration signal data; the multimodal feature vector set includes a vibration feature vector set and an acoustic emission feature vector set; The method of using a bandpass filtering algorithm to preprocess the multimodal signal data to generate a multimodal feature vector set includes: Performing data cleaning on the initial acoustic emission signal data and the initial vibration signal data respectively to generate intermediate acoustic emission signal data and intermediate vibration signal data; Time-aligning the intermediate acoustic emission signal data and the intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data; Based on the bandpass filtering algorithm, high-frequency signal extraction is performed on multiple time-step acoustic emission sub-signals in the target acoustic emission signal data, and multiple acoustic emission high-frequency signals are output; Based on the bandpass filtering algorithm, extract low-frequency signals from multiple time-step vibration sub-signals in the target vibration signal data, and output multiple vibration low-frequency signals; Feature extraction is performed on the plurality of acoustic emission high-frequency signals and the plurality of vibration low-frequency signals respectively to generate a vibration feature vector set and an acoustic emission feature vector set.

3. The rotating component fault early warning method based on multi-modal data fusion according to claim 2 is characterized in that: The attention mechanism-based recurrent neural network includes a recurrent neural network and an attention mechanism network; The recurrent neural network based on the attention mechanism is used to perform feature fusion on the multimodal feature vector set and output the target health index, including: Normalizing the vibration feature vector set and the acoustic emission feature vector set respectively, and outputting a normalized vibration feature vector set and a normalized acoustic emission feature vector set; Performing time alignment on the normalized vibration feature vector set and the normalized acoustic emission feature vector set, and outputting a target vibration feature vector set and a target acoustic emission feature vector set; The target vibration feature vector set and the target acoustic emission feature vector set are respectively used as inputs of a recurrent neural network, and a vibration hidden state vector corresponding to the target vibration feature vector set and an acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set are output; Using an attention mechanism network to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector according to the vibration hidden state vector and the acoustic emission hidden state vector respectively; Splicing the vibration context vector and the acoustic emission context vector to generate a splicing vector; The concatenated vector is mapped and a target health indicator is output.

4. The rotating component fault early warning method based on multi-modal data fusion according to claim 3 is characterized in that: The attention mechanism network is used to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector according to the vibration hidden state vector and the acoustic emission hidden state vector, respectively, including: Performing linear transformation on the vibration hidden state vector and the acoustic emission hidden state vector respectively to generate a plurality of vibration attention scores and a plurality of acoustic emission attention scores; Nonlinearly mapping the plurality of vibration attention scores and the plurality of acoustic emission attention scores is performed respectively, and outputting a plurality of time-step vibration attention weights and a plurality of time-step acoustic emission attention weights; Weighting the vibration attention weights of the multiple time steps and the vibration features of the multiple time steps in the vibration hidden state vector to generate a vibration context vector; The acoustic emission attention weights of the multiple time steps and the acoustic emission features of the multiple time steps in the acoustic emission hidden state vector are weighted to generate an acoustic emission context vector.

5. The rotating component fault early warning method based on multi-modal data fusion according to claim 1 is characterized in that: The preset optimization search algorithm includes a particle swarm optimization algorithm and a Viterbi algorithm; the preset optimization search algorithm is based on the initial hidden Markov model to generate a target hidden state sequence according to the target health indicator, including: The particle swarm optimization algorithm is used to train the initial hidden Markov model and determine the target hidden Markov model; Taking the target health indicator as an observation sequence and inputting it into the target hidden Markov model, outputting the target observation sequence probability; The target hidden state sequence is calculated using the Viterbi algorithm according to the target observation sequence probability.

6. The rotating component fault early warning method based on multi-modal data fusion according to claim 1, characterized in that: The method of using a preset nonlinear Wiener degradation model to generate a rotating component fault warning result according to the target hidden state sequence and the multimodal signal data includes: Dividing the target hidden state sequence to generate a normal state sequence, a fault development state sequence, and a fault failure state sequence; Screening the multimodal signal data based on the fault occurrence point corresponding to the normal state sequence to determine the multimodal signal data before degradation; The preset nonlinear Wiener degradation model is used to issue an early warning based on the multimodal signal data before degradation, the fault deterioration point threshold corresponding to the fault development state sequence, and the complete failure point threshold corresponding to the fault failure state sequence, thereby generating a rotating component fault early warning result.

7. A rotating component fault warning device with multi-modal data fusion, characterized in that: include: An acquisition module, used to acquire multimodal signal data and generate a multimodal feature vector set according to the multimodal signal data using a bandpass filtering algorithm; A feature fusion module, used to perform feature fusion on the multimodal feature vector set using a recurrent neural network based on an attention mechanism, and output a target health indicator; An output sequence module, for generating a target hidden state sequence according to the target health indicator using an initial hidden Markov model based on a preset optimization search algorithm; The output result module is used to generate a rotating component fault warning result according to the target latent state sequence and the multimodal signal data by using a preset nonlinear Wiener degradation model.

8. A computer device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the rotating component fault warning method of multimodal data fusion as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the rotating component fault warning method of multimodal data fusion as described in any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the rotating component fault warning method based on multimodal data fusion as described in any one of claims 1-6.

Images