Method and apparatus for early warning of faults in rotating components based on multimodal data fusion

Abstract

This invention discloses a method and apparatus for early warning of rotating component faults based on multimodal data fusion, addressing the technical problem of poor early warning performance in existing methods. The method includes acquiring multimodal signal data and preprocessing it using a bandpass filtering algorithm to generate a multimodal feature vector set; fusing the multimodal feature vector set using an attention-based recurrent neural network to output a target health index; generating a target hidden state sequence based on the target health index using an initial hidden Markov model and a pre-set optimization search algorithm; and generating a rotating component fault early warning result using a pre-set nonlinear Wiener degradation model based on the target hidden state sequence and the multimodal signal data.

Description

Technical Field

[0001] This invention relates to the field of rotating machinery technology, and in particular to a method and apparatus for early warning of faults in rotating components based on multimodal data fusion. Background Technology

[0002] Rotating components in injection molding machines, flexible winding machines, and other rotating machinery play an irreplaceable role in ensuring stable equipment operation. Damage to these components during production can lead to product defects or even catastrophic equipment failure. According to authoritative statistics, approximately 30% of equipment failures are directly attributed to damage to rotating components, further highlighting the importance of early warning mechanisms for rotating component failures in injection molding machines.

[0003] Currently, most data samples reflecting the operating status of mechanical rotating parts are collected using vibration acceleration sensors. However, the data samples collected in actual work are also mixed with signals from various vibration sources such as other equipment and components, as well as unavoidable environmental background noise. This causes the signals that can effectively reflect fault information of mechanical rotating parts to be severely interfered with or even completely submerged in noise, greatly increasing the difficulty of fault early warning.

[0004] Most existing methods for early warning of rotating component failures are based on machine learning to calculate data distribution under the condition of information from a single sensor, thereby predicting the failure time of the rotating component. However, the information from a single sensor is not comprehensive enough and the reflection of the degree of damage is not intuitive enough, resulting in poor early warning effect. Summary of the Invention

[0005] This invention provides a method and apparatus for early warning of rotating component faults based on multimodal data fusion, which solves the technical problem of poor early warning effect caused by existing methods for early warning of rotating component faults.

[0006] The first aspect of this invention provides a method for early warning of faults in rotating components based on multimodal data fusion, comprising:

[0007] Multimodal signal data is acquired, and a bandpass filtering algorithm is used to preprocess the multimodal signal data to generate a multimodal feature vector set;

[0008] A recurrent neural network based on an attention mechanism is used to fuse the multimodal feature vector set and output the target health index.

[0009] Based on a pre-optimized search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health indicators;

[0010] A pre-set nonlinear Wiener degradation model is used to generate a fault warning result for the rotating component based on the target hidden state sequence and the multimodal signal data.

[0011] Optionally, the multimodal signal data includes initial acoustic emission signal data and initial vibration signal data; the multimodal feature vector set includes a vibration feature vector set and an acoustic emission feature vector set; the preprocessing of the multimodal signal data using a bandpass filtering algorithm to generate the multimodal feature vector set includes:

[0012] Data cleaning was performed on the initial acoustic emission signal data and the initial vibration signal data to generate intermediate acoustic emission signal data and intermediate vibration signal data.

[0013] Time alignment is performed on the intermediate acoustic emission signal data and the intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data;

[0014] Based on the bandpass filtering algorithm, high-frequency signals are extracted from multiple time-step acoustic emission sub-signals in the target acoustic emission signal data, and multiple high-frequency acoustic emission signals are output.

[0015] Based on the bandpass filtering algorithm, low-frequency signals are extracted from multiple time-step vibration sub-signals in the target vibration signal data, and multiple low-frequency vibration signals are output.

[0016] Feature extraction is performed on multiple high-frequency acoustic emission signals and multiple low-frequency vibration signals to generate a vibration feature vector set and an acoustic emission feature vector set.

[0017] Optionally, the attention-based recurrent neural network includes a recurrent neural network and an attention mechanism network; the step of using the attention-based recurrent neural network to perform feature fusion on the multimodal feature vector set and output the target health indicator includes:

[0018] The vibration feature vector set and the acoustic emission feature vector set are normalized respectively, and the normalized vibration feature vector set and the normalized acoustic emission feature vector set are output.

[0019] Time alignment is performed on the normalized vibration feature vector set and the normalized acoustic emission feature vector set to output the target vibration feature vector set and the target acoustic emission feature vector set;

[0020] The target vibration feature vector set and the target acoustic emission feature vector set are respectively used as inputs to a recurrent neural network, and the vibration hidden state vector corresponding to the target vibration feature vector set and the acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set are output.

[0021] An attention mechanism network is used to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector, respectively, based on the vibration hidden state vector and the acoustic emission hidden state vector.

[0022] The vibration context vector and the acoustic emission context vector are concatenated to generate a concatenated vector;

[0023] The concatenated vector is mapped to output the target health index.

[0024] Optionally, the attention mechanism network generates a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector based on the vibration hidden state vector and the acoustic emission hidden state vector, respectively, including:

[0025] The vibration latent state vector and the acoustic emission latent state vector are linearly transformed respectively to generate multiple vibration attention scores and multiple acoustic emission attention scores.

[0026] A nonlinear mapping is performed on the multiple vibration attention scores and multiple acoustic emission attention scores respectively to output multiple time-step vibration attention weights and multiple time-step acoustic emission attention weights;

[0027] A vibration context vector is generated by weighting the vibration attention weights of multiple time steps and the vibration features of multiple time steps in the vibration latent state vector.

[0028] The acoustic emission attention weights at multiple time steps and the acoustic emission features at multiple time steps in the acoustic emission latent state vector are weighted to generate an acoustic emission context vector.

[0029] Optionally, the pre-set optimized search algorithm includes particle swarm optimization and Viterbi algorithm; the step of generating the target hidden state sequence based on the pre-set optimized search algorithm and using an initial hidden Markov model according to the target health index includes:

[0030] The particle swarm optimization algorithm is used to train the initial hidden Markov model and determine the target hidden Markov model.

[0031] The target health index is used as an observation sequence and input into the target hidden Markov model to output the probability of the target observation sequence.

[0032] The Viterbi algorithm is used to calculate the target hidden state sequence based on the probability of the target observation sequence.

[0033] Optionally, the step of generating a rotating component fault early warning result based on the target hidden state sequence and the multimodal signal data using a pre-set nonlinear Wiener degradation model includes:

[0034] The target hidden state sequence is divided into normal state sequence, fault development state sequence, and fault failure state sequence.

[0035] The multimodal signal data is filtered based on the fault occurrence points corresponding to the normal state sequence to determine the multimodal signal data before degradation.

[0036] The pre-set nonlinear Wiener degradation model is used to generate early warning results for rotating components based on the pre-degradation multimodal signal data, the fault deterioration point threshold corresponding to the fault development state sequence, and the complete failure point threshold corresponding to the fault failure state sequence.

[0037] A second aspect of the present invention provides a multimodal data fusion-based rotating component fault early warning device, comprising:

[0038] The acquisition module is used to acquire multimodal signal data and generate a multimodal feature vector set based on the multimodal signal data using a bandpass filtering algorithm;

[0039] The feature fusion module is used to perform feature fusion on the multimodal feature vector set using a recurrent neural network based on an attention mechanism, and output the target health index.

[0040] The output sequence module is used to generate a target hidden state sequence based on the target health indicators using an initial hidden Markov model and a pre-set optimized search algorithm.

[0041] The output results module is used to generate a fault warning result for the rotating component based on the target hidden state sequence and the multimodal signal data using a preset nonlinear Wiener degradation model.

[0042] A computer device provided in a third aspect of the present invention includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the multimodal data fusion rotating component fault early warning method as described in any of the preceding claims.

[0043] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the steps of the rotating component fault early warning method for multimodal data fusion as described in any of the preceding claims.

[0044] The fifth aspect of the present invention provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein, when the program instructions are executed by a computer, the computer performs the steps of the multimodal data fusion rotating component fault early warning method as described in any of the preceding claims.

[0045] As can be seen from the above technical solutions, the present invention has the following advantages:

[0046] The above-mentioned technical solution of the present invention provides a method for early warning of rotating component faults based on multimodal data fusion. First, multimodal signal data is acquired, and a bandpass filtering algorithm is used to preprocess the multimodal signal data to generate a multimodal feature vector set. Next, a recurrent neural network based on an attention mechanism is used to fuse the multimodal feature vector set, outputting a target health indicator. Based on a pre-set optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence based on the target health indicator. Finally, a pre-set nonlinear Wiener degradation model is used to generate a rotating component fault warning result based on the target hidden state sequence and the multimodal signal data. Based on the above scheme, the process of generating a target health indicator using a recurrent neural network based on an attention mechanism based on the preprocessed and fused multimodal feature vector set, and then combining this with a pre-set optimization search algorithm, using an initial hidden Markov model and a pre-set nonlinear Wiener degradation model to process the target health indicator and generate a rotating component fault warning result, demonstrates that the present invention utilizes a recurrent neural network based on an attention mechanism to adaptively fuse multimodal data features, achieving multimodal information complementarity, thereby obtaining richer fault information and improving the early warning effect. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the steps of a multimodal data fusion-based rotating component fault early warning method according to Embodiment 1 of the present invention.

[0049] Figure 2 The fusion pre-acoustic emission RMS and vibration RMS diagram provided in Embodiment 1 of the present invention;

[0050] Figure 3 This is a schematic diagram of the fused health indicators (target health indicators) provided in Embodiment 1 of the present invention;

[0051] Figure 4 This is a schematic diagram of the processing flow of the target health indicator provided in Embodiment 1 of the present invention;

[0052] Figure 5 This is a flowchart illustrating the division of different state stages throughout the lifespan of a rotating component according to Embodiment 1 of the present invention.

[0053] Figure 6 This is a diagram showing the full lifecycle signals and state division of a rotating component provided in Embodiment 1 of the present invention;

[0054] Figure 7 This is an incremental distribution diagram of adjacent time steps of the fused signal provided in Embodiment 1 of the present invention;

[0055] Figure 8 This is a diagram illustrating the early warning effect of RUL prediction during the fault deterioration stage provided in Embodiment 1 of the present invention.

[0056] Figure 9 This is a diagram illustrating the early warning effect of a complete failure RUL prediction provided in Embodiment 1 of the present invention.

[0057] Figure 10 This is a schematic diagram of the processing flow of the rotating component fault early warning result provided in Embodiment 1 of the present invention;

[0058] Figure 11 This is an overall framework diagram of the multimodal data fusion-based rotating component fault early warning method provided in Embodiment 2 of the present invention;

[0059] Figure 12 This is a structural block diagram of a rotating component fault early warning device based on multimodal data fusion, provided in Embodiment 3 of the present invention. Detailed Implementation

[0060] This invention provides a method and apparatus for early warning of rotating component faults based on multimodal data fusion, which addresses the technical problem of poor early warning effect caused by existing methods for early warning of rotating component faults.

[0061] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a multimodal data fusion-based rotating component fault early warning method provided in Embodiment 1 of the present invention.

[0063] This invention provides a multimodal data fusion-based method for early warning of faults in rotating components, comprising:

[0064] Step 101: Acquire multimodal signal data and preprocess the multimodal signal data using a bandpass filtering algorithm to generate a multimodal feature vector set.

[0065] Multimodal signal data includes initial acoustic emission signal data and initial vibration signal data.

[0066] The multimodal feature vector set includes the vibration feature vector set and the acoustic emission feature vector set.

[0067] It should be noted that the multimodal data is preprocessed based on a bandpass filtering algorithm to ensure its validity. The acquired multimodal signal data includes vibration signals and acoustic emission signals from the rotating component throughout its entire lifecycle (acoustic emission sampling frequency of 500 kHz and vibration sensor sampling frequency of 20 kHz).

[0068] Specifically, the process of preprocessing multimodal signal data using a bandpass filtering algorithm to generate a multimodal feature vector set can be achieved by executing the following sub-steps S11 to S15:

[0069] Step S11: Clean the initial acoustic emission signal data and the initial vibration signal data respectively to generate intermediate acoustic emission signal data and intermediate vibration signal data;

[0070] Step S12: Time-align the intermediate acoustic emission signal data and intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data;

[0071] Step S13: Based on the bandpass filtering algorithm, high-frequency signals are extracted from the acoustic emission sub-signals of multiple time steps in the target acoustic emission signal data, and multiple acoustic emission high-frequency signals are output.

[0072] Step S14: Based on the bandpass filtering algorithm, extract low-frequency signals from multiple time-step vibration sub-signals in the target vibration signal data and output multiple low-frequency vibration signals.

[0073] Step S15: Extract features from multiple high-frequency acoustic emission signals and multiple low-frequency vibration signals to generate a vibration feature vector set and an acoustic emission feature vector set.

[0074] It should be noted that separate data cleaning and time alignment of the vibration and acoustic emission signals ensure data validity. Since the acoustic emission sensor primarily collects high-frequency signals, while the vibration sensor focuses on low-frequency information, to maximize the inclusion of more frequency information within the same time frame and achieve frequency domain complementarity between the vibration and acoustic emission signals, bandpass filtering is used after time alignment of the acoustic emission and vibration data. This extracts the acoustic emission sub-signals (time-step acoustic emission sub-signals) from each time step of the target acoustic emission signal data and the vibration sub-signals (time-step vibration sub-signals) from each time step of the target vibration signal data, outputting multiple high-frequency acoustic emission signals (10kHz-250kHz) and multiple low-frequency vibration signals (0-10kHz). Thus, the two modal data achieve frequency domain information complementarity (0-250kHz).

[0075] Furthermore, feature extraction was performed on the two modal data respectively, constructing a vibration feature vector set and an acoustic emission feature vector set; both the vibration feature vector set and the acoustic emission feature vector set include 17-dimensional feature data, as shown in Table 1:

[0076] Table 1 Feature Set

[0077]

[0078] In this embodiment, multimodal signal data is acquired, and a bandpass filtering algorithm is used to preprocess the multimodal signal data to generate a multimodal feature vector set.

[0079] Step 102: Use an attention-based recurrent neural network to fuse features of the multimodal feature vector set and output the target health index.

[0080] It should be noted that the recurrent neural network based on the attention mechanism is the LSTM-AT (Long Short-Term Memory-Attention) neural network, which is a recurrent neural network (LSTM) with an attention mechanism, consisting of a recurrent neural network and an attention mechanism network.

[0081] Further, step 102 may include the following sub-steps S21-S26:

[0082] Step S21: Normalize the vibration feature vector set and the acoustic emission feature vector set respectively, and output the normalized vibration feature vector set and the normalized acoustic emission feature vector set;

[0083] Step S22: Time-align the normalized vibration feature vector set and the normalized acoustic emission feature vector set, and output the target vibration feature vector set and the target acoustic emission feature vector set;

[0084] It should be noted that at each time step, the acoustic emission data and vibration data are standardized and time-aligned to obtain acoustic emission data samples and vibration data samples for each time step. Since the acoustic emission data and vibration data of rotating components often contain positive and negative values, a method of normalizing each extracted feature is used for preprocessing to standardize each sample and prevent certain samples from having an excessive impact on the training of the model. The normalization calculation is shown in the following formula:

[0085] ;

[0086] Where X represents the normalized data; x represents the data before normalization; The minimum value of the sample feature; This represents the maximum value of the sample feature.

[0087] Step S23: Take the target vibration feature vector set and the target acoustic emission feature vector set as inputs to the recurrent neural network, and output the vibration hidden state vector corresponding to the target vibration feature vector set and the acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set, respectively.

[0088] It should be noted that the preprocessed target vibration feature vector set and target acoustic emission feature vector set are respectively input into an LSTM+AT network (a recurrent neural network based on an attention mechanism) to extract feature vectors for acoustic emission and vibration, respectively. The following section uses the acoustic emission feature dataset features (acoustic emission hidden state vector) H... AE Take the extraction process as an example.

[0089] Recurrent neural networks for acoustic emission hidden state vector H AE The extraction process is as follows:

[0090] (1) Forgotten Gate The forget gate determines the memory unit at the current time step. How much of it comes from memory units from the previous time step? :

[0091] ;

[0092] in, The forgotten state at time t (time step); Here is the weight matrix for the forget gate; For the offset of the forget gate; It is the sigmoid activation function; Let be the hidden state at time t-1, and represent the hidden state at the previous time step. This is the input for the current time step.

[0093] (2) Input Gate The input gate determines the input at the current time step. How much of it can be written into the current memory cell?

[0094] ;

[0095] in, The input state at time t; It is the sigmoid activation function; This is the weight matrix of the input gate; Let be the hidden state at time t-1, and represent the hidden state at the previous time step. This is the current input vector; This is the bias of the input gate.

[0096] (3) Candidate memory units Candidate memory units generate new candidate memories. These memories will be input into the memory unit at the current time step:

[0097] ;

[0098] in, These are candidate memory units at time t; This is the weight matrix for candidate memory units; Bias for candidate memory cells; Let be the hidden state at time t-1, and represent the hidden state at the previous time; tanh is the hyperbolic tangent activation function.

[0099] (4) Current memory unit Update the memory units at the current time step based on the outputs of the forget gate and the input gate. :

[0100] ;

[0101] in, The memory unit for the current time step; Let t represent the forgotten state at time t; This is the memory unit at time t-1; The input state at time t; Let be the candidate memory unit at time t.

[0102] (5) Output gate The output gate determines the memory unit at the current time step. Which parts of the data will be output to the features at the current time step?

[0103] ;

[0104] in, The output state at time t; This is the weight matrix of the output gate; For the output gate bias; It is the sigmoid activation function; Let be the hidden state at time t-1, and let represent the hidden state at the previous time step, i.e., the feature of the previous time step.

[0105] (6) Current features Features of the current time step It consists of output gates and current memory units. Calculated using the nonlinear activation function (tanh):

[0106] ;

[0107] in, For current features; The memory unit for the current time step; t represents the output state at time t; tanh is the hyperbolic tangent activation function.

[0108] (7) Overall formula expression:

[0109] ;

[0110] in, Input for the current time step; Features of the previous time step; This is the memory unit from the previous time step; , , , It is the weight matrix of each gate; , , , These are the bias terms for each door; is the Sigmoid activation function; tanh is the hyperbolic tangent activation function.

[0111] Furthermore, after the above processing, multiple [items] will be obtained. Constructing the acoustic emission hidden state vector H AE The principle behind extracting the vibration latent state vector is the same as that for extracting the acoustic emission latent state vector, and will not be elaborated further in this invention.

[0112] Step S24: Using an attention mechanism network, generate the vibration context vector corresponding to the vibration hidden state vector and the acoustic emission context vector corresponding to the acoustic emission hidden state vector, respectively, based on the vibration hidden state vector and the acoustic emission hidden state vector.

[0113] Specifically, step S24 may include the following sub-steps S241-S244:

[0114] Step S241: Perform linear transformations on the vibration latent state vector and the acoustic emission latent state vector respectively to generate multiple vibration attention scores and multiple acoustic emission attention scores;

[0115] Step S242: Perform nonlinear mapping on multiple vibration attention scores and multiple acoustic emission attention scores respectively, and output multiple time-step vibration attention weights and multiple time-step acoustic emission attention weights;

[0116] Step S243: Weight the vibration attention weights and vibration features of multiple time steps in the vibration latent state vector at multiple time steps to generate a vibration context vector;

[0117] Step S244: Weight the acoustic emission attention weights and acoustic emission features of multiple time steps in the acoustic emission latent state vector to generate an acoustic emission context vector.

[0118] Step S25: Concatenate the vibration context vector and the acoustic emission context vector to generate a concatenated vector;

[0119] Step S26: Map the spliced ​​vector and output the target health index.

[0120] It should be noted that an attention mechanism is used to calculate the weighted output at each time step. Specifically, a linear transformation is used to calculate the weights at each time step. Assume the input is the acoustic emission hidden state vector H. AE N is the feature dimension, T is the total time step, and the score is calculated through linear transformation:

[0121] ;

[0122] Where S is the acoustic emission attention score; V is the first weight matrix; W is the second weight matrix; H AE is the acoustic emission hidden state vector; tanh is the tanh activation function.

[0123] Furthermore, the attention score is non-linearly mapped using the softmax activation function to obtain the weights at each time step:

[0124] ;

[0125] in, Let be the acoustic emission attention weight at time step , and let represent the acoustic emission attention weight at time step t. Let be the acoustic emission attention score at time step t; softmax is the softmax activation function.

[0126] Furthermore, by weighting the features across all time steps, the final acoustic emission context vector is obtained:

[0127] ;

[0128] in, This is the acoustic emission context vector; Let be the acoustic emission attention weight at time step , and let represent the acoustic emission attention weight at time step t. Let T be the acoustic emission characteristics at each time step, i.e., the acoustic emission characteristics at the t-th time step; T is the total time step.

[0129] It is worth mentioning that the principle of obtaining the acoustic emission context vector is the same as that of obtaining the vibration context vector. By performing the same processing on the target vibration feature vector set, the vibration feature vector H is obtained. V (Vibration hidden state vector), based on the above steps, the vibration context vector is obtained. (Vibration context vector).

[0130] Furthermore, the context vectors of the vibration and acoustic emission data are concatenated:

[0131] ;

[0132] in, `concat` is for concatenating vectors; `concat` is for concatenating.

[0133] Further, please refer to Figures 2-3 The concatenated acoustic emission mode and vibration mode context vectors obtained above (the concatenated vector) are mapped through a ReLU activation function and a fully connected layer to obtain the final health index (target health index). The fused life-cycle health index map, i.e., the signal comparison map before and after the fusion of acoustic emission and vibration signals, is shown below. Figures 2-3 As shown.

[0134] For example, please refer to Figure 4 Each signal is processed independently using a recurrent neural network (LSTM) with an attention mechanism to obtain a feature vector H. Then, the score of H is calculated using attention. Based on the attention score, the context vector is fused and mapped through a ReLU activation function and a fully connected layer to obtain the final health indicator (target health indicator).

[0135] In this embodiment, a recurrent neural network based on an attention mechanism is used to fuse features of a multimodal feature vector set and output the target health index.

[0136] Step 103: Based on the pre-set optimized search algorithm, the initial hidden Markov model is used to generate the target hidden state sequence according to the target health indicators.

[0137] Pre-defined optimization search algorithms include particle swarm optimization and Viterbi algorithm.

[0138] Specifically, step 103 may include the following sub-steps S31-S33:

[0139] Step S31: Use the particle swarm optimization algorithm to train the initial hidden Markov model and determine the target hidden Markov model.

[0140] Step S32: Input the target health indicators as observation sequences into the target hidden Markov model and output the probability of the target observation sequence;

[0141] It should be noted that the parameters of the initial Hidden Markov Model (HMM) are optimized using the Particle Swarm Optimization (PSO) algorithm to maximize the log-likelihood of the observed signal. Specifically, the parameters (transition matrix, mean, covariance) of the initial HMM are optimized using PSO to maximize the log-likelihood of the observed signal; where each particle represents a set of initialization parameters for the HMM model.

[0142] ;

[0143] in, These are the initialization parameters; transmat is the state transition matrix. , The number of states in this invention is 3. (Including "normal operating status", "fault development status" and "fault deterioration and failure status"); The mean for each state, ; The variance for each state; It is the set of real numbers.

[0144] The rules for updating the velocity and position of each particle are as follows:

[0145] ;

[0146] Where w is the inertial weight, which dynamically adjusts the particle velocity; , As acceleration factors, they respectively control the influence of the particle's own experience and the collective experience; This represents the historical best position of particle i; The globally optimal position; This represents the particle's current position. This represents the particle's update speed at the current moment.

[0147] The fitness function of a particle:

[0148] ;

[0149] ;

[0150] in, The log-likelihood of the signal and the log-likelihood of the observed sequence X are used to optimize the model parameters. For given parameters Under the given conditions, the probability of observing sequence X; For given parameters Under the given conditions, the joint probability of the observed sequence X and the hidden state sequence Q.

[0151] Furthermore, the forward algorithm is used to calculate until the maximum log-likelihood value is found. This process can be divided into three parts:

[0152] 1) Initialization:

[0153] ;

[0154] in, The probability of being in state i at time t is calculated by the forward algorithm; Let be the initial state distribution, representing the probability of being in state i at the initial time. For the observation value at the initial time t=1 The probability in state i.

[0155] 2) Recursion:

[0156] ;

[0157] in, The probability of being in state j at time t is calculated by the forward algorithm; Let be the observation at time t in state j. probability This represents the transition probability from state i to state j; Let be the forward probability of being in state i at time t−1.

[0158] 3) Termination:

[0159] ;

[0160] in, For given parameters Under the given conditions, the probability of the observed sequence X (i.e., the probability of the target observed sequence); Let be the forward probability of being in state i at time T. This value is obtained recursively from time 1 to time T.

[0161] Step S33: Calculate the target hidden state sequence based on the target observation sequence probability using the Viterbi algorithm.

[0162] It should be noted that the Viterbi algorithm is used to calculate the optimal state sequence Q*, and the state at each time step t is... This represents the stage of the signal. The Hidden MM model uses the Viterbi algorithm to calculate the most probable sequence of hidden states:

[0163] ;

[0164] in, This is the optimal state sequence (target hidden state sequence). To find Q reaches its maximum value.

[0165] ;

[0166] in, For given parameters Under the condition that, the joint probability of the observed sequence X and the hidden state sequence Q; Let Q be the probability of the hidden state sequence, representing the probability given the parameters. Under the condition that the hidden state sequence Q appears; Let X be the probability of observing the sequence X, representing the probability given the parameter. Given the hidden state sequence Q, the probability of the observed sequence X occurring.

[0167] Specifically, the processing of the target hidden state sequence can be divided into three parts:

[0168] 1) Initialization:

[0169] ;

[0170] in, For given parameters Under the given conditions, the first state of the hidden state sequence Q is i, and the first observation is observed. The probability of; For given parameters Under the condition, the probability that the first state i of the hidden state sequence Q is ; For given parameters Given that the first state of the hidden state sequence Q is i, the first observation is observed. The probability of; This indicates that at time point t=1, there is no previous state, because this is the beginning of the sequence.

[0171] 2) Recursion:

[0172] ;

[0173] in, This represents the maximum probability of reaching state j at time point t. To select the maximum value among all possible states i; This represents the maximum probability of reaching state i at time point t-1. Let be the transition probability from state i to state j; In a given state Under these conditions, the observed values The probability of; Let t be the state preceding the optimal path to state j at time t. To select the maximum value among all possible states i, return i that maximizes the expression.

[0174] 3) Recursion:

[0175] ;

[0176] in, This is the optimal hidden state at time point T, which is a state in the optimal state sequence found through backtracking; To find at time T such that The maximum state i; The maximum probability of reaching state i at time point T is calculated through a recursive step. To reach the optimal state at time point t+1. The previous state; This indicates that the backtracking process starts from time point T-1 and gradually backtracks to time point 1.

[0177] Furthermore, the optimal hidden state sequence Q* is obtained by calculating using the PSO-based HMM model. That is, the target hidden state sequence.

[0178] For example, please refer to Figure 5 Since the mean and variance characteristics at different stages reflect the behavioral characteristics of the signal under different physical states, this invention uses the obtained health indicators as the observation sequence input into a Hidden Markov Model (HMM). A Particle Swarm Optimization (PSO) algorithm (including particle initialization and fitness calculation steps) is employed to find the optimal initial HMM parameters. These optimal parameters are then updated, and corresponding state information is extracted from the observation data. The Viterbi algorithm is used to calculate the most probable hidden state. Finally, through signal analysis, the state results are output, dividing the entire lifecycle of the rotating component into three stages: "normal operation state," "fault development state," and "fault deterioration and failure state." The process is as follows: Figure 5 As shown.

[0179] In this embodiment, based on a pre-optimized search algorithm, an initial hidden Markov model is used to generate a sequence of hidden states of the target based on the target health indicators.

[0180] Step 104: Using a pre-set nonlinear Wiener degradation model, generate a fault warning result for the rotating component based on the target hidden state sequence and multimodal signal data.

[0181] Specifically, step 104 may include the following sub-steps S41-S43:

[0182] Step S41: Divide the target hidden state sequence to generate a normal state sequence, a fault development state sequence, and a fault failure state sequence;

[0183] It should be noted that you should refer to [link / reference]. Figure 6 The obtained target hidden state sequence is divided into three stages, corresponding to three hidden states (i.e., normal state sequence, fault development state sequence, and fault failure state sequence). It is assumed that the observation probability follows a Gaussian distribution, and the hidden states are divided into normal, fault development, and fault failure states. The particle swarm optimization algorithm (PSO) and hidden Markov model (HMM) are used to achieve adaptive identification of the development stages of rotating components, enhancing automated processing capabilities, effectively filtering out irrelevant fault information such as noise, while ensuring the retention of key fault degradation information.

[0184] Step S42: Filter the multimodal signal data based on the fault occurrence points corresponding to the normal state sequence to determine the multimodal signal data before degradation;

[0185] It should be noted that, based on the normal state sequence, the fault development state sequence, and the fault failure state sequence, three time points can be obtained for the rotating component: the fault occurrence point, the fault deterioration point, and the complete failure point. After detecting the fault occurrence point of the rotating component, it is determined that the rotating component has begun to degrade. The healthy data before degradation is selected, that is, the acoustic emission sub-signal and the vibratory sub-signal of the time step before the fault occurrence point are selected from the multimodal signal data as the multimodal signal data before degradation.

[0186] Step S43: Using a pre-set nonlinear Wiener degradation model, early warning is generated based on the multimodal signal data before degradation, the fault deterioration point threshold corresponding to the fault development state sequence, and the complete failure point threshold corresponding to the fault failure state sequence, and the rotating component fault early warning result is generated.

[0187] It should be noted that the Wiener process is a type of stochastic process. Due to its clear physical interpretation and favorable mathematical properties, it is widely used in degradation modeling and remaining lifetime prediction. It has been verified that the increments of adjacent time steps of the fused signal satisfy the requirement of a normal distribution, N(0, 0.06), as shown... Figure 7 As shown, a nonlinear Wiener process can be used for prediction.

[0188] Furthermore, this invention employs a nonlinear Wiener process to characterize the stochastic degradation process of the rotating component. Based on the model assumptions, the performance value X(t) of the rotating component can be expressed as:

[0189] ;

[0190] in, This represents the performance state at time t; This represents the initial state of the system, which is assumed to be 0. The drift term describes the nonlinear changes in system performance degradation over time. The drift coefficient, It is a nonlinear function of time, where a is a constant, describing the rate of change with time; The diffusion term represents the random fluctuations in the system's state. The diffusion coefficient describes the range of fluctuations in the degradation process. This is standard Brownian motion.

[0191] Furthermore, in order to estimate the drift coefficient and diffusion coefficient The least squares method can be used. The specific steps are as follows:

[0192] First, perform the drift coefficient. Calculation:

[0193] ;

[0194] Where T is the trend vector; X is the actual degradation data; and Q is the covariance matrix.

[0195] Next, the diffusion coefficient is calculated. Calculation:

[0196] ;

[0197] Where N is the number of samples.

[0198] Furthermore, the remaining lifetime L is defined as the degradation process X(t) from the current state X(t0) = x i Initially, the time required to reach the fault threshold D is: X(t0+L)=D, x i This represents the current point in time of degradation.

[0199] The extended Wiener process model is as follows:

[0200] ;

[0201] in, In time Degradation rate at that time; For the present The degradation rate of the point; It is a process that changes randomly within a time period L.

[0202] At time step t=t0, the state of the degradation process is X(t0)=x iFrom this point onward, the remaining lifetime L satisfies:

[0203] ;

[0204] Will Normalization of the expression (divided by) ):

[0205] ;

[0206] Furthermore, Brownian motion Satisfies a normal distribution:

[0207] ;

[0208] Therefore, the standardized random variable is:

[0209] ;

[0210] Where Z is the standardized random variable; It is a standardized integral distribution.

[0211] Based on the above expression, we can obtain:

[0212] ;

[0213] In summary, the probability density function f for solving L (i.e., the pre-set nonlinear Wiener degradation model) can be obtained. L (l), specifically:

[0214] ;

[0215] in, This refers to the current time point; This represents the current drift coefficient; This represents the current diffusion coefficient; is the remaining value of the current degradation distance fault threshold; l is the time step of the remaining lifetime.

[0216] It is worth mentioning that the Wiener model (pre-set nonlinear Wiener degradation model) is trained using pre-degradation health data (pre-degradation multimodal signal data), and two thresholds are set to obtain the rotating component fault early warning results. The rotating component fault early warning results include the RUL distribution from the fault occurrence point to the fault deterioration point and the RUL distribution from the fault occurrence point to the complete failure point. Specifically, the threshold of the fault deterioration point D1 (i.e., the fault deterioration point threshold corresponding to the fault development state sequence) and the threshold D2 of the complete failure point (i.e., the complete failure point threshold corresponding to the fault failure state sequence) are used. Inputting the fault deterioration point D1 threshold into the above-mentioned RUL probability density function (i.e., the trained pre-set nonlinear Wiener degradation model) yields the RUL distribution from the fault occurrence point to the fault deterioration point of the rotating component, as shown below. Figure 8 As shown. By inputting the threshold D2 of the complete failure point into the aforementioned RUL probability density function, the RUL distribution (Remaining Useful Life Distribution) from the point of failure of the rotating component to the point of complete failure of the rotating component can be obtained, as shown. Figure 9 As shown.

[0217] Depend on Figures 8-9 It can be seen that the RUL distribution predicted by the nonlinear Wiener process model is close to the actual degradation time of the rotating component, which can achieve early warning.

[0218] For example, please refer to Figure 10 This invention derives two thresholds based on three defined failure time points: the failure occurrence point, the failure deterioration point, and the complete failure point: a threshold D1 for the failure deterioration point and a threshold D2 for the complete failure point. The model is trained using performance data prior to the failure deterioration point (i.e., multimodal signal data before degradation) to complete performance degradation modeling. Then, parameter estimation and remaining lifetime prediction are performed. Specifically, the threshold D1 for the failure deterioration point is input into a nonlinear Wiener degradation model to obtain the RUL probability density function for each time step, thus yielding the RUL distribution from the bearing failure occurrence point to the failure deterioration point. Similarly, the threshold D2 for the complete failure point is input into the degradation model to obtain the RUL distribution from the bearing failure occurrence point to the complete failure point. Based on the obtained RUL distribution, the time it takes for the bearing to return to its normal state to the two threshold points can be determined, enabling early warning of failures.

[0219] In this embodiment, a preset nonlinear Wiener degradation model is used to generate a fault warning result for the rotating component based on the target hidden state sequence and multimodal signal data.

[0220] For comparison of technical effectiveness, existing technologies can be used as a reference. Currently, although there are many methods for bearing condition monitoring, each has its limitations. Temperature change monitoring is unsuitable for online dynamic monitoring due to its lag; vibration signal monitoring, although widely used, is easily affected by multiple factors, significantly reducing the accuracy and reliability of early fault diagnosis; oil monitoring methods have limited sensitivity and require periodic sampling of lubricating oil, which also makes it difficult to meet the needs of online monitoring. In contrast, acoustic emission signal monitoring can keenly detect early bearing damage, but traditional methods, based on impact data analysis, struggle to accurately pinpoint the fault source and its condition.

[0221] With the rapid advancements in computer processing power, real-time processing of acoustic emission waveform signals has become possible. Consequently, numerous scholars have dedicated themselves to researching acoustic emission waveform signal processing in order to improve the accuracy of bearing life prediction. Currently, life prediction technologies are mainly divided into three categories: physical model-based, data-driven, and hybrid model-based. Among these, data-driven methods, with their full utilization of historical data, have achieved remarkable success in the field of fault prediction.

[0222] Data-driven methods can be further subdivided into machine learning-based data-driven approaches and deep learning-based big data training. While deep learning methods can achieve extremely high prediction accuracy with massive amounts of training data, acquiring full lifecycle signals in real-world production environments remains a significant challenge. Machine learning-based methods, on the other hand, excel under limited sample conditions. There are various machine learning-based prediction methods, such as LSTM networks based on time-series data and data-driven approaches based on statistics and stochastic processes. LSTM neural networks based on time-series data capture long-term dependencies, improving prediction accuracy, while Wiener process-based prediction methods can efficiently compute data distributions and predict failure times even with limited sample data.

[0223] Furthermore, prediction methods relying solely on vibration signals remain insufficient, failing to effectively capture early fault signals and providing a lack of intuitive reflection of damage severity. Acoustic emission sensors, with their keen ability to capture early signals, have become a rising star in fault prediction research. Combining the advantages of acoustic emission with vibration signals for early identification and prediction of bearing operating conditions holds promise for fault warning and effectively preventing significant losses caused by bearing failure.

[0224] Given the favorable trend of vibration signals in reflecting the entire lifespan of bearings, and the keen perception and damage characterization capabilities of acoustic emission signals for early faults, this invention proposes a multimodal data fusion-based fault early warning method for rotating components to address the aforementioned issues. This invention combines the advantages of LSTM networks and Wiener processes, and innovatively integrates acoustic emission sensor signals with vibration signals, achieving full lifespan monitoring and early warning for bearings. By fusing acoustic emission and vibration signals using an LSTM+attention mechanism, the monotonicity of the signals is improved while preserving the phased nature of acoustic emission and the trend of vibration. The fused health indicators are then used as an observation matrix and input into a particle swarm optimization (PSO)-based Hidden Markov Model (HMM). The optimal HMM parameters are found using the PSO algorithm, and the Viterbi algorithm is used to find fault threshold points at different stages. Finally, a nonlinear Wiener process is used to predict the arrival time of different stages based on the fault threshold points, enabling timely early warning. This innovative achievement provides solid technical support for the maintenance management and lifespan extension of industrial equipment.

[0225] Specifically, this invention first uses multimodal sensors to collect raw signals throughout the entire lifespan of the bearing (this invention uses acoustic emission sensors and vibration sensors). Next, the raw signals are preprocessed and cleaned, and time-domain and time-frequency domain features are extracted from different types of raw signals to form feature sets. Features of the acoustic emission and vibration signal feature sets are extracted using an LSTM time-series neural network method. Attention scores are calculated for the acoustic emission and vibration feature sets using an attention mechanism, and the feature signals from the two modes are fused based on these attention scores to construct a health index. The health feature index with the highest monotonicity score is used as the observation matrix and input into a Hidden Markov Model (HMM). The bearing's entire lifespan is divided into three states: normal, fault development, and fault deterioration / failure, resulting in three time points: the fault occurrence point, the fault deterioration point, and the complete failure point. Finally, two thresholds are set: a threshold D1 for the fault deterioration point and a threshold D2 for the complete failure point. The fault deterioration point threshold D1 is input into a nonlinear Wiener degradation model to obtain the RUL probability density function for each time step, thus yielding the RUL distribution from the bearing fault occurrence point to the bearing fault deterioration point. Similarly, by inputting the threshold D2 of the complete failure point into the degradation model, the RUL distribution of the bearing failure point to the complete failure point can be obtained. Based on the obtained RUL distribution, the time it takes for the bearing to go from the normal state to the two threshold points can be known, thus realizing early warning of the failure.

[0226] In this embodiment of the invention, a method for early warning of rotating component faults based on multimodal data fusion is provided. First, multimodal signal data is acquired and preprocessed using a bandpass filtering algorithm to generate a multimodal feature vector set. Next, a recurrent neural network based on an attention mechanism is used to fuse the multimodal feature vector set, outputting a target health indicator. Based on a pre-set optimization search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence based on the target health indicator. Finally, a pre-set nonlinear Wiener degradation model is used to generate a rotating component fault warning result based on the target hidden state sequence and the multimodal signal data. Based on the above scheme, the process of generating a target health indicator using a recurrent neural network based on an attention mechanism from the preprocessed and fused multimodal feature vector set, and then processing the target health indicator using a pre-set optimization search algorithm with an initial hidden Markov model and a pre-set nonlinear Wiener degradation model to generate a rotating component fault warning result, demonstrates that this invention utilizes a recurrent neural network based on an attention mechanism to adaptively fuse multimodal data features, achieving multimodal information complementarity, thereby obtaining richer fault information and improving the early warning effect.

[0227] For better explanation, refer to Figure 11 This diagram illustrates the overall framework of the rotating component fault early warning method based on multimodal data fusion provided in Embodiment 2 of the present invention. It should be noted that this embodiment only provides a brief description of the general flow of the rotating component fault early warning method based on multimodal data fusion. The specific implementation process of each step can be understood by referring to the relevant content in the foregoing embodiments, and will not be elaborated here. It is understood that the present invention does not impose any limitations on this.

[0228] The overall framework of the multimodal data fusion-based rotating component fault early warning method proposed in this invention can be divided into three parts: a feature extraction module, a data processing module, and a fault early warning module. In the feature extraction module, vibration and acoustic emission sensors are preprocessed, extracting 17 feature values ​​for each. In the data processing module, data cleaning and fusion are performed, using LSTM+AT (LSTM+Attention Temporal Neural Network) to fuse acoustic emission and vibration signals. This method cleverly utilizes the distinct phases of acoustic emission signals and the strong trends in vibration signals to achieve information complementarity between acoustic emission and vibration signals, thus predicting the fault occurrence time in advance. In the fault early warning module, a hidden Markov model based on particle swarm optimization is used to divide the signal into three states: normal state, fault development state, and fault deterioration and failure state. Three time points are obtained: fault occurrence point, fault deterioration point, and complete failure point. Then, a nonlinear Wiener process degradation model is established, trained using data before the fault occurrence point, and two thresholds are set: the threshold D1 for the fault deterioration point and the threshold D2 for the complete failure point. The remaining lifetime (RUL) distribution of the thresholds corresponding to the fault deterioration point D1 and the complete failure point D2 is predicted using a nonlinear Wiener process degradation model, so as to obtain the remaining lifetime of the fault deterioration point and the complete failure point and achieve fault early warning.

[0229] In this embodiment of the invention, an LSTM temporal neural network with attention mechanism is used to adaptively fuse multimodal data features, achieving complementary multimodal information and thus obtaining richer fault information, overcoming the limitations of insufficient information from a single sensor. Furthermore, the fused data can provide phased early warning, effectively reducing the risks associated with rotating component failure.

[0230] Please see Figure 12 , Figure 12 This is a structural block diagram of a rotating component fault early warning device based on multimodal data fusion, provided in Embodiment 3 of the present invention.

[0231] This invention provides a multimodal data fusion-based rotating component fault early warning device, comprising:

[0232] The acquisition module 1201 is used to acquire multimodal signal data and generate a multimodal feature vector set based on the multimodal signal data using a bandpass filtering algorithm;

[0233] The feature fusion module 1202 is used to perform feature fusion on the multimodal feature vector set using a recurrent neural network based on an attention mechanism, and output the target health index.

[0234] The output sequence module 1203 is used to generate the target hidden state sequence based on the target health index using an initial hidden Markov model and a preset optimized search algorithm.

[0235] The output result module 1204 is used to generate a fault warning result for the rotating component based on the target hidden state sequence and multimodal signal data using a preset nonlinear Wiener degradation model.

[0236] Furthermore, the multimodal signal data includes initial acoustic emission signal data and initial vibration signal data; the multimodal feature vector set includes a vibration feature vector set and an acoustic emission feature vector set; the acquisition module 1201 is specifically used for:

[0237] Data cleaning was performed on the initial acoustic emission signal data and the initial vibration signal data to generate intermediate acoustic emission signal data and intermediate vibration signal data.

[0238] Time alignment is performed on intermediate acoustic emission signal data and intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data;

[0239] Based on the bandpass filtering algorithm, high-frequency signals are extracted from the acoustic emission sub-signals of multiple time steps in the target acoustic emission signal data, and multiple high-frequency acoustic emission signals are output.

[0240] Based on the bandpass filtering algorithm, low-frequency signals are extracted from multiple time-step vibration sub-signals in the target vibration signal data, and multiple low-frequency vibration signals are output.

[0241] Feature extraction is performed on multiple high-frequency acoustic emission signals and multiple low-frequency vibration signals to generate vibration feature vector sets and acoustic emission feature vector sets, respectively.

[0242] Furthermore, the attention-based recurrent neural network includes a recurrent neural network and an attention mechanism network; the feature fusion module 1202 includes:

[0243] The first submodule is used to normalize the vibration feature vector set and the acoustic emission feature vector set respectively, and output the normalized vibration feature vector set and the normalized acoustic emission feature vector set;

[0244] The second submodule is used to perform time alignment on the normalized vibration feature vector set and the normalized acoustic emission feature vector set, and output the target vibration feature vector set and the target acoustic emission feature vector set.

[0245] The third submodule is used to take the target vibration feature vector set and the target acoustic emission feature vector set as inputs to the recurrent neural network, and output the vibration hidden state vector corresponding to the target vibration feature vector set and the acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set, respectively.

[0246] The fourth submodule is used to generate the vibration context vector corresponding to the vibration hidden state vector and the acoustic emission context vector corresponding to the acoustic emission hidden state vector using an attention mechanism network, respectively, based on the vibration hidden state vector and the acoustic emission hidden state vector.

[0247] The fifth submodule is used to concatenate the vibration context vector and the acoustic emission context vector to generate a concatenated vector;

[0248] The sixth submodule is used to map the concatenated vectors and output the target health indicators.

[0249] Furthermore, the fourth submodule is specifically used for:

[0250] Linear transformations are performed on the vibration latent state vector and the acoustic emission latent state vector respectively to generate multiple vibration attention scores and multiple acoustic emission attention scores.

[0251] Nonlinear mapping is performed on multiple vibration attention scores and multiple acoustic emission attention scores respectively to output multiple time-step vibration attention weights and multiple time-step acoustic emission attention weights;

[0252] A vibration context vector is generated by weighting the vibration attention weights and vibration features at multiple time steps in the vibration latent state vector.

[0253] The acoustic emission attention weights at multiple time steps and the acoustic emission features at multiple time steps in the acoustic emission latent state vector are weighted to generate an acoustic emission context vector.

[0254] Furthermore, the pre-defined optimization search algorithms include particle swarm optimization and Viterbi algorithm; the output sequence module 1203 is specifically used for:

[0255] The particle swarm optimization algorithm is used to train the initial hidden Markov model and determine the target hidden Markov model.

[0256] The target health indicators are used as observation sequences and input into the target hidden Markov model to output the probability of the target observation sequence.

[0257] The Viterbi algorithm is used to calculate the target hidden state sequence based on the probability of the target observation sequence.

[0258] Furthermore, the output module 1204 is specifically used for:

[0259] The target hidden state sequence is divided into normal state sequence, fault development state sequence, and fault failure state sequence;

[0260] Multimodal signal data is filtered based on the fault occurrence points corresponding to the normal state sequence to determine the multimodal signal data before degradation.

[0261] A pre-set nonlinear Wiener degradation model is used to generate early warning results for rotating components based on multimodal signal data before degradation, fault deterioration point thresholds corresponding to fault development state sequences, and complete failure point thresholds corresponding to fault failure state sequences.

[0262] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0263] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the multimodal data fusion rotating component fault early warning method as described in any of the above embodiments.

[0264] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the multimodal data fusion rotating component fault early warning method as described in any of the above embodiments.

[0265] This invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a rotating component fault early warning method based on multimodal data fusion as described in any of the above embodiments.

[0266] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for early warning of faults in rotating components based on multimodal data fusion, characterized in that, include: Multimodal signal data is acquired, and a bandpass filtering algorithm is used to preprocess the multimodal signal data to generate a multimodal feature vector set; A recurrent neural network based on an attention mechanism is used to fuse the multimodal feature vector set and output the target health index. Based on a pre-optimized search algorithm, an initial hidden Markov model is used to generate a target hidden state sequence according to the target health indicators; A pre-set nonlinear Wiener degradation model is used to generate a fault warning result for the rotating component based on the target hidden state sequence and the multimodal signal data; The multimodal signal data includes initial acoustic emission signal data and initial vibration signal data; the multimodal feature vector set includes a vibration feature vector set and an acoustic emission feature vector set. The preprocessing of the multimodal signal data using a bandpass filtering algorithm to generate a multimodal feature vector set includes: Data cleaning was performed on the initial acoustic emission signal data and the initial vibration signal data to generate intermediate acoustic emission signal data and intermediate vibration signal data. Time alignment is performed on the intermediate acoustic emission signal data and the intermediate vibration signal data to generate target acoustic emission signal data and target vibration signal data; Based on the bandpass filtering algorithm, high-frequency signals are extracted from multiple time-step acoustic emission sub-signals in the target acoustic emission signal data, and multiple high-frequency acoustic emission signals are output. Based on the bandpass filtering algorithm, low-frequency signals are extracted from multiple time-step vibration sub-signals in the target vibration signal data, and multiple low-frequency vibration signals are output. Feature extraction is performed on multiple acoustic emission high-frequency signals and multiple vibration low-frequency signals respectively to generate a vibration feature vector set and an acoustic emission feature vector set; The pre-defined optimized search algorithm includes particle swarm optimization and Viterbi algorithm; the step of generating the target hidden state sequence based on the pre-defined optimized search algorithm and an initial hidden Markov model according to the target health index includes: The particle swarm optimization algorithm is used to train the initial hidden Markov model and determine the target hidden Markov model. The target health index is used as an observation sequence and input into the target hidden Markov model to output the probability of the target observation sequence. The Viterbi algorithm is used to calculate the target hidden state sequence based on the probability of the target observation sequence.

2. The method for early warning of rotating component faults based on multimodal data fusion according to claim 1, characterized in that, The attention-based recurrent neural network includes a recurrent neural network and an attention mechanism network; The step of using an attention-based recurrent neural network to fuse features from the multimodal feature vector set and output a target health indicator includes: The vibration feature vector set and the acoustic emission feature vector set are normalized respectively, and the normalized vibration feature vector set and the normalized acoustic emission feature vector set are output. Time alignment is performed on the normalized vibration feature vector set and the normalized acoustic emission feature vector set to output the target vibration feature vector set and the target acoustic emission feature vector set; The target vibration feature vector set and the target acoustic emission feature vector set are respectively used as inputs to a recurrent neural network, and the vibration hidden state vector corresponding to the target vibration feature vector set and the acoustic emission hidden state vector corresponding to the target acoustic emission feature vector set are output. An attention mechanism network is used to generate a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector, respectively, based on the vibration hidden state vector and the acoustic emission hidden state vector. The vibration context vector and the acoustic emission context vector are concatenated to generate a concatenated vector; The concatenated vector is mapped to output the target health index.

3. The method for early warning of rotating component faults based on multimodal data fusion according to claim 2, characterized in that, The attention mechanism network generates a vibration context vector corresponding to the vibration hidden state vector and an acoustic emission context vector corresponding to the acoustic emission hidden state vector based on the vibration hidden state vector and the acoustic emission hidden state vector, respectively, including: The vibration latent state vector and the acoustic emission latent state vector are linearly transformed respectively to generate multiple vibration attention scores and multiple acoustic emission attention scores. A nonlinear mapping is performed on the multiple vibration attention scores and multiple acoustic emission attention scores respectively to output multiple time-step vibration attention weights and multiple time-step acoustic emission attention weights; A vibration context vector is generated by weighting the vibration attention weights of multiple time steps and the vibration features of multiple time steps in the vibration latent state vector. The acoustic emission attention weights at multiple time steps and the acoustic emission features at multiple time steps in the acoustic emission latent state vector are weighted to generate an acoustic emission context vector.

4. The method for early warning of rotating component faults based on multimodal data fusion according to claim 1, characterized in that, The step of generating a fault warning result for the rotating component using a pre-set nonlinear Wiener degradation model based on the target hidden state sequence and the multimodal signal data includes: The target hidden state sequence is divided into normal state sequence, fault development state sequence, and fault failure state sequence. The multimodal signal data is filtered based on the fault occurrence points corresponding to the normal state sequence to determine the multimodal signal data before degradation. The pre-set nonlinear Wiener degradation model is used to generate early warning results for rotating components based on the pre-degradation multimodal signal data, the fault deterioration point threshold corresponding to the fault development state sequence, and the complete failure point threshold corresponding to the fault failure state sequence.

5. A multimodal data fusion-based rotating component fault early warning device, applied to the multimodal data fusion-based rotating component fault early warning method of claim 1, characterized in that, include: The acquisition module is used to acquire multimodal signal data and generate a multimodal feature vector set based on the multimodal signal data using a bandpass filtering algorithm; The feature fusion module is used to perform feature fusion on the multimodal feature vector set using a recurrent neural network based on an attention mechanism, and output the target health index. The output sequence module is used to generate a target hidden state sequence based on the target health indicators using an initial hidden Markov model and a pre-set optimized search algorithm. The output results module is used to generate a fault warning result for the rotating component based on the target hidden state sequence and the multimodal signal data using a preset nonlinear Wiener degradation model.

6. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the rotating component fault early warning method based on multimodal data fusion as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the rotating component fault early warning method based on multimodal data fusion as described in any one of claims 1-4.

8. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein when the program instructions are executed by a computer, the computer performs the multimodal data fusion method for early warning of rotating component faults as described in any one of claims 1-4.

Images