A mechanical equipment fault diagnosis method based on multi-modal deep clustering

Abstract

The application discloses a kind of mechanical equipment fault diagnosis methods based on multi-modal depth clustering, its steps include:1.collection mechanical equipment operating state sensor signal data, time-frequency image data is generated by wavelet packet transform, and signal data and time-frequency image data are preprocessed, and data sample set is constructed;2.build multi-modal unsupervised network, network structure includes convolutional autoencoder module, gated recurrent unit autoencoder module, multi-modal fusion module and clustering module;3.train multi-modal unsupervised network, and optimize network parameter;4.utilize the multi-modal unsupervised network model of well-trained mechanical equipment is diagnosed to fault.The application can extract the depth feature of different modal data from unlabelled multi-modal data and realize the adaptive fusion of multi-modal feature, so as to improve the effect of mechanical equipment fault diagnosis.

Description

Technical Field

[0001] This method relates to the field of mechanical equipment fault diagnosis, specifically a mechanical equipment fault diagnosis method based on multimodal deep clustering. Background Technology

[0002] As the complexity of mechanical equipment continues to increase, the requirements for its reliability and safety also rise significantly. In industrial production, mechanical equipment inevitably malfunctions, and failure to diagnose the type of malfunction in a timely manner can lead to substantial economic losses and serious safety accidents. Therefore, effective fault diagnosis of mechanical equipment is crucial for ensuring its safe operation.

[0003] Currently, fault diagnosis methods for mechanical equipment mainly include manual diagnosis methods and data-driven intelligent diagnosis methods. Manual diagnosis methods refer to fault diagnosis performed by diagnostic personnel using domain knowledge and experience through manual interpretation. However, this method is easily affected by environmental factors and human subjectivity, resulting in low efficiency and accuracy. Compared to manual diagnosis methods, data-driven intelligent diagnosis methods can directly construct key fault features from sensor data and perform diagnosis through fault mode recognition methods, offering significant advantages in efficiency and accuracy. Data-driven intelligent diagnosis methods can be further divided into traditional methods and deep learning methods. Compared to the former, the latter can automatically extract deep features from large amounts of sensor data, reducing reliance on signal processing methods and expert experience, and better capturing fault characteristic information.

[0004] However, existing deep learning-based fault diagnosis methods still have shortcomings in the fault diagnosis of mechanical equipment:

[0005] (1) On the one hand, existing deep learning-based fault diagnosis methods require a large amount of sensor data with fault labels during model training. However, in reality, there are problems such as unstable operating conditions and high labor costs, making the collection of sufficient labeled data very expensive. Therefore, it is necessary to study how to make full use of a large amount of unlabeled sensor data for fault diagnosis.

[0006] (2) On the other hand, existing deep learning-based fault diagnosis methods only extract features and identify fault types from single-modal data such as one-dimensional signals or images, and rarely consider the complementary information between different modal data. Therefore, using different modal data for fault diagnosis has important research value. Summary of the Invention

[0007] To address the aforementioned problems in existing technologies, this invention provides a mechanical equipment fault diagnosis method based on multimodal deep clustering. This method aims to extract deep features of different modal data from unlabeled multimodal data and achieve adaptive fusion of multimodal features, thereby improving the effectiveness of mechanical equipment fault diagnosis.

[0008] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0009] The mechanical equipment fault diagnosis method based on multimodal deep clustering of this invention is characterized by the following steps:

[0010] Step 1, construct the data sample set:

[0011] V sensors are used to collect raw signal data of the mechanical equipment's operating status. This raw signal data is then converted into time-frequency image data using wavelet packet transform. Both the raw signal data and the time-frequency image data are then normalized. The normalized signal data and time-frequency image data are then divided into samples to construct a sample set. and in, and Let y represent the signal data of the i-th sample and its corresponding time-frequency image data of the i-th sample, respectively. i Let N represent the fault label of the i-th sample, N represent the number of samples, V represent the number of sensors, m represent the feature dimension of each sensor signal data, and h×h represent the resolution of each time-frequency image data.

[0012] Step 2: Construct a multimodal unsupervised network, including: a gated recurrent unit autoencoder module, a convolutional autoencoder module, a multimodal fusion module, and a clustering module.

[0013] Step 2.1: Initialize the parameters in the multimodal unsupervised network;

[0014] Step 2.2: According to the correspondence between signal samples and time-frequency image samples, the signal data and its corresponding time-frequency image data are sequentially input into the gated recursive unit autoencoder module and the convolutional autoencoder module for feature extraction, and the corresponding depth representation features are output. and Therefore, the signal data of the i-th sample can be constructed using equation (3). With reconstructed signal data Reconstruction loss function between

[0015]

[0016] In equation (3), ||·|| FDenotes the F-norm of a matrix;

[0017] Construct the i-th time-frequency image data using equation (6) With reconstructed time-frequency image data Reconstruction loss function between

[0018]

[0019] Step 2.3: Input the depth representation features extracted by the gated recurrent unit autoencoder module and the convolutional autoencoder module into the multimodal fusion module to obtain multimodal depth feature fusion.

[0020] Step 2.4: Input the multimodal fusion features of all samples into the clustering module for clustering:

[0021] Step 2.4.1: The clustering module initializes the cluster centroids of the multimodal fusion features of N samples using the k-means clustering algorithm. μ j Let K represent the centroid of the j-th cluster, and K represent the number of cluster centroids.

[0022] Step 2.4.2, the clustering module uses equation (10) to calculate the soft-assignment distribution. in, Let represent the probability that the multimodal fusion feature of the i-th sample belongs to the j-th cluster:

[0023]

[0024] Step 2.4.3, the clustering module uses equation (11) to calculate the auxiliary target distribution. in, The probability that the multimodal fusion feature of the i-th sample belongs to the target of the j-th cluster:

[0025]

[0026] Step 2.4.4, the clustering module uses equation (12) to calculate the soft-assignment distribution Q. F and auxiliary target distribution P F The KL divergence between them is used as the clustering loss function L. c :

[0027]

[0028] Step 2.4.5, the clustering module uses equation (13) to calculate the clustering result of the i-th sample. And used as the predicted fault label for the i-th sample:

[0029]

[0030] In equation (13), argmax represents the index value corresponding to the maximum value of the output function;

[0031] Step 3: Train the multimodal unsupervised network and optimize the network parameters:

[0032] Step 3.1, use equation (14) to calculate the total loss function L of the multimodal unsupervised network. Total :

[0033]

[0034] In equation (14), α, β, and γ represent three hyperparameters that balance the effects of different loss functions;

[0035] Step 3.2, to minimize the total loss function L Total With the goal of optimizing and updating the parameters of the multimodal unsupervised network through backpropagation and adaptive moment estimation optimization algorithms, the training ends when the number of iterations reaches the set upper limit, thus obtaining a trained multimodal unsupervised network model for fault diagnosis of mechanical equipment.

[0036] The mechanical equipment fault diagnosis method based on multimodal deep clustering described in this invention is also characterized in that step 2.2 includes the following steps:

[0037] Step 2.2.1, the gated recursive unit autoencoder module uses equation (1) to process the signal data of the i-th sample. Encoding is performed to obtain the signal data of the i-th sample. Depth representation features in, This represents the signal data of the i-th sample. The depth representation feature at time step t, where d represents the feature dimension of each channel in the depth representation feature, and T s This indicates the number of time steps divided when inputting signal data into the gated recursive unit self-encoder:

[0038]

[0039] In equation (1), This represents the weight parameters of the encoding layer in the gated recursive unit autoencoder module. This represents the bias vector of the encoding layer in the gated recursive unit autoencoder module. This represents the activation function of the encoding layer in the gated recursive unit autoencoder module;

[0040] Step 2.2.2, the gated recursive unit autoencoder module uses equation (2) to process the depth representation feature Z. i S Perform decoding to obtain the reconstructed signal data.

[0041]

[0042] In equation (2), This represents the weight parameters of the decoding layer in the gated recursive unit autoencoder module. This represents the bias vector of the decoding layer in the gated recursive unit autoencoder module. This represents the activation function of the decoding layer in the gated recursive unit autoencoder module;

[0043] Step 2.2.3, the convolutional autoencoder module uses equation (4) to process the i-th time-frequency image data. Encoding operations are performed to obtain the i-th time-frequency image data. Depth representation features in, Represents the i-th time-frequency image data The depth representation feature of the v-th channel:

[0044]

[0045] In equation (4), This represents the weight parameters of the encoding layer in the convolutional autoencoder module. This represents the bias vector of the coding layer in the convolutional autoencoder module. This represents the activation function of the coding layer in the convolutional autoencoder module;

[0046] Step 2.2.4, the convolutional autoencoder module uses equation (5) to represent the depth features. Decoding is performed to obtain the reconstructed time-frequency image data.

[0047]

[0048] In equation (5), This represents the weight parameters of the decoding layer in the convolutional autoencoder module. This represents the bias vector of the decoding layer in the convolutional autoencoder module. This represents the activation function of the decoding layer in the convolutional autoencoder module.

[0049] Step 2.3 includes the following steps:

[0050] Step 2.3.1, the multimodal fusion module uses equation (7) to obtain the similarity matrix between the two modal depth features.

[0051]

[0052] In equation (7), represents the parameter matrix of the two modes in the multimodal fusion module, tanh(·) represents the hyperbolic tangent activation function, and T represents the transpose;

[0053] Step 2.3.2, the multimodal fusion module uses equation (8) to obtain the attention weights of the two modalities.

[0054]

[0055] In equation (8), This represents the parameter vectors of two modes in the multimodal fusion module. These represent the signal data of the i-th sample, respectively. Depth representation features Attention weights Representing the i-th time-frequency image data respectively Depth representation features The attention weights are denoted by softmax(·), which represents the normalized exponential activation function.

[0056] Step 2.3.3: The multimodal fusion module uses equation (9) to perform a weighted summation of the depth features of different modalities to obtain the multimodal fusion features.

[0057]

[0058] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the mechanical equipment fault diagnosis method, and the processor is configured to execute the program stored in the memory.

[0059] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, performs the steps of the mechanical equipment fault diagnosis method.

[0060] Compared with existing technologies, the beneficial effects of this invention are reflected in:

[0061] 1. This invention proposes an unsupervised method for diagnosing mechanical equipment faults. It utilizes a deep clustering method based on multimodal fusion to diagnose mechanical equipment faults, overcoming the problem of insufficient labeled data in real industrial production. This effectively achieves mechanical equipment fault diagnosis with lower diagnostic costs and has greater practical application value.

[0062] 2. This invention utilizes gated recursive unit autoencoders and convolutional autoencoders to extract the depth representation features of signal data and time-frequency image data, respectively. This overcomes the problem that existing mechanical equipment fault diagnosis methods only use single-mode data and ignore the complementarity between different modes of data, thereby extracting fault feature information more comprehensively and improving the accuracy of mechanical equipment fault diagnosis.

[0063] 3. This invention introduces an attention mechanism for multimodal fusion, which can adaptively adjust the contribution of different modalities, thereby better fusing the deep representation features of different modalities and improving the performance of mechanical equipment fault diagnosis. Attached Figure Description

[0064] Figure 1 This is a schematic flowchart of the method of the present invention;

[0065] Figure 2 This is a structural diagram of the multimodal unsupervised network model of the present invention. Detailed Implementation

[0066] In this embodiment, as Figure 1 As shown, the process of a mechanical equipment fault diagnosis method based on multimodal deep clustering includes the following steps:

[0067] Step 1, construct the data sample set:

[0068] V sensors are used to collect raw signal data of the mechanical equipment's operating status. This raw signal data is then converted into time-frequency image data using wavelet packet transform. Both the raw signal data and the time-frequency image data are then normalized. The normalized signal data and time-frequency image data are then divided into samples to construct a sample set. and in, and Let y represent the signal data of the i-th sample and its corresponding time-frequency image data of the i-th sample, respectively. i Let N represent the fault label of the i-th sample, N represent the number of samples, V represent the number of sensors, m represent the feature dimension of each sensor signal data, and h×h represent the resolution of each time-frequency image data.

[0069] Step 2: Construct a multimodal unsupervised network, including: a gated recurrent unit autoencoder module, a convolutional autoencoder module, a multimodal fusion module, and a clustering module.

[0070] Step 2.1: Initialize the parameters in the multimodal unsupervised network;

[0071] Step 2.2: According to the correspondence between signal samples and time-frequency image samples, the signal data and its corresponding time-frequency image data are sequentially input into the gated recursive unit autoencoder module and the convolutional autoencoder module for feature extraction, and the corresponding depth representation features are output.

[0072] Step 2.2.1, the gated recursive unit autoencoder module uses equation (1) to process the signal data of the i-th sample. Encoding is performed to obtain the signal data of the i-th sample. Depth representation features in, This represents the signal data of the i-th sample. The depth representation feature at time step t, where d represents the feature dimension of each channel in the depth representation feature, and T s This indicates the number of time steps divided when inputting signal data into the gated recursive unit self-encoder:

[0073]

[0074] In equation (1), This represents the weight parameters of the encoding layer in the gated recursive unit autoencoder module. This represents the bias vector of the encoding layer in the gated recursive unit autoencoder module. This represents the activation function of the encoding layer in the gated recursive unit autoencoder module;

[0075] like Figure 2 As shown, the encoding layer of the gated recursive unit autoencoder consists of bidirectional gated recursive units. These units extract forward and backward information from the time slice data. In the last gated recursive unit, the forward hidden state vector of the i-th sample data at time step t is... and backward hidden vector After concatenation, the deep temporal features at time step t are obtained. and The hidden layer features at all time steps are obtained by concatenating the forward and backward hidden vectors of all time steps, denoted as . That is, the depth representation features extracted by the gated recursive unit autoencoder are obtained. Without loss of generality, omitting the forward sign, the calculation process of the forward hidden state vector at time step t is as follows:

[0076]

[0077] In equation (2), R represents the update gate for the i-th sample at time step t. i (t) This represents the reset gate for the i-th sample at time step t. This represents the reset state of the i-th sample at time step t. Let represent the hidden state of the i-th sample after being updated at time step t. represents the input of the i-th sample at time step t, and the other variables are the parameters of the i-th sample in the bidirectional gated recursive unit. tanh(·) represents the hyperbolic tangent activation function, and σ(·) represents the Sigmoid activation function.

[0078] Step 2.2.2, the gated recursive unit autoencoder module uses equation (3) to represent the depth features. Perform decoding to obtain the reconstructed signal data.

[0079]

[0080] In equation (3), This represents the weight parameters of the decoding layer in the gated recursive unit autoencoder module. This represents the bias vector of the decoding layer in the gated recursive unit autoencoder module. This represents the activation function of the decoding layer in the gated recursive unit autoencoder module;

[0081] Step 2.2.3: The gated recursive unit autoencoder module uses equation (4) to construct the signal data of the i-th sample. With reconstructed signal data Reconstruction loss function between

[0082]

[0083] In equation (4), ||·|| F Denotes the F-norm of a matrix;

[0084] Step 2.2.4, the convolutional autoencoder module uses equation (5) to process the i-th time-frequency image data. Encoding operations are performed to obtain the i-th time-frequency image data. Depth representation features in, Represents the i-th time-frequency image data The depth representation feature of the v-th channel:

[0085]

[0086] In equation (5), This represents the weight parameters of the encoding layer in the convolutional autoencoder module. This represents the bias vector of the coding layer in the convolutional autoencoder module. This represents the activation function of the coding layer in the convolutional autoencoder module;

[0087] like Figure 2 As shown, the encoding layer of the convolutional autoencoder consists of three convolutional layers and one fully connected layer. Each convolutional layer includes one 2D convolution operation, one activation operation, and one max pooling operation. The third convolutional layer is followed by a fully connected layer. The encoding layer extracts spatial information from the image data and outputs a feature vector in the last fully connected layer. And l = d × V, after reshaping, the depth representation features are obtained. The calculation process of the encoding layer of a convolutional autoencoder is as follows:

[0088] d i =Fc(Flat(Conv1(Conv2(Conv3(X i I (6)

[0089] In equation (6), Conv1(·), Conv2(·), and Conv3(·) represent convolutional layer operations, Flat(·) represents flattening operations, and Fc(·) represents fully connected layer operations.

[0090] Step 2.2.5, the convolutional autoencoder module uses equation (7) to represent the depth features. Decoding is performed to obtain the reconstructed time-frequency image data.

[0091]

[0092] In equation (7), This represents the weight parameters of the decoding layer in the convolutional autoencoder module. This represents the bias vector of the decoding layer in the convolutional autoencoder module. This represents the activation function of the decoding layer in the convolutional autoencoder module;

[0093] like Figure 2 As shown, the decoding layer of the convolutional autoencoder consists of one fully connected layer and three transposed convolutional layers. Each transposed convolutional layer contains one two-dimensional transposed convolution operation and one activation operation. The decoding layer reconstructs the image data, converting the feature vector d... iThe output vector obtained after the fully connected layer is reshaped and then input into three transposed convolutional layers. The reconstructed image data is output from the last transposed convolutional layer. The calculation process of the decoding layer of the convolutional autoencoder is as follows:

[0094]

[0095] In equation (8), TransConv1(·), TransConv2(·), and TransConv3(·) represent transpose convolutional layer operations, and Reshape(·) represents reshaping operations;

[0096] Step 2.2.6: The convolutional autoencoder module uses equation (9) to construct the i-th time-frequency image data. With reconstructed time-frequency image data Reconstruction loss function between

[0097]

[0098] Step 2.3: Input the depth representation features extracted by the gated recurrent unit autoencoder module and the convolutional autoencoder module into the multimodal fusion module to achieve multimodal depth feature fusion.

[0099] Step 2.3.1: The multimodal fusion module uses equation (10) to obtain the similarity matrix between the two modal depth features.

[0100]

[0101] In equation (10), represents the parameter matrix of the two modes in the multimodal fusion module, tanh(·) represents the hyperbolic tangent activation function, and T represents the transpose;

[0102] Step 2.3.2: The multimodal fusion module uses equation (11) to obtain the attention weights of the two modalities.

[0103]

[0104] In equation (11), This represents the parameter vectors of two modes in the multimodal fusion module. These represent the signal data of the i-th sample, respectively. Depth representation features Attention weights Representing the i-th time-frequency image data respectively Depth representation features The attention weights are denoted by softmax(·), which represents the normalized exponential activation function.

[0105] Step 2.3.3: The multimodal fusion module uses equation (12) to perform a weighted summation of the depth features of different modalities to obtain the multimodal fusion features.

[0106]

[0107] Step 2.4: Input the multimodal fusion features of all samples into the clustering module for clustering:

[0108] Step 2.4.1: The clustering module initializes the cluster centroids of the multimodal fusion features of N samples using the k-means clustering algorithm. μ j Let K represent the centroid of the j-th cluster, and K represent the number of cluster centroids.

[0109] Step 2.4.2, the clustering module uses equation (13) to calculate the soft-assignment distribution. in, Let represent the probability that the multimodal fusion feature of the i-th sample belongs to the j-th cluster:

[0110]

[0111] Step 2.4.3: The clustering module uses equation (14) to calculate the auxiliary target distribution. in, The probability that the multimodal fusion feature of the i-th sample belongs to the target of the j-th cluster:

[0112]

[0113] Step 2.4.4: The clustering module uses equation (15) to calculate the soft-assignment distribution Q. F and auxiliary target distribution P F The KL divergence between them is used as the clustering loss function P. F :

[0114]

[0115] Step 2.4.5: The clustering module uses equation (16) to calculate the clustering result of the i-th sample. And used as the predicted fault label for the i-th sample:

[0116]

[0117] In equation (16), argmax represents the index value corresponding to the maximum value of the output function;

[0118] Step 3: Train the multimodal unsupervised network and optimize the network parameters:

[0119] Step 3.1, use equation (17) to calculate the total loss function L of the multimodal unsupervised network. Total :

[0120]

[0121] In equation (14), α, β, and γ represent three hyperparameters that balance the effects of different loss functions;

[0122] Step 3.2, to minimize the total loss function L Total With the goal of optimizing and updating the parameters of the multimodal unsupervised network through backpropagation and adaptive moment estimation optimization algorithms, the training ends when the number of iterations reaches the set upper limit, thus obtaining a trained multimodal unsupervised network model for fault diagnosis of mechanical equipment.

[0123] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0124] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

Claims

1. A method for fault diagnosis of mechanical equipment based on multimodal deep clustering, characterized in that, Includes the following steps: Step 1, construct the data sample set: use Each sensor collects raw signal data of the mechanical equipment's operating status. This raw signal data is then converted into time-frequency image data using wavelet packet transform. Both the raw signal data and the time-frequency image data are then normalized. The normalized signal data and time-frequency image data are then divided into samples to construct a sample set. and ,in, and They represent the first The signal data of the nth sample and its corresponding nth sample One time-frequency image data, Indicates the first Fault labels for each sample Indicates the number of samples. Indicates the number of sensors. This represents the characteristic dimension of each sensor signal data. This indicates the resolution of each time-frequency image data point; Step 2: Construct a multimodal unsupervised network, including: a gated recurrent unit autoencoder module, a convolutional autoencoder module, a multimodal fusion module, and a clustering module. Step 2.1: Initialize the parameters in the multimodal unsupervised network; Step 2.2: According to the correspondence between signal samples and time-frequency image samples, the signal data and its corresponding time-frequency image data are sequentially input into the gated recursive unit autoencoder module and the convolutional autoencoder module for feature extraction, and the corresponding depth representation features are output. and Thus, the first equation (3) is used to construct the second equation. Signal data of one sample With reconstructed signal data Reconstruction loss function between : (3) In equation (3), Denotes the F-norm of a matrix; Construct the first using equation (6) Time-frequency image data With reconstructed time-frequency image data Reconstruction loss function between : (6) Step 2.3: Input the depth representation features extracted by the gated recurrent unit autoencoder module and the convolutional autoencoder module into the multimodal fusion module to obtain multimodal depth feature fusion. ; Step 2.4: Input the multimodal fusion features of all samples into the clustering module for clustering: Step 2.4.1: The clustering module is initialized using the k-means clustering algorithm. Cluster centroids of multimodal fusion features of individual samples , Let j represent the centroid of the j-th cluster. Indicates the number of cluster centroids; Step 2.4.2, the clustering module uses equation (10) to calculate the soft-assignment distribution. ,in, Indicates the first The probability that the multimodal fusion features of a sample belong to the j-th cluster: (10) Step 2.4.3, the clustering module uses equation (11) to calculate the auxiliary target distribution. ,in, No. The probability that the multimodal fusion features of a sample belong to the target of the j-th cluster: (11) Step 2.4.4, the clustering module uses equation (12) to calculate the soft-assignment distribution. and auxiliary target distribution The KL divergence between them is used as the clustering loss function. : (12) Step 2.4.5, the clustering module uses equation (13) to calculate the first... Clustering results of individual samples and as the first Predicted fault labels for each sample: (13) In equation (13), This indicates the index value corresponding to the maximum value of the output function; Step 3: Train the multimodal unsupervised network and optimize the network parameters: Step 3.1: Calculate the total loss function of the multimodal unsupervised network using equation (14). : (14) In equation (14), These represent three hyperparameters that balance the effects of different loss functions; Step 3.2, to minimize the total loss function With the goal of optimizing and updating the parameters of the multimodal unsupervised network through backpropagation and adaptive moment estimation optimization algorithms, the training ends when the number of iterations reaches the set upper limit, thus obtaining a trained multimodal unsupervised network model for fault diagnosis of mechanical equipment.

2. The mechanical equipment fault diagnosis method based on multimodal deep clustering according to claim 1, characterized in that, Step 2.2 includes the following steps: Step 2.2.1, the gated recursive unit self-encoder module uses equation (1) to process the first... Signal data of one sample Perform encoding operations to obtain the first... Signal data of one sample Depth representation features ,in, Indicates the first Signal data of one sample The depth representation features at the t-th time step, The depth represents the feature dimension of each channel in the feature representation. This indicates the number of time steps divided when inputting signal data into the gated recursive unit self-encoder: (1) In equation (1), This represents the weight parameters of the encoding layer in the gated recursive unit autoencoder module. This represents the bias vector of the encoding layer in the gated recursive unit autoencoder module. This represents the activation function of the encoding layer in the gated recursive unit autoencoder module; Step 2.2.2, the gated recursive unit autoencoder module uses equation (2) to represent the depth features. Perform decoding to obtain the reconstructed signal data. : (2) In equation (2), This represents the weight parameters of the decoding layer in the gated recursive unit autoencoder module. This represents the bias vector of the decoding layer in the gated recursive unit autoencoder module. This represents the activation function of the decoding layer in the gated recursive unit autoencoder module; Step 2.2.3, the convolutional autoencoder module uses equation (4) to process the first... Time-frequency image data Perform encoding operations to obtain the first... Time-frequency image data Depth representation features ,in, Indicates the first Time-frequency image data The depth representation feature of the v-th channel: (4) In equation (4), This represents the weight parameters of the encoding layer in the convolutional autoencoder module. This represents the bias vector of the coding layer in the convolutional autoencoder module. This represents the activation function of the coding layer in the convolutional autoencoder module; Step 2.2.4, the convolutional autoencoder module uses equation (5) to represent the depth features. Decoding is performed to obtain the reconstructed time-frequency image data. : (5) In equation (5), This represents the weight parameters of the decoding layer in the convolutional autoencoder module. This represents the bias vector of the decoding layer in the convolutional autoencoder module. This represents the activation function of the decoding layer in the convolutional autoencoder module.

3. The mechanical equipment fault diagnosis method based on multimodal deep clustering according to claim 2, characterized in that, Step 2.3 includes the following steps: Step 2.3.1, the multimodal fusion module uses equation (7) to obtain the similarity matrix between the two modal depth features. : (7) In equation (7), This represents the parameter matrix for two modes in the multimodal fusion module. Let T denote the hyperbolic tangent activation function, and T denote the transpose. Step 2.3.2, the multimodal fusion module uses equation (8) to obtain the attention weights of the two modalities. : (8) In equation (8), This represents the parameter vectors of two modes in the multimodal fusion module. They represent the first Signal data of one sample Depth representation features Attention weights They represent the first Time-frequency image data Depth representation features Attention weights This represents the normalized exponential activation function; Step 2.3.3: The multimodal fusion module uses equation (9) to perform a weighted summation of the depth features of different modalities to obtain the multimodal fusion features. : (9)。 4. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the mechanical equipment fault diagnosis method according to any one of claims 1-3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is run by the processor, it performs the steps of the mechanical equipment fault diagnosis method according to any one of claims 1-3.

Images