Rotating machinery working condition fault recognition method based on multi-modal coordination clustering network
By using a multimodal coordinated clustering network, multimodal data of rotating machinery is collected and processed to generate discriminative operating condition vectors. By combining the tensor product mechanism and the entropy regular soft clustering method, the operating condition clustering results are optimized, achieving efficient identification of operating conditions of rotating machinery and solving the problems of neglecting intermodal consistency and structural relationships in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU VOCATIONAL INSTITUTE OF INDUSTRIAL TECHNOLOGY
- Filing Date
- 2026-04-02
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies neglect semantic consistency and potential structural relationships between modalities, resulting in poor robustness and weak generalization ability of clustering results for fault identification of rotating machinery operating conditions. Furthermore, they are difficult to effectively handle the incompleteness, heterogeneity, and redundancy between modalities.
Data is collected by installing multimodal sensors, a modality-specific encoder is constructed to generate the original embedding, a multimodal semantic graph is established to generate semantically enhanced embedding vectors through random sample walks, and adversarial generative networks and sparse soft clustering methods are used to optimize the clustering results. Finally, the working condition is identified through an MLP network.
It improves the accuracy of multimodal data processing and the precision of rotating machinery condition identification, solves the problem of modal data imbalance, and enhances the robustness and efficiency of identification.
Smart Images
Figure CN122286352A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mechanical fault identification technology, and in particular to a method for identifying faults in rotating machinery based on multimodal coordinated clustering networks. Background Technology
[0002] In modern industrial systems, rotating machinery, as a core component for power transmission and energy conversion, is widely used in key fields such as power, metallurgy, petrochemicals, and rail transportation. With the increasing speed and complexity of operating conditions, the operating status of rotating machinery is gradually exhibiting variability, nonlinearity, and strong coupling characteristics, placing higher demands on accurate monitoring and fault diagnosis. In recent years, the rapid development of sensor technology has driven advancements in state perception methods based on multi-source data. In particular, the fusion of multimodal data such as vibration, current, and temperature signals has made it possible to describe the operating conditions of rotating machinery in multiple dimensions. However, traditional single-modal fault diagnosis methods, due to their limited information redundancy and weak expressive power, are increasingly unable to meet the refined identification needs under complex operating conditions. Therefore, fault identification technology that integrates multimodal data has become a research hotspot. The key lies in how to efficiently extract the potential correlation features of each modality's data and perform cross-modal collaborative expression and clustering modeling.
[0003] Although existing research has made some progress in multimodal feature fusion and representation learning, there are still shortcomings. Existing technologies usually ignore the importance of semantic consistency and potential structural relationships between modalities, resulting in poor robustness and weak generalization ability of clustering results, and it is difficult to effectively deal with the incompleteness, heterogeneity and redundancy between modalities. Summary of the Invention
[0004] In view of the aforementioned existing problems, the present invention is proposed.
[0005] Therefore, this invention provides a method for identifying faults in rotating machinery based on multimodal coordinated clustering networks. This method addresses the problem that existing technologies neglect the importance of semantic consistency and potential structural relationships between modes, resulting in poor robustness and weak generalization ability of clustering results, and difficulty in effectively handling the incompleteness, heterogeneity, and redundancy between modes.
[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:
[0007] In a first aspect, the present invention provides a method for identifying faults in the operating conditions of rotating machinery based on a multimodal coordinated clustering network, comprising,
[0008] Multimodal sensors are installed on rotating machinery to collect multimodal data. Each type of data is preprocessed and a modality-specific encoder is constructed to generate the original embedding.
[0009] The original embedding is padded to construct an embedding matrix. Based on the embedding matrix, a multimodal semantic graph is constructed. Random sample walks are performed to generate semantically enhanced embedding vectors and generate class distributions to construct tensor product inputs. An adversarial generative network is established and a conditional adversarial discriminator is defined for training. The trained adversarial generative network generates a working condition representation vector.
[0010] Initialize the clustered working condition representation vector and define the structure contrast loss function. Simultaneously, use sparse soft clustering based on entropy regularization to cluster the working condition representation vector. Use joint loss optimization to optimize the sparse soft clustering and output the clustering results. Use an MLP network to identify the working conditions based on the clustering results.
[0011] The results of the working condition identification are displayed and stored in the database.
[0012] As a preferred embodiment of the rotating machinery fault identification method based on multimodal coordinated clustering network described in this invention, the following steps are included: preprocessing each modal data and constructing a modality-specific encoder to generate the original embedding; using a fifth-order Butterworth bandpass filter to suppress noise and normalize the collected modal data; constructing an independent deep convolutional encoder for each modal data; and extracting the original embedding from each modal data using the deep convolutional encoder. ;
[0013] Form a modality embedding set by combining all the original embeddings of the same modality. .
[0014] As a preferred embodiment of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network described in this invention, the following steps are taken: The original embedding is padded to construct an embedding matrix; a multimodal semantic graph is constructed based on the embedding matrix; random sample walks are performed to generate semantically enhanced embedding vectors; a category distribution is generated to construct a tensor product input; an adversarial generative network is established; a conditional adversarial discriminator is defined and trained; and the operating condition representation vector generated by the trained adversarial generative network includes…
[0015] The mode with the highest number of original embeddings among all modes is selected, and the number of original embeddings in that mode is used as the imputation benchmark. Based on the benchmark Calculate the original embedding padding for each mode. ;
[0016] The upper limit of repeated replication of the original mode is set by using equal-interval repeated sampling. ;
[0017] The original embeddings are selected at equal intervals from the modality embedding set and copied to form a copied embedding set. And fill the remaining part using the zero vector. ;
[0018] The remaining part is supplemented with the modality embedding set, the copy embedding set, and the zero vector. splicing to form a complete modal embedding set ;
[0019] Integrate the complete modal embedding sets of all modalities into a unified node set V;
[0020] The nodes are grouped using the K-means clustering algorithm, and the clustering label for each node is obtained. Define the edges connecting nodes. ;
[0021] Calculate the random walk probability of each node based on the edges connecting the nodes. ;
[0022] Let T be the number of walks for each node. Sort the connected nodes from highest to lowest random walk probability, and select the top T connected nodes to generate semantically enhanced embedding vectors. ;
[0023] The semantically enhanced embedding vector input modality mapping function is uniformly mapped to the shared embedding space generated modality mapping. And use a label classifier to generate a category distribution for each modality mapping. ;
[0024] Modality mapping and category distribution Tensor product fusion to generate two-dimensional tensors ;
[0025] The adversarial generative network consists of a generator and a discriminator. The adversarial generative network is trained by constructing a conditional adversarial loss using training data.
[0026] Minimize the conditional adversarial loss to train the adversarial generative network, using a two-dimensional tensor. The input to the trained adversarial generative network is the output job condition representation vector. .
[0027] As a preferred embodiment of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network described in this invention, the following steps are taken: Initializing the clustered operating condition representation vectors and defining a structure contrast loss function; simultaneously using entropy regularization-based sparse soft clustering to cluster the operating condition representation vectors; optimizing the sparse soft clustering using joint loss optimization; outputting the clustering result; performing initial clustering of the operating condition representation vectors using spectral clustering; assigning pseudo-labels to each operating condition representation vector based on the clustering result; calculating the average confidence score of the clustering assignment for each pair of operating condition representation vectors with the same pseudo-label; if it is greater than a confidence threshold, then the two are considered correlated. It is 1 if it is true, otherwise it is 0.
[0028] Calculate the structural similarity weights between the working condition representation vectors. ;
[0029] The structural contrast loss function is defined based on structural similarity weights. ;
[0030] Given H as the number of clusters and initial cluster centers, calculate the soft clustering probability distribution of each working condition representation vector. ;
[0031] The target clustering loss is defined based on the soft clustering probability distribution. ;
[0032] The structural contrast loss function and the target clustering loss are weighted and fused to form the final joint loss function. The initial cluster centers are updated by minimizing the final joint loss function, and the clustering is re-clustered based on the updated cluster centers. The clustering results are output, and cluster labels are assigned to the working condition representation vectors according to the clustering results.
[0033] As a preferred embodiment of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network described in this invention, the step of using an MLP network to identify operating conditions based on clustering results refers to constructing and training an MLP network, forming a set of operating condition representation vectors with assigned clustering labels, and outputting the rotating machinery operating condition identification result in the trained MLP network.
[0034] As a preferred embodiment of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network described in this invention, wherein: the installation of multimodal sensors on the rotating machinery to collect multimodal data refers to the installation of multimodal sensors on the rotating machinery to collect multimodal data of the rotating machinery, wherein the multimodal data includes vibration data, sound data and current data.
[0035] As a preferred embodiment of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network described in this invention, the step of displaying the operating condition identification results refers to displaying the operating condition identification results to the staff after obtaining the operating condition identification results of the rotating machinery, and simultaneously displaying the multimodal data of the rotating machinery to assist the staff in performing rotating machinery maintenance.
[0036] As a preferred embodiment of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network described in this invention, the "storing to database" refers to storing the operating condition identification results and multimodal data in the database and classifying and sorting them according to timestamps.
[0037] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network as described in the first aspect of the present invention.
[0038] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the rotating machinery fault identification method based on multimodal coordinated clustering network as described in the first aspect of the present invention.
[0039] The beneficial effects of this invention are as follows: This invention collects multimodal data of rotating machinery, extracts the original embeddings by constructing a modality-specific encoder, and performs walk enhancement based on the embedding matrix to build a semantic graph. Discriminative working condition vectors are generated in the generative adversarial network. At the same time, the working condition clustering results are optimized through the tensor product mechanism and the entropy regular soft clustering method, and the final recognition is achieved by combining it with MLP. This improves the accuracy of multimodal data processing and fusion, solves the problem of modality data imbalance, and greatly improves the accuracy and efficiency of rotating machinery working condition recognition. Attached Figure Description
[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0041] Figure 1 This is a flowchart of the rotating machinery fault identification method based on multimodal coordinated clustering network in Example 1. Detailed Implementation
[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0043] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.
[0044] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.
[0045] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a method for identifying faults in the operating conditions of rotating machinery based on a multimodal coordinated clustering network, including the following steps:
[0046] S1. Install multimodal sensors on rotating machinery to collect multimodal data, preprocess each type of modal data and construct a modality-specific encoder to generate the original embedding;
[0047] Specifically, installing multimodal sensors on rotating machinery to collect multimodal data refers to installing multimodal sensors on rotating machinery to collect multimodal data of the rotating machinery, including vibration data, sound data, and current data.
[0048] Furthermore, each modality of data is preprocessed, and a modality-specific encoder is constructed to generate the original embeddings. Noise suppression and normalization are performed on the collected modality data using a fifth-order Butterworth bandpass filter. An independent deep convolutional encoder is constructed for each modality of data, each consisting of 3 layers of convolution + BatchNorm + ReLU + max pooling, and is trained. The original embeddings are extracted from each modality of data using the deep convolutional encoder. :
[0049]
[0050] in For the depth convolutional encoder of the z-th mode, This is the i-th data point of the z-th mode;
[0051] Form a modality embedding set by combining all the original embeddings of the same modality. .
[0052] The Butterworth filter, due to its flat and ripple-free passband characteristics, is better able to preserve effective frequency domain information when processing vibration signals from rotating machinery, avoiding the suppression or weakening of key feature frequency components by traditional filters. Especially in multimodal signals (such as vibration, current, and temperature) exhibiting spectral crossover and feature migration phenomena, the fifth-order filter can effectively enhance feature fidelity and improve the accuracy of subsequent feature extraction. Normalization not only improves model convergence speed but also performs scale alignment of various modes at the input layer, providing a unified data foundation for subsequent convolutional feature extraction. A structure of 3 convolutional layers + BatchNorm + ReLU + max pooling is employed. The structure possesses strong feature extraction and nonlinear modeling capabilities, effectively adapting to the texture patterns or temporal structures of different modal data. Furthermore, the convolutional kernel parameters are completely independent, which helps to extract the unique semantic features of each modality to the greatest extent. It integrates all the original embeddings under each modality into a set, which not only meets the needs of subsequent graph construction but also implicitly contains embedding distribution information, providing a foundation for graph neural network modeling and sample similarity measurement. This structural organization method differs from the traditional "flat feature splicing" processing method, making the model more scalable and interpretable, and it can also flexibly handle small sample scenarios or modality-deficient problems.
[0053] S2. Complete the original embedding to construct an embedding matrix. Based on the embedding matrix, construct a multimodal semantic graph, perform random sample walks to generate semantically enhanced embedding vectors, generate class distributions, construct tensor product inputs, establish an adversarial generative network, define a conditional adversarial discriminator for training, and generate working condition representation vectors through the trained adversarial generative network.
[0054] Specifically, the original embeddings are padded to construct an embedding matrix. Based on the embedding matrix, a multimodal semantic graph is constructed, and a random sample walk is performed to generate semantically enhanced embedding vectors. A class distribution is generated to construct a tensor product input. An adversarial generative network is established, and a conditional adversarial discriminator is defined and trained. The trained adversarial generative network generates a work condition representation vector, including...
[0055] The mode with the highest number of original embeddings among all modes is selected, and the number of original embeddings in that mode is used as the imputation benchmark. Based on the benchmark Calculate the original embedding padding for each mode. ;
[0056] The upper limit of repeated replication of the original mode is set by using equal-interval repeated sampling. :
[0057]
[0058] in The number of original embeddings in the z-th modality;
[0059] The original embeddings are selected at equal intervals from the modality embedding set and copied to form a copied embedding set. :
[0060]
[0061] in For the copied original set of embeddings, This indicates rounding down. The original embedding is copied;
[0062] And fill the remaining part using the zero vector. :
[0063]
[0064] The remaining part is supplemented with the modality embedding set, the copy embedding set, and the zero vector. splicing to form a complete modal embedding set ;
[0065] Integrate the complete modal embedding sets of all modalities into a unified node set V:
[0066]
[0067] in For the i-th node embedding, there is a corresponding embedding in the complete modal embedding set, where M is the total number of modalities;
[0068] The nodes are grouped using the K-means clustering algorithm, and the clustering label for each node is obtained. Define the edges connecting nodes. :
[0069]
[0070] in Let i be the cluster label of the i-th node in the z-th modality. The cluster label for the j-th node in the k-th modality;
[0071] Calculate the random walk probability of each node based on the edges connecting the nodes. :
[0072]
[0073] in Let be the connection weight between the i-th node and the j-th node;
[0074] Let T be the number of walks for each node. Sort the connected nodes from highest to lowest random walk probability, and select the top T connected nodes to generate semantically enhanced embedding vectors. :
[0075]
[0076] in Embed for the t-th wandering connection node;
[0077] The semantically enhanced embedding vector input modality mapping function is uniformly mapped to the shared embedding space generated modality mapping. And use a label classifier to generate a category distribution for each modality mapping. :
[0078]
[0079] in and For classifier parameters, For the i-th mode mapping, , For modal mapping functions, d represents the modality dimension, and d represents the shared embedding space dimension.
[0080] Modality mapping and category distribution Tensor product fusion to generate two-dimensional tensors :
[0081]
[0082] The adversarial generative network (GAN) consists of a generator and a discriminator. The generator converts a two-dimensional tensor into a working condition representation vector, and the discriminator evaluates the generator's output. The GAN is trained using a conditional adversarial loss constructed from training data.
[0083]
[0084] in Let be the distribution of the i-th class of mode s. Let be the i-th two-dimensional tensor of mode s. To and The corresponding target modality category distribution, The corresponding two-dimensional tensor is obtained from the training data. The output of the discriminator has a range of [value missing]. , Calculate the expected value;
[0085] Minimize the conditional adversarial loss to train the adversarial generative network, using a two-dimensional tensor. The input to the trained adversarial generative network is the output job condition representation vector. .
[0086] Using the modality with the most embeddings as the completion benchmark, and employing a strategy of equal-interval duplication and zero-vector completion, this approach preserves the original distribution patterns to the greatest extent possible without introducing falsified unstructured information. When constructing the semantic graph, K-means clustering is used to group all nodes and define connection edges based on label consistency. A random walk strategy extracts contextual information from the global graph topology, fusing the target node with its semantically similar neighbors to enhance the semantic integrity of the nodes. By introducing a label classifier to obtain the category distribution of each modality mapping, and then performing a tensor product operation with the modality mapping results, classification information and feature information are tightly combined, significantly improving the ability of the expression vector to discriminate specific working conditions. This not only enhances the fusion between modal semantics but also... The depth also provides structured supervision for the training of downstream generative networks, making the generated results closer to the semantic features of real working conditions. By using a conditional adversarial generative network to model the working condition representation vector, it can not only learn complex nonlinear mappings between data, but also supervise the generation quality during training through the conditional adversarial loss function. This strategy constructs a generative network with conditional input by simultaneously considering modal tensors and target class distributions, effectively avoiding the problem of generation results deviating from semantic labels under purely data-driven conditions. Especially in cases where there is class overlap and insufficient samples in complex working condition data of rotating machinery, the adversarial mechanism can automatically generate embedding representations with discriminative and generalizable capabilities, greatly improving the robustness of the overall system in recognizing unseen working conditions.
[0087] S3. Initialize the clustered working condition representation vector and define the structure contrast loss function. Simultaneously, use sparse soft clustering based on entropy regularization to cluster the working condition representation vector. Use joint loss optimization to optimize the sparse soft clustering and output the clustering results. Use an MLP network to identify the working conditions based on the clustering results.
[0088] Specifically, the clustering process initializes the working condition representation vectors and defines a structure-contrast loss function. Simultaneously, sparse soft clustering based on entropy regularization is used to cluster the working condition representation vectors. Joint loss optimization is then applied to optimize the sparse soft clustering, outputting the clustering results. Spectral clustering is then used to initially cluster the working condition representation vectors, and pseudo-labels are assigned to each working condition representation vector based on the clustering results. The average confidence score of the clustering assignment for each pair of working condition representation vectors with the same pseudo-label is calculated. If the average confidence score is greater than a confidence threshold, the two are considered correlated. It is 1 if it is true, otherwise it is 0.
[0089] Calculate the structural similarity weights between the working condition representation vectors. :
[0090]
[0091] in For scale hyperparameters, and Let the vectors represent the i-th and j-th working conditions;
[0092] The structural contrast loss function is defined based on structural similarity weights. :
[0093]
[0094] in For temperature parameters, Let be the structural similarity weight between the i-th working condition representation vector and the k-th working condition representation vector. Let B be the cosine similarity between the i-th working condition representation vector and the k-th working condition representation vector, and let B be the total number of working condition representation vectors.
[0095] Given H as the number of clusters and initial cluster centers, calculate the soft clustering probability distribution of each working condition representation vector. :
[0096]
[0097] in and The h-th and j-th initial cluster centers;
[0098] The target clustering loss is defined based on the soft clustering probability distribution. :
[0099]
[0100] The structural contrast loss function and the target clustering loss are weighted and fused to form the final joint loss function. The initial cluster centers are updated by minimizing the final joint loss function, and the clustering is re-clustered based on the updated cluster centers. The clustering results are output, and cluster labels are assigned to the working condition representation vector according to the clustering results, such as low load, high load, high speed, etc.
[0101] Spectral clustering, as an effective unsupervised method for extracting low-dimensional manifold structures from data, utilizes the graph Laplacian matrix to reduce the dimensionality of the high-dimensional feature space and extract structural principal components. This effectively captures the latent manifold structures in operational data. The resulting pseudo-labels not only retain the initial data distribution characteristics but also provide a basis for subsequent supervised modeling (such as pseudo-label similarity construction), helping to overcome the lack of semantic constraints in traditional unsupervised clustering and strengthening the semantic guidance of the clustering process. By calculating the average confidence score of cluster assignments among samples with the same pseudo-label and setting a confidence threshold to determine their connectivity, a binary structural graph is constructed. This mechanism essentially introduces confidence judgment and noise filtering into the establishment of edges in the graph structure, effectively avoiding the problems caused by initial clustering errors. This approach eliminates unreliable connections while maintaining the sparsity of the structure graph and reducing computational overhead. By introducing a fusion loss of structural similarity weights and cosine similarity, the structure-contrast loss function effectively measures the consistency of samples in the structural adjacency graph and the directional proximity in the representation space. It imposes additional closing constraints on samples with similar structures but large directional deviations, and strengthens the separation for semantically inconsistent samples, thereby improving the clarity of the discrimination boundaries of various working conditions in the clustering space. An entropy regularization term is used to construct a sparse soft clustering model. By enhancing the difference in selection confidence of each sample among multiple cluster centers, a "weighted tilt" of the clustering distribution is achieved. That is, while retaining the flexibility of soft clustering, excessive ambiguity is limited, thereby improving the discriminativeness and semantic aggregation power of cluster labels.
[0102] Furthermore, based on the clustering results, the MLP network is used for working condition identification. This involves constructing and training an MLP network, and then forming a set of working condition representation vectors assigned with clustering labels. The trained MLP network outputs the working condition identification results for rotating machinery, including normal operation and bearing damage.
[0103] S4. Display the working condition identification results and store them in the database;
[0104] Specifically, installing multimodal sensors on rotating machinery to collect multimodal data refers to installing multimodal sensors on rotating machinery to collect multimodal data of the rotating machinery, including vibration data, sound data, and current data.
[0105] Furthermore, displaying the operating condition identification results means that after obtaining the operating condition identification results of the rotating machinery, the results are displayed to the staff, and the multimodal data of the rotating machinery are displayed simultaneously to assist the staff in maintaining the rotating machinery.
[0106] This embodiment also provides a computer device applicable to the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network proposed in the above embodiment.
[0107] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.
[0108] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the rotating machinery operating condition fault identification method based on a multimodal coordinated clustering network as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0109] In summary, this invention collects multimodal data of rotating machinery, extracts the original embeddings by constructing a modality-specific encoder, and performs walk enhancement based on the embedding matrix to build a semantic graph. It generates discriminative working condition vectors in a generative adversarial network, optimizes the working condition clustering results through a tensor product mechanism and an entropy regular soft clustering method, and achieves final recognition by combining MLP. This improves the accuracy of multimodal data processing and fusion, solves the problem of modality data imbalance, and significantly enhances the accuracy and efficiency of rotating machinery working condition recognition.
[0110] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for identifying faults in rotating machinery based on multimodal coordinated clustering networks, characterized in that: include, Multimodal sensors are installed on rotating machinery to collect multimodal data. Each type of data is preprocessed and a modality-specific encoder is constructed to generate the original embedding. The original embedding is padded to construct an embedding matrix. Based on the embedding matrix, a multimodal semantic graph is constructed. Random sample walks are performed to generate semantically enhanced embedding vectors and generate class distributions to construct tensor product inputs. An adversarial generative network is established and a conditional adversarial discriminator is defined for training. The trained adversarial generative network generates a working condition representation vector. Initialize the clustered working condition representation vector and define the structure contrast loss function. Simultaneously, use sparse soft clustering based on entropy regularization to cluster the working condition representation vector. Use joint loss optimization to optimize the sparse soft clustering and output the clustering results. Use an MLP network to identify the working conditions based on the clustering results. The results of the working condition identification are displayed and stored in the database.
2. The rotating machinery fault identification method based on multimodal coordinated clustering network as described in claim 1, characterized in that: The process of preprocessing each modality's data and constructing a modality-specific encoder to generate the original embedding involves using a fifth-order Butterworth bandpass filter for noise suppression and normalization on the collected modality data, constructing an independent deep convolutional encoder for each modality's data, and extracting the original embedding from each modality's data using the deep convolutional encoder. ; Form a modality embedding set by combining all the original embeddings of the same modality. .
3. The rotating machinery fault identification method based on multimodal coordinated clustering network as described in claim 2, characterized in that: The process involves padding the original embeddings to construct an embedding matrix, building a multimodal semantic graph based on the embedding matrix, performing random sample walks to generate semantically enhanced embedding vectors, generating a class distribution to construct a tensor product input, establishing an adversarial generative network, defining a conditional adversarial discriminator for training, and generating a working condition representation vector through the trained adversarial generative network. The mode with the highest number of original embeddings among all modes is selected, and the number of original embeddings in that mode is used as the imputation benchmark. Based on the benchmark Calculate the original embedding padding for each mode. ; The upper limit of repeated replication of the original mode is set by using equal-interval repeated sampling. ; The original embeddings are selected at equal intervals from the modality embedding set and copied to form a copied embedding set. And fill in the remaining part using the zero vector. The modality embedding set, the copy embedding set, and the zero vector are used to pad the remaining parts. splicing to form a complete modal embedding set ; Integrate the complete modal embedding sets of all modalities into a unified node set V; The nodes are grouped using the K-means clustering algorithm, and the clustering label for each node is obtained. Define the edges connecting nodes. ; Calculate the random walk probability of each node based on the edges connecting the nodes. ; Let T be the number of walks for each node. Sort the connected nodes from highest to lowest random walk probability, and select the top T connected nodes to generate semantically enhanced embedding vectors. ; The semantically enhanced embedding vector input modality mapping function is uniformly mapped to the shared embedding space generated modality mapping. And use a label classifier to generate a category distribution for each modality mapping. ; Modality mapping and category distribution Tensor product fusion to generate two-dimensional tensors ; The adversarial generative network consists of a generator and a discriminator. The adversarial generative network is trained by constructing a conditional adversarial loss using training data. Minimize the conditional adversarial loss to train the adversarial generative network, using a two-dimensional tensor. The input to the trained adversarial generative network is the output working condition representation vector. .
4. The rotating machinery fault identification method based on multimodal coordinated clustering network as described in claim 3, characterized in that: The process involves initializing the clustered working condition representation vectors and defining a structure-contrast loss function. Simultaneously, sparse soft clustering based on entropy regularization is used to cluster the working condition representation vectors. Joint loss optimization is employed to optimize the sparse soft clustering, outputting the clustering results. This involves initial clustering of the working condition representation vectors using spectral clustering and assigning pseudo-labels to each working condition representation vector based on the clustering results. The average confidence score of the clustering assignment for each pair of working condition representation vectors with the same pseudo-label is calculated. If the average confidence score is greater than a confidence threshold, then the two are considered correlated. It is 1 if it is true, otherwise it is 0; Calculate the structural similarity weights between the working condition representation vectors. ; The structural contrast loss function is defined based on structural similarity weights. ; Given H as the number of clusters and initial cluster centers, calculate the soft clustering probability distribution of each working condition representation vector. ; The target clustering loss is defined based on the soft clustering probability distribution. : The structural contrast loss function and the target clustering loss are weighted and fused to form the final joint loss function. The initial cluster centers are updated by minimizing the final joint loss function, and the clustering is re-clustered based on the updated cluster centers. The clustering results are output, and cluster labels are assigned to the working condition representation vectors according to the clustering results.
5. The rotating machinery fault identification method based on multimodal coordinated clustering network as described in claim 4, characterized in that: The step of using an MLP network to identify operating conditions based on clustering results refers to constructing and training an MLP network, forming a set of operating condition representation vectors with assigned clustering labels, and outputting the rotating machinery operating condition identification results in the trained MLP network.
6. The method for identifying faults in rotating machinery based on multimodal coordinated clustering networks as described in claim 5, characterized in that: The phrase "installing multimodal sensors on rotating machinery to collect multimodal data" refers to installing multimodal sensors on rotating machinery to collect multimodal data of the rotating machinery, including vibration data, sound data, and current data.
7. The rotating machinery fault identification method based on multimodal coordinated clustering network as described in claim 6, characterized in that: The process of displaying the operating condition identification results refers to displaying the operating condition identification results of the rotating machinery to the staff after obtaining the operating condition identification results, and simultaneously displaying the multimodal data of the rotating machinery to assist the staff in maintaining the rotating machinery.
8. The method for identifying faults in rotating machinery based on multimodal coordinated clustering networks as described in claim 7, characterized in that: The term "store to database" refers to storing the operating condition identification results and multimodal data in the database and classifying and sorting them by timestamp.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the rotating machinery operating condition fault identification method based on multimodal coordinated clustering network as described in any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the rotating machinery fault identification method based on multimodal coordinated clustering network as described in any one of claims 1 to 8.