Fault diagnosis digital twinning monitoring method and system based on deep learning

Abstract

The invention discloses a fault diagnosis digital twinning monitoring method and system based on deep learning, and belongs to the technical field of intelligent manufacturing fault diagnosis and digital twinning. The monitoring method is realized based on the system, and comprises the following steps of: constructing equipment and part models through three-dimensional modeling software, and acquiring an equipment running state and a part vibration signal in real time; constructing and training a deep learning fault diagnosis model by combining the actually measured signal and a transient dynamics simulation signal based on a three-dimensional model; and finally, establishing data connection between physical equipment and the virtual model, realizing virtual-real synchronization by using an operation data driving model, and integrating a diagnosis model to carry out real-time fault diagnosis and visual rendering to form a low-delay digital twin closed loop. According to the invention, by integrating digital twinning and deep learning technologies, real-time virtual-real synchronization and online fault diagnosis of the equipment operation state are realized, the manual inspection load and shutdown risk are effectively reduced, and data support is provided for predictive maintenance.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing fault diagnosis and digital twin technology, specifically to a fault diagnosis digital twin monitoring method and system based on deep learning. Background Technology

[0002] With the deepening development of Industry 4.0 and intelligent manufacturing, digital twin technology is regarded as a key means to achieve visualization, intelligence, and self-optimization of production processes. By constructing a 3D model in virtual space that is highly consistent with the physical entity and continuously injecting real-time data, digital twins can provide fault diagnosis analysis, process optimization, and decision support in the design, manufacturing, and operation and maintenance stages. Traditional fault diagnosis methods often rely on periodic shutdowns for manual inspection, which cannot provide real-time and intuitive feedback. At the same time, how to integrate equipment operation data and diagnostic results and display them in real time on a visualization platform is also a current technological challenge. In addition, the technical means of data synchronization and real-time updates still need to be improved to achieve millisecond-level virtual-real synchronization between equipment operating status and the virtual scene, forming an end-to-end, low-latency digital twin closed loop.

[0003] To address the aforementioned problems, this invention proposes a deep learning-based digital twin monitoring method and system for fault diagnosis. This innovative approach overcomes the limitations of existing technologies and provides an effective intelligent and digital solution for the development of smart manufacturing. Summary of the Invention

[0004] This invention proposes a digital twin monitoring method and system for fault diagnosis based on deep learning, which provides an efficient and intuitive way to digitally display the operating status of equipment.

[0005] The technical solution of this invention is as follows: A deep learning-based digital twin monitoring method for fault diagnosis includes the following steps: Step 1: Construct 3D models of the equipment and components; Step 2: Collect real-time equipment operating status data and vibration acceleration signals of the equipment components to be monitored. The vibration acceleration signals include vibration acceleration signals under various fault types of equipment components and vibration acceleration signals under normal operating conditions. Step 3: Train a one-dimensional convolutional long short-term memory neural network (1D-CNN-LSTM) deep learning fault diagnosis model. The data comes from vibration acceleration signals collected by vibration acceleration sensors and vibration acceleration signals under various fault types obtained by high-fidelity modeling of bearing transient dynamics simulation. Features are extracted from the training set signals and input into the model for training. Step 4: Establish a data connection between the equipment and the 3D model. Drive the 3D model to move in real time by collecting the equipment's operating status data to achieve virtual-real synchronization. Input the collected vibration signals into the deep learning fault diagnosis model to obtain and display the fault diagnosis results. Step 5: Visualize and render the digital twin scene that integrates the deep learning fault diagnosis model.

[0006] Furthermore, the specific process of step 1 is as follows: 1.1) Use SolidWorks to model the equipment and its components, creating an initial 3D model; 1.2) Export the 3D model file processed by SolidWorks to provide a data foundation for subsequent steps; 1.3) Store the final 3D model in a unified model library to support virtual space modeling and virtual-real synchronization of the digital twin system.

[0007] Furthermore, the specific process of step 2 is as follows: 2.1) Map the operating status parameters of machine tool doors, industrial robot axis positions, speeds, etc. during the operation of production line equipment to subscribed nodes and write them to the corresponding fields of the MySQL database with a refresh cycle of no more than 500ms; 2.2) Vibration acceleration sensors installed on the outer surface of the corresponding components (bearings) of the equipment are used to collect the signals of the components under various fault types and vibration acceleration signals under normal operating conditions at a sampling frequency of not less than 20kHz.

[0008] Furthermore, in step 3: the obtained original vibration acceleration signal is divided into seven categories according to the bearing condition label: "normal, inner ring defect, outer ring defect, rolling element defect" and "early, middle and failure period", and endpoint alignment and segmentation are performed; wavelet packet denoising, Z-Score normalization and Hilbert envelope spectrum analysis are performed on each segment of the signal in sequence to extract time domain (root mean square, kurtosis, etc.) and frequency domain (dominant frequency amplitude, frequency band energy, etc.) feature vectors, and feature indicators suitable for this object are selected.

[0009] Furthermore, in step 3: a 1D-CNN-LSTM network consisting of 3 layers of one-dimensional convolutional layers (3-7 kernels, 32-128 channels), 2 layers of bidirectional LSTM (64-128 hidden units), and 1 layer of fully connected Softmax classifier is constructed as a deep learning fault diagnosis model. The denoised one-dimensional signal and its feature vector are input in parallel, and the output of the convolutional layer is concatenated with the output of the LSTM after global average pooling.

[0010] Furthermore, the training method for the deep learning fault diagnosis model is as follows: The model is trained using a mixed dataset of simulated and measured signals, and then adjusted using pure measured signals. During training, the learning rate of the optimizer is set, and weighted cross-entropy is used as the loss function. Overfitting of the model is controlled by cross-validation, cosine annealing learning rate scheduling, and early stopping strategy.

[0011] Furthermore, the specific process of step 4 is as follows: 4.1) Establish a real-time data channel between the equipment and the digital twin system to transmit the operational data collected by the sensors to the deep learning diagnostic model deployed on the server in real time; 4.2) Receive diagnostic results via a two-way communication protocol and feed the results back to the user interface of the digital twin system; 4.3) Parse device data nodes using industrial communication protocols and subscribe to device status data; 4.4) Based on the received real-time data, dynamically adjust the position, motion state, and visualization parameters of the corresponding 3D model in the digital twin model to achieve synchronous mapping between physical devices and virtual scenes.

[0012] Furthermore, the specific process of step 5 is as follows: By integrating a deep learning diagnostic model through the application programming interface of a 3D rendering engine, data interaction between the diagnostic logic and the visualization system is achieved; a general rendering pipeline and programmable shader technology are used to dynamically visualize and render equipment operating status data and diagnostic results; thus forming an end-to-end, low-latency digital twin closed loop.

[0013] A deep learning-based digital twin monitoring system for fault diagnosis includes: Physical workshop layer: includes physical equipment and data sensing modules; the data sensing modules include PLCs and sensor devices, responsible for collecting data during equipment operation, such as the position and speed of machine tool doors and industrial robot axes, vibration signals of parts, and other status parameters; and transmitting the collected data to the database; Virtual workshop layer: includes 3D modeling and layout construction related to the physical workshop layer; Data transmission layer: used to receive and store data from the physical data acquisition layer; run deep learning fault diagnosis models to analyze the received vibration acceleration signals and generate fault diagnosis results; Application layer: It connects with the physical workshop layer and the data transmission layer in real time, and uses the operating status parameters to drive the 3D model in the virtual workshop layer in real time, realizing the synchronous mapping between physical equipment and virtual scene; it integrates a deep learning fault diagnosis model, and performs visualization rendering and display of equipment operating status and diagnosis results, forming an end-to-end, low-latency (latency less than 100 milliseconds) digital twin closed loop.

[0014] The beneficial effects of this invention are as follows: 1) This invention uses a deep learning model for online fault diagnosis of equipment components, which greatly reduces the workload of manual periodic inspections and reduces the risk of equipment downtime. It also provides data support for the development of predictive maintenance strategies. 2) This invention achieves synchronous transmission of deep learning diagnostic model data and traditional PLC register data through data transmission technology, which is helpful for realizing digital intelligent manufacturing; 3) This invention combines a digital twin system and a deep learning model through the Unity platform, achieving millisecond-level virtual-real synchronization between the device's operating status and the virtual scene, while also enabling real-time online fault diagnosis, thus forming an end-to-end, low-latency digital twin system. Attached Figure Description

[0015] Figure 1 A schematic diagram of the architecture of a deep learning-based digital twin monitoring system for fault diagnosis provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the system data transmission process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a digital twin monitoring platform provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, advantages, and technical solutions of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the embodiments described below are only for explaining the invention and are not limited to it. The invention will be described in detail below, and through these details, those skilled in the art will fully understand the invention.

[0017] like Figure 1 As shown in the figure, the fault diagnosis digital twin monitoring method based on deep learning proposed in this embodiment of the invention includes the following steps: S1. Integrate SolidWorks for 3D model design and processing. Details are as follows: The creation of the 3D geometric model includes: modeling the machine tool and robotic arm using SolidWorks, including its geometric shape and structural parameters; creating an initial 3D model using SolidWorks' modeling tools and performing detailed design to meet requirements; subsequently, exporting the SolidWorks-processed model file to provide a data foundation for subsequent work; and finally, storing the final 3D models in a unified model library to support the virtual space modeling and virtual-real synchronization functions of the subsequent digital twin system. This process is fundamental to building the digital twin system, ensuring the accurate and efficient design of the virtual model.

[0018] S2. Utilize KepServer to collect real-time equipment operating status data and store the data in a MySQL database; use vibration acceleration sensors to collect vibration signals from the monitored equipment components (bearings). Details are as follows: Create a data channel in KepServer, configure its connection to the PLC module of the blade processing equipment, and set the communication protocol and address information. Add labels for the machine tool door, the position of each axis of the industrial robot, speed, and other operating status parameters, ensuring that these labels correspond one-to-one with the corresponding data points in the PLC module. After mapping to a subscribing node via the OPC UA protocol, write the corresponding fields in the MySQL database with a refresh cycle of no more than 500ms; use vibration acceleration sensors installed on the outer surface of key equipment components (bearings) to collect bearing vibration signals at a sampling frequency of no less than 20kHz.

[0019] S3. Training a 1D-CNN-LSTM deep learning fault diagnosis model. Data is derived from vibration acceleration signals collected by vibration acceleration sensors and obtained through high-fidelity modeling and transient dynamics simulation of bearings under various fault types. Feature extraction is performed on the signals, and the data is input into the model for training. Details are as follows: The obtained original vibration acceleration signals were divided into seven categories according to bearing condition labels: "normal, inner ring defect, outer ring defect, rolling element defect" and "early, middle, and failure period". Endpoint alignment and segmentation were performed using a window length of 2048 points. Wavelet packet denoising, Z-score normalization, and Hilbert envelope spectrum analysis were performed on each signal segment to extract time-domain (root mean square, kurtosis, etc.) and frequency-domain (dominant frequency amplitude, frequency band energy, etc.) feature vectors, and feature indicators suitable for this object were selected. A 1D-CNN-LSTM network consisting of 3 layers of one-dimensional convolutional layers (kernel size 3-7, number of channels 32-128), 2 layers of bidirectional LSTM (hidden units 64-128), and 1 layer of fully connected Softmax classifier was constructed. The denoised one-dimensional signal and its feature vector were input in parallel. The output of the convolutional layer was concatenated with the LSTM output after global average pooling. The training was performed using a 3:1 mixed dataset of simulated and measured signals, and then fine-tuned using pure measured signals.

[0020] S4. The diagnostic model is integrated into the digital twin system to achieve digitization, visualization, and intelligence. Sensor data is pushed to a deep learning model on the server for diagnosis using FastAPI technology, and the diagnostic results are received synchronously. The Unity OPC UA SDK technology is used to implement virtual-real synchronization during device operation. Specifically: This embodiment also provides a deep learning-based fault diagnosis digital twin monitoring system, including a physical workshop layer, a virtual workshop layer, a data transmission layer, and an application layer.

[0021] The physical workshop layer includes physical equipment and data sensing modules; the data sensing modules include PLCs and sensor devices, which are responsible for collecting various data during equipment operation, such as the position and speed of machine tool doors and industrial robot axes, vibration signals of parts, and other status parameters.

[0022] The virtual workshop layer uses SolidWorks to render high-fidelity 3D models of physical equipment and builds the overall layout.

[0023] like Figure 2 As shown, the data transmission layer uses a bidirectional gRPC interface written in C# on the Unity side to push sensor data streams in real time to the deep learning diagnostic model deployed on the FastAPI+gRPC-Python server, synchronously receiving diagnostic results and displaying them in the UI panel. The Unity client parses the KepServerEX node tree and creates a Subscription using the Unity OPC UA SDK, and subscribes to the BladeDiag node with a 50ms refresh cycle using the MonitoredItem mechanism. When a data change callback is received, the position of the 3D model, shader color mapping, etc., are dynamically updated in the Main Thread, thereby achieving millisecond-level virtual-real synchronization between the device's operating status and the virtual scene.

[0024] The application layer calls the API to integrate a deep learning diagnostic model and uses URP + Shader Graph to visualize and render the data of the digital twin system, ultimately realizing an end-to-end, low-latency digital twin system.

[0025] like Figure 3 As shown, this system achieves highly synchronized operation of the physical production process in virtual space through real-time interaction between the virtual and physical workshops. The production line's operating status and vibration signals of key components are mapped in real-time to the digital twin platform via data acquisition and transmission modules, ensuring the digitization, visualization, and traceability of production process data. Based on this, the system integrates a deep learning fault diagnosis and prediction module, enabling real-time health monitoring and status assessment of key components such as bearings, timely detection of potential faults, and output of diagnostic results. This solution effectively reduces reliance on traditional periodic manual inspections, lowering the risk of equipment downtime due to delayed fault detection. Simultaneously, the digital twin monitoring platform provides a comprehensive view of operating status, historical data, and diagnostic results through a visual interface, offering intuitive and comprehensive decision support for maintenance personnel, thereby achieving intelligent, predictive, and low-cost operation and maintenance management of the production line.

[0026] The above detailed description of the present invention is only a preferred embodiment and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A digital twin monitoring method for fault diagnosis based on deep learning, characterized in that, Includes the following steps: Step 1: Construct 3D models of the equipment and components using 3D modeling software; Step 2: Collect real-time equipment operating status data and vibration acceleration signals of the equipment components to be monitored; Step 3: Construct a deep learning fault diagnosis model. Use the vibration signals of each fault type in the vibration acceleration signals of equipment components and the vibration acceleration signals of each fault type obtained by transient dynamic simulation based on the three-dimensional model of the components as the training set. Extract features from the training set signals and input them into the deep learning fault diagnosis model for training. Step 4: Establish a data connection between the equipment and the 3D model. Drive the 3D model to move in real time by collecting the equipment's operating status data to achieve virtual-real synchronization. Input the collected vibration acceleration signal into the deep learning fault diagnosis model to obtain and display the fault diagnosis results in real time. Step 5: Visualize and render the digital twin scene that integrates the deep learning fault diagnosis model.

2. The deep learning-based digital twin monitoring method for fault diagnosis according to claim 1, characterized in that, The specific process of step 1 is as follows: 1.1) Use SolidWorks to model the equipment and its components, creating an initial 3D model; 1.2) Export the 3D model file to provide a data foundation for subsequent steps; 1.3) Store the final 3D model in a unified model library to support virtual space modeling and virtual-real synchronization of the digital twin system.

3. The deep learning-based digital twin monitoring method for fault diagnosis according to claim 1, characterized in that, The specific process of step 2 is as follows: 2.1) Map the operating status parameters of the production line equipment during operation to subscribing nodes, and then write them to the corresponding fields in the MySQL database; 2.2) Vibration acceleration sensors installed on the outer surface of the corresponding components of the equipment are used to collect the signals of the components under various fault types and the vibration acceleration signals under normal operating conditions.

4. The deep learning-based digital twin monitoring method for fault diagnosis according to claim 1, characterized in that, In step 3: the obtained original vibration acceleration signal is classified according to the bearing condition label, and the endpoints are aligned and segmented; wavelet packet denoising, Z-Score normalization, and Hilbert envelope spectrum analysis are performed on each signal segment in sequence to extract time-domain and frequency-domain feature vectors, and the corresponding feature indicators are selected.

5. The deep learning-based digital twin monitoring method for fault diagnosis according to claim 1, characterized in that, In step 3, the deep learning fault diagnosis model adopts a one-dimensional convolutional long short-term memory neural network 1D-CNN-LSTM network. The 1D-CNN-LSTM network includes 3 one-dimensional convolutional layers, 2 bidirectional LSTM layers and 1 fully connected Softmax classifier. It inputs the denoised one-dimensional signal and its feature vector in parallel. The output of the convolutional layer is concatenated with the output of the LSTM after global average pooling.

6. The deep learning-based digital twin monitoring method for fault diagnosis according to claim 1, characterized in that, The training method for the deep learning fault diagnosis model in step 3 is as follows: The model is trained using a mixed dataset of simulated and measured signals, and then adjusted using pure measured signals. During training, the learning rate of the optimizer is set, and weighted cross-entropy is used as the loss function. Overfitting of the model is controlled by cross-validation, cosine annealing learning rate scheduling, and early stopping strategy.

7. The deep learning-based digital twin monitoring method for fault diagnosis according to claim 1, characterized in that, The specific process of step 4 is as follows: 4.1) Establish a real-time data channel between the equipment and the digital twin system to transmit the operational data collected by the sensors to the deep learning diagnostic model deployed on the server in real time; 4.2) Receive diagnostic results via a two-way communication protocol and feed the results back to the user interface of the digital twin system; 4.3) Parse device data nodes using industrial communication protocols and subscribe to device status data; 4.4) Based on the received real-time data, dynamically adjust the position, motion state, and visualization parameters of the corresponding 3D model in the digital twin model to achieve synchronous mapping between physical devices and virtual scenes.

8. The deep learning-based digital twin monitoring method for fault diagnosis according to claim 1, characterized in that, The specific process of step 5 is as follows: By integrating a deep learning diagnostic model through the application programming interface of a 3D rendering engine, data interaction between the diagnostic logic and the visualization system is achieved; a general rendering pipeline and programmable shader technology are used to dynamically visualize and render equipment operating status data and diagnostic results; thus forming an end-to-end, low-latency digital twin closed loop.

9. A deep learning-based digital twin monitoring system for fault diagnosis, used to implement the method described in any one of claims 1-8, characterized in that, include: Physical workshop layer: includes physical equipment and data sensing modules; The data sensing module includes a PLC and sensor devices, which are responsible for collecting data during equipment operation and transmitting the collected data to the database; Virtual workshop layer: includes 3D modeling and layout construction related to the physical workshop layer; Data transmission layer: used to receive and store data from the physical data acquisition layer; run deep learning fault diagnosis models to analyze the received vibration acceleration signals and generate fault diagnosis results; Application Layer: Connects with the physical workshop layer and data transmission layer in real time, uses operating status parameters to drive the 3D model in the virtual workshop layer in real time, and realizes synchronous mapping between physical equipment and virtual scene; integrates deep learning fault diagnosis model, and performs visualization rendering and display of equipment operating status and diagnosis results, forming an end-to-end, low-latency digital twin closed loop.

Images