A rolling bearing fault diagnosis method and device based on GCN-ResNet

Abstract

The present invention relates to industrial bearing condition monitoring, and in particular to a rolling bearing fault diagnosis method and device based on GCN-ResNet. This method addresses the problems in industrial Internet of Things applications, such as poor data fusion diagnostic capabilities for equipment status, low accuracy in diagnosing equipment status using a single modality, and high model complexity, thereby improving the real-time and accuracy of bearing condition monitoring. The rolling bearing fault diagnosis device based on GCN-ResNet includes a data acquisition unit, a data forwarding unit, and a data fusion analysis unit; the data acquisition unit is used to acquire multimodal sensor data and transmit the data to a server; the data forwarding unit is responsible for collaborative communication between the server and the client; the data fusion analysis unit is used to fuse and analyze multimodal sensor data and monitor the bearing condition in real time.

Description

Technical Field

[0001] The present invention relates to industrial bearing condition monitoring, and more particularly to a rolling bearing fault diagnosis method and device based on GCN-ResNet. Background Art

[0002] Currently, the trend is to promote the digitalization, networking, intelligence, high-end development, and greenness of the manufacturing industry. By promoting new industrialization and developing the digital economy, and accelerating the development of artificial intelligence technology, we are driving the transformation and upgrading of traditional manufacturing to intelligent manufacturing. Intelligent manufacturing uses sensors to capture key data in real time and transmits it to a host computer for data fusion and analysis. The goal is to promptly detect anomalies, optimize production processes, and predict equipment failures, thereby improving production efficiency and product quality. The Industrial Internet of Things (IIoT) deeply integrates industrial manufacturing with technologies such as the Internet of Things, artificial intelligence, cloud computing, and big data analytics, enabling the interconnection of equipment, production lines, and management systems, and driving the development of intelligent manufacturing.

[0003] In the context of intelligent manufacturing, the deployment of advanced manufacturing equipment and sensing devices at industrial sites is increasing. Industrial network protocols are complex and diverse, and data exhibits multimodal characteristics, such as large data volumes, low value density, and multiple sources and dimensions. Due to factors such as complex production environments and limited equipment energy consumption, traditional single-modal analysis methods often miss key status data and contain significant amounts of errors and redundant information. This leads to distortions in the actual equipment status curve and low data reliability, making it difficult to fully monitor the changing characteristics of the physical world. Multimodal data analysis methods, which incorporate multiple aspects of information, can more accurately reflect the true state of equipment. However, effective fusion of multimodal data is difficult, increasing processing and analysis complexity and, in turn, impacting the accuracy of real-time monitoring and decision-making. Furthermore, the heterogeneity and inconsistency of multimodal data can lead to data conflicts and redundancies, further complicating data cleaning and fusion. These issues make it difficult to fully reflect the true state and changing characteristics of equipment using multimodal data in complex industrial environments, posing challenges to the implementation of intelligent manufacturing. Therefore, how to effectively integrate multimodal data, improve data accuracy and reliability, and then monitor the processing process in real time and ensure that the perception data maps the current physical state of the equipment with high quality has become an important issue that needs to be urgently addressed in the field of intelligent manufacturing. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies and provides a rolling bearing fault diagnosis method and device based on GCN-ResNet. This method addresses the problems of poor equipment status diagnosis using data fusion, low accuracy in diagnosing equipment status using a single modality, and high model complexity in industrial IoT applications. It improves the real-time and accuracy of bearing condition monitoring.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a rolling bearing fault diagnosis device based on GCN-ResNet, including a data acquisition unit, a data forwarding unit and a data fusion analysis unit; the data acquisition unit is used to realize the acquisition of multimodal sensor data and transmit the data to the server; the data forwarding unit is responsible for the collaborative communication between the server and the client; the data fusion analysis unit is used to fuse and analyze multimodal sensor data and monitor the bearing status in real time.

[0006] Furthermore, the data acquisition unit includes a data transmission component and a data acquisition module.

[0007] The data transmission component is used to collect vibration data generated by bearing rotation using an RS485 temperature and vibration sensor, and transmit the data to a data acquisition module via an RS485 to USB converter.

[0008] The data acquisition module is used to realize the communication between the sensor equipment and the data acquisition part, and receives data from the data transmission component through the RS485 to USB device.

[0009] The data acquisition module also forwards the received data to the host computer, and the host computer can send an inquiry request to the data acquisition module through the data forwarding module in the data forwarding unit, and the data acquisition module responds to the request, thereby realizing the interaction between the data acquisition module and the data forwarding part to ensure the real-time performance of the sensor's multimodal data transmission.

[0010] Furthermore, the data forwarding unit includes a network configuration module, a data forwarding module, a data sampling module, a data storage module, a data visualization display module, and a message mapping module.

[0011] The network configuration module completes the interconnection configuration between the server network and the client network, ensures that the data forwarding module can send query instructions to the data acquisition module in real time, and enables the data acquisition module to respond to data to the data forwarding module immediately.

[0012] After completing the network configuration, the data forwarding module can send a query instruction to the data acquisition module, and the data acquisition module responds to the data forwarding module with the collected data, thereby realizing real-time communication between the server and the client.

[0013] The data sampling module is used by the client to configure the data sampling rate of the server, and controls the rate at which the data forwarding module sends query instructions through three different sampling frequencies: fast, medium, and slow. The higher the sampling rate, the greater the amount of data collected per unit time.

[0014] The data storage module is used to store data obtained by sampling at different sampling frequencies locally on the client, and the stored data needs to be cleaned to filter out low-quality data in order to perform data fusion analysis.

[0015] The data visualization module is used to dynamically visualize the data obtained by sampling at different sampling frequencies. These data can be message mapped through a dynamic line graph to timely reflect data changes.

[0016] The message mapping module is used to map data into a dynamic line graph, which makes it easier for researchers to observe changes in vibration data in real time. It can also reflect the connection status between the server and the client and analyze the network connection status.

[0017] Furthermore, the data fusion analysis unit includes a data cleaning module, a data conversion module, a feature extraction module, a feature fusion module, and a model training module.

[0018] The data cleaning module is used to clean the data stored locally, remove low-quality data at the beginning and end of the vibration, and retain high-quality vibration data when the bearing is fully rotated.

[0019] The data conversion module is used to convert the cleaned time series data into visual data.

[0020] The feature extraction module uses ResNet152 and Wide_Resnet50-2 to extract features from the visual data converted by the data conversion module.

[0021] The feature fusion module constructs a graph structure dataset to prepare for feature fusion training based on the GCN (graph convolutional network) model for the different features extracted by the feature extraction module.

[0022] The model training module is trained using a graph structure data set constructed by a feature fusion module to achieve the purpose of accurately classifying bearing fault types.

[0023] Furthermore, the fault diagnosis method is based on a data fusion analysis unit and specifically includes the following steps:

[0024] S1: Data cleaning:

[0025] The collection time does not exceed three minutes to ensure that the faulty bearing does not work for too long.

[0026] The data stored locally is cleaned, the data with insufficient vibration is deleted, and the data with sufficient vibration of the bearing is retained, so as to extract the characteristics of the vibration data under normal working conditions.

[0027] S2: Data Encoding into Images: The cleaned data is encoded into images based on the Gram Angular Field (GAF) algorithm. The sensor vibration signal is converted into image data to meet the requirements of feature extraction of the pre-trained model. This includes:

[0028] Data type identification: Identify ten types of multimodal data collected by temperature and vibration sensors, including temperature modal data and triaxial modal data; triaxial modal data includes triaxial (X, Y, Z) velocity, triaxial displacement, and triaxial acceleration.

[0029] Data mapping: The temperature modal data is mapped to a grayscale image, and the triaxial modal data is mapped to an RGB image, ultimately obtaining four image data.

[0030] S3: The four encoded image data are respectively subjected to feature extraction using ResNet152 and Wide_Resnet50-2. Each image data is extracted into two 1024-dimensional abstract feature representations.

[0031] S4: Use the star topology analysis method to construct a graph structure and represent the features in the form of a graph to represent the mutual influence between each mode.

[0032] S5: Build a two-layer GCN (graph convolutional network) model and train the model on the graph structure dataset. After training, use the model to monitor the bearing status in real time.

[0033] Furthermore, since the acquisition time of the faulty bearing does not exceed three minutes, insufficient rotation of the bearing will occur each time sampling starts and ends. In this case, the data stored locally needs to be cleaned to delete the data of insufficient vibration.

[0034] The first step is to divide the time series data into three equal parts and take the middle part for calculation.

[0035] The second step is to calculate the lower margin of the middle part of the data based on the IQR (interquartile range) method. The specific method is:

[0036] Sort the middle data in ascending order according to the single mode. Taking the X-axis speed as an example, calculate the first quartile Q1 and the third quartile Q3. When the middle data is divided into an even number (2n) or an odd number (2n+1) of values.

[0037] The first quartile Q1 is the median of n values from small to large. The third quartile Q3 is the median of n values from large to small.

[0038] The calculation formula for the interquartile range (IQR) is as follows:

[0039] IQR=Q3-Q1

[0040] The formula for calculating the lower bound is as follows:

[0041] lower_bound = Q1 - 1.5 × IQR

[0042] Where lower_bound represents the lower boundary value;

[0043] If the calculated lower boundary value is less than 0, the lower boundary is changed to the smallest positive number greater than 0; then, the lower boundary value of each mode is obtained according to this step.

[0044] The third step is to clean the data at both ends. The first part is cleaned in order from the beginning to the end of the time. If any modal value is less than the lower boundary of the mode, this row of data is deleted; when a row of data appears and all data are greater than the lower boundary value, the cleaning is stopped; the third part is cleaned in order from the end to the beginning of the time. If any modal value is less than the lower boundary of the mode, this row of data is deleted; when a row of data appears and all data are greater than the lower boundary value, the cleaning is stopped.

[0045] Furthermore, the specific steps of S2 are as follows:

[0046] The first step is to use the Min-Max scaling method to scale the cleaned time series data to [-1, 1]. The calculation formula is as follows:

[0047]

[0048] Among them, X is the value of the modal data, X min is the minimum value of a series of time series data of this mode, X max It is the maximum value of a series of time series data of this mode.

[0049] The second step is to map the time series data X′ processed by the Min-Max scaling method to polar coordinates. The specific steps are as follows:

[0050] Time series data consists of two parts: the time series value and its corresponding timestamp. These two variables are mapped to angles and radii. Assuming that the time series consists of N timestamps and the corresponding value x, GAF uses arccos(x) to map the time series value to angles in the range [0, π]. Next, the time interval [0, 1] is divided into N equal parts, resulting in N+1 points {0, ..., i, ..., 1}. Zero is then discarded and these points are continuously associated with the time series. The functional relationship formula for all algorithm parameters is as follows:

[0051]

[0052] In the formula, x i It is a single data of a series of time series data X′ that has just been processed, such as x1 is the first data in X′, that is, i∈N.

[0053] The third step is to obtain θ i , r i Construct the Gram-like matrix G. The specific steps are as follows:

[0054] The entire encoding is treated as a combination of bijective functions, which losslessly maps the time series to polar coordinates while preserving long-term dependencies through the r coordinate. Due to the limitations of the dot product in the two-dimensional polar coordinate space, the GAF method customizes a dot product-like operation. This operation depends only on the angle to explain the individual and joint information at the two angles. The calculation formula is as follows:

[0055]

[0056] Where θ1 and θ2 are the angles after encoding x1 and x2; then we can construct the following Gram-like matrix G, which is calculated as follows:

[0057]

[0058] G successfully encodes time series data into images, making it easier to use pre-trained models to extract abstract features of the images.

[0059] The fourth step is to classify different modes according to their correlation with other modes. Since the Gram-like matrix G constructed by a single mode is a one-dimensional n-order square matrix, it is regarded as a single-channel matrix storing grayscale images. In the industrial bearing fault classification task, it is stipulated that:

[0060] (1) It is stipulated that single-modal data such as temperature, which has no strong correlation with other modes, is mapped into a single-channel grayscale image. Its channel is copied twice to obtain three three-channel matrices of the same data and input them into the ResNet 152 and Wide_resnet50-2 models to match the input data type of the model.

[0061] (2) It is stipulated that for three-axis modal data, such as three-axis velocity, this type of data with three modes and a strong correlation between the three modes is mapped into an RGB image; specifically, the Gram-like matrix constructed by the X-axis velocity is regarded as the R channel data, the Gram-like matrix constructed by the Y-axis velocity is regarded as the G channel data, and the Gram-like matrix constructed by the Z-axis velocity is regarded as the B channel data. The matrices with three separate channels are merged to obtain the storage form of the RGB image; in this way, this three-channel matrix can be input into the ResNet152 and Wide_resnet50-2 models to match the input data type of the model.

[0062] Furthermore, the steps for extracting features using the ResNet152 model in S3 are:

[0063] Because the feature extraction algorithm is based on the PyTorch deep learning framework, when the input image size is not 224×224, the image will be adaptively resized to 224×224.

[0064] Step 1: The data passes through the first convolutional layer Conv1, which uses a 7×7 convolution kernel, meaning that each convolution operation processes a 7×7 pixel block. This convolutional layer generates 64 output channels, or 64 feature maps, with a final output size of 112×112.

[0065] Step 2: The data passes through a 3×3 max pooling layer with a stride of 2 to reduce the spatial size of the feature map while retaining the salient features of each region. Next, the data passes through the second convolutional layer, Conv2, which constructs the bottleneck residual module, where each module consists of the following three convolutional layers.

[0066] A 1×1 convolutional layer with 64 output channels is used for dimensionality reduction.

[0067] A 3×3 convolutional layer with 64 output channels is used for feature extraction.

[0068] A 1×1 convolutional layer with an output channel of 256 is used for dimensionality increase.

[0069] This module is repeated 3 times; the final output size is 56×56.

[0070] Step 3: The data passes through the third convolutional layer Conv3, which builds the bottleneck residual module, where each module consists of the following 3 convolutional layers:

[0071] A 1×1 convolutional layer with 128 output channels is used for dimensionality reduction.

[0072] A 3×3 convolutional layer with 128 output channels is used for feature extraction.

[0073] A 1×1 convolutional layer with an output channel of 512 is used for dimensionality increase.

[0074] This module is repeated 8 times; the final output size is 28×28.

[0075] Step 4: The data passes through the fourth convolutional layer Conv4, which builds the bottleneck residual module, where each module consists of the following three convolutional layers:

[0076] A 1×1 convolutional layer with 256 output channels is used for dimensionality reduction.

[0077] A 3×3 convolutional layer with 256 output channels is used for feature extraction.

[0078] A 1×1 convolutional layer with an output channel of 1024 is used for dimensionality increase.

[0079] This module is repeated 36 times; the final output size is 14×14.

[0080] Step 5: The data passes through the fourth convolutional layer Conv5, which builds the bottleneck residual module, where each module consists of the following 3 convolutional layers:

[0081] A 1×1 convolutional layer with 512 output channels is used for dimensionality reduction.

[0082] A 3×3 convolutional layer with 512 output channels is used for feature extraction.

[0083] A 1×1 convolutional layer with an output channel of 2048 is used for dimensionality increase.

[0084] This module is repeated 3 times; the final output size is 7×7.

[0085] Step 6: The data passes through the average pooling layer to obtain 1024-dimensional abstract features; the model has a total of 152 layers;

[0086] The difference between the Wide_ResNet50-2 model and the ResNet152 model is that the number of output channels of the 3×3 convolutional layer of the bottleneck residual block of the Wide_ResNet50-2 is twice that of the original ResNet50, and the model has a total of 50 layers; therefore, the steps for extracting features using the Wide_ResNet50-2 model are the same as those for the ResNet152 model, and the dimensions of the output features are the same.

[0087] The difference is the network structure of the model, the number of layers, the number of channels and the weights of the model, and the numerical value of the features obtained are different. The specific network structure Figure 4 shown.

[0088] Furthermore, in S4, the star topology analysis method is used to construct a graph structure, and the features are represented in the form of a graph to represent the mutual influence between the modes. This is a step that must be performed before inputting into the GCN. Specifically, it includes:

[0089] 4.1. Construct a single-modal star subgraph, connecting a central vertex to two leaf vertices, with no edges connecting the two leaf vertices; according to regulations, there is a total of 1 temperature modal data and 3 triaxial modal data, so 4 single-modal star subgraphs need to be constructed.

[0090] 4.2. Assign the two 1024-dimensional features of one modality extracted by the two pre-trained models to the two leaf vertices. Assign the zero vector with the same dimension as the leaf vertices as the vertex feature of the center vertex. Repeat this step for each of the four subgraphs.

[0091] 4.3. Perform a full connection operation on the central vertices of the four unimodal star-shaped subgraphs. That is, except for the leaf vertices, the subgraph formed by the central vertices is an undirected complete graph.

[0092] Furthermore, in S5, establishing a two-layer GCN model includes the following steps:

[0093] A. Set up a two-layer GCN model:

[0094] Based on the GCN feature fusion algorithm, the model sets two GCN layers.

[0095] B. Feature absorption of the first GCN layer:

[0096] Each central vertex in the first GCN layer only absorbs the features of its leaf vertices.

[0097] C. Feature absorption of the second GCN layer:

[0098] In the second GCN layer, each central vertex not only absorbs the features of its leaf vertices, but also absorbs the features of other central vertices that have been fused with their respective leaf vertices.

[0099] D. Calculate the layered propagation rules and calculate the feature updates of each layer of GCN.

[0100] The calculation formula is as follows:

[0101]

[0102] in, is the adjacency matrix of the undirected graph G with self-connection added. N is the identity matrix. W (l) Yes th The trainable weight matrix of the layer, ReLU(·)=max(0,·). Yes th The feature matrix after layer activation. H (0) =X, is the node feature initially passed in.

[0103] E. Considering node neighbor information: Each iteration considers the node’s neighbor information, allowing the model to gradually understand the local and global structures in the graph.

[0104] F. The central vertex absorbs all node information: The central vertex can absorb the information of all nodes in the second iteration.

[0105] G. Feature fusion: After completing feature fusion, all attributes in the graph are considered.

[0106] H. Average pooling is used for graph-level classification: the average value of the node features in the last layer of GCN is taken as the input of the classification layer. The calculation formula is as follows:

[0107]

[0108] I. Use the cross entropy loss function to calculate the loss of the model:

[0109] x G The logits tensor x and the corresponding label y obtained by the classification layer, the loss function uses the cross entropy loss, and the calculation formula is as follows:

[0110]

[0111] J. Packaging training samples:

[0112] In one round of propagation, n training samples are packaged. The model propagates forward once and obtains n C-dimensional logits tensors x = {x1 (c) , x2 (c) , x3 (c) ,…,x n (c)}, C is the number of categories;

[0113] K, classification weight and label:

[0114] The weights of the C classes are ω={ω1,ω2,ω3,…,ω c}, the labels of n samples are y = {y1, y2, y3, ..., y n},in Represents the value of dimension j in the i-th sample, y i Represents the label corresponding to the i-th sample. Represents y i The weight corresponding to the class.

[0115] I. Use the ADAM (ADaptive Moment Estimation) optimizer to perform adaptive learning rate adjustment by combining momentum optimization, and then use the first and second moments of the gradient to calculate the discrete universal learning rate of various parameters.

[0116] Compared with the prior art, the present invention has beneficial effects.

[0117] 1. The present invention uses the Gram angle field to encode the time series data of industrial sensors. This data-driven method can convert time series information into images losslessly and preserve the long-term dependencies of the sequence.

[0118] 2. The present invention does not require traditional signal processing technology or image processing means to extract features, thus avoiding the subjectivity of manual feature extraction.

[0119] 3. This paper integrates computer vision into industrial process status monitoring and machine fault diagnosis by using transfer learning (TL). Using pre-trained models to extract features provides a better representation of images and can address sample sparsity and generalization issues in industrial processing tasks with limited data.

[0120] 4. This paper uses GCN to better capture the complex relationships hidden between different modalities through nonlinear feature integration and adaptive weight learning. Furthermore, the proposed method offers very low computational overhead and high accuracy, addressing the fundamental issues of high complexity and computational overhead in hybrid models. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] The present invention is further described below with reference to the accompanying drawings and specific embodiments. The scope of protection of the present invention is not limited to the following description.

[0122] Figure 1 This is an overview diagram of the system structure of the present invention.

[0123] Figure 2 Schematic diagram of the system structure of the present invention.

[0124] Figure 3 Schematic diagram of the client data acquisition system of the present invention.

[0125] Figure 4 Schematic diagram of data fusion analysis of the present invention. DETAILED DESCRIPTION

[0126] In order to make the purpose, technical solutions and beneficial effects of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0127] A rolling bearing fault diagnosis method and device based on GCN-ResNet, the system structure overview is shown in the figure below: Figure 1As shown in the figure, the system consists of a data acquisition part, a data forwarding part, and a data fusion and analysis part. The data acquisition part collects multimodal sensor data and sends it to the server. The data forwarding part is responsible for the coordinated communication between the server and the client. The data fusion and analysis part is used to fuse and analyze multimodal sensor data for real-time monitoring of bearing status.

[0128] Figure 2 The figure shows a schematic diagram of the system structure of the present invention. A rolling bearing fault diagnosis device based on GCN-ResNet includes a data transmission component and a data acquisition module; the data acquisition part mainly includes: the data transmission component is responsible for collecting the vibration data of the servo motor controlling the bearing rotation through the RS485 temperature vibration sensor, and sending the data to the data acquisition module through the RS485 to USB converter. The data acquisition module realizes the communication between the sensor device and the transmission module, and adopts the RS485 communication mechanism to ensure the real-time performance of the sensor's multimodal data transmission. The network configuration module of the server and client in the data forwarding part completes the interconnection configuration of the server network and the client network to ensure that the server and the client can communicate normally. The data sampling module realizes the client configuring the data sampling rate of the server. The data forwarding module forwards the server data to the client to realize real-time communication. The data storage module realizes the data storage locally on the client. The data visualization module dynamically visualizes the data to complete the function of timely reflecting data changes. The message mapping module obtains the connection status between the server and the client and analyzes the network connection status. In the data fusion and analysis part, the data cleaning module cleans the data, removes the start and end stages of vibration, and retains the vibration data of the bearing when it is fully rotated; the data conversion module converts the collected time series data into visual data, the feature extraction module extracts the features of the converted visual data, the feature fusion module uses the GCN model to fuse the multimodal data collected by the sensor, and the model training module trains the graph structure data set with fused features to achieve the purpose of classifying the bearing fault types.

[0129] The acquisition equipment includes a temperature and vibration sensor, an RS485-to-USB converter, a Raspberry Pi, a Raspberry Pi display, and a host computer. The sensor is deployed on the vibrating bearing to collect multimodal data. The converter performs protocol conversion, enabling the Raspberry Pi to successfully receive the data. The data is then forwarded to the host computer, enabling wireless network transmission. The Raspberry Pi display monitors real-time communication between the Raspberry Pi and the sensor. The host computer analyzes the real-time data, generates a visualization curve showing dynamic changes, and stores the data for data fusion analysis.

[0130] Figure 3The figure shows a schematic diagram of the client data acquisition system of the present invention. A real-time data acquisition software based on PyQt, which collects multimodal data from sensors and displays them visually, monitors the connection status between the server and the client in real time, and provides feedback through human-computer interaction. First, enter the IP address of the Raspberry Pi, open the listening port number on the server data acquisition system on the Raspberry Pi, select the serial port number of the RS485 to USB converter, set the device baud rate, and establish a communication connection between the client and the server. The stable result shows the bearing vibration data in this state, the Z-axis speed spectrum shows the real-time parameters at different speeds, and the data visualization shows the real-time changes in the temperature, three-axis acceleration, three-axis speed, and three-axis displacement of the sensor in the stable state. The communication message shows the client sending the query instruction, returning the data in the sensor register, and the process of the Raspberry Pi responding through the wireless network.

[0131] Figure 4 The data fusion analysis diagram of the present invention is shown below. The specific process is as follows:

[0132] First, the time series is scaled to [-1, 1] using the Min-Max scaling method.

[0133]

[0134] We need to consider the values of a time series and their corresponding timestamps, which are represented by angles and radius, respectively. Assuming a time series consists of N timestamps and a corresponding value x, GAF uses arccos(x) to map the values to angles, with angle values ranging from [0 to π]. Next, we divide the time interval [0, 1] into N equal parts, resulting in N+1 points {0, …, i, …, 1}. We then discard zero and continuously associate these points with the time series. The functional relationship between all the algorithm's parameters is as follows:

[0135]

[0136] The entire encoding, as a combination of bijective functions, losslessly maps the time series to polar coordinates while preserving long-term dependencies through the r-coordinate. Due to the limitations of the dot product in two-dimensional polar coordinate space, GAF defines a custom dot product-like operation. This operation depends only on the angle to interpret both individual and joint information from two angles. The calculation formula is as follows:

[0137]

[0138] Where θ1 and θ2 are the angles after encoding x1 and x2. So we can construct the following Gram-like matrix G, and the calculation formula is as follows:

[0139]

[0140] The time series data is successfully encoded into an image through G. At this time, the image is extracted using ResNet152 and Wide_ResNet50-2 respectively.

[0141] The ResNet152 network structure is as follows Figure 4 As shown in (a), the first layer includes a convolution kernel of size 7×7, a channel number of 64, and a stride of 2. After a 3×3 maximum pooling layer with a stride of 2, the second layer includes three convolution kernels of size 1×1, a channel number of 64, a convolution kernel of size 3×3, a channel number of 64, a 1×1 convolution kernel, and a channel number of 256. The third layer includes eight convolution kernels of size 1×1, a channel number of 128, a convolution kernel of size 3×3, a channel number of 128, a 1×1 convolution kernel, and a channel number of 256. The convolution kernel and the number of channels are 512. The fourth layer is a convolution layer with 36 convolution kernels of size 1×1 and the number of channels is 256, a convolution kernel of size 3×3 and the number of channels is 256, a convolution kernel of size 1×1 and the number of channels is 1024. The fifth layer is a convolution layer with 3 convolution kernels of size 1×1 and the number of channels is 512, a convolution kernel of size 3×3 and the number of channels is 512, a convolution kernel of size 1×1 and the number of channels is 2048. The last layer is the average pooling layer excluding the classification layer.

[0142] The Wide_ResNet50-2 network structure is as follows Figure 4 As shown in (b), the first layer includes a convolution kernel of size 7×7, a channel number of 64, and a stride of 2. After a 3×3 maximum pooling layer with a stride of 2, the second layer includes three convolution kernels of size 1×1, a channel number of 64, a convolution kernel of size 3×3, a channel number of 128, a convolution kernel of size 1×1, and a convolution layer with a channel number of 256. The third layer includes four convolution kernels of size 1×1, a channel number of 128, a convolution kernel of size 3×3, a channel number of 256, and a convolution kernel of size 1×1. The convolution kernel and the number of channels are 512. The fourth layer is 6 convolution kernels of size 1×1 and the number of channels is 256, the convolution kernel of size 3×3 and the number of channels is 512, and the convolution kernel of size 1×1 and the number of channels is 1024. The fifth layer is 3 convolution kernels of size 1×1 and the number of channels is 512, the convolution kernel of size 3×3 and the number of channels is 1024, and the convolution kernel of size 1×1 and the number of channels is 2048. The last layer is the average pooling layer excluding the classification layer.

[0143] After extracting features using the two models, a star topology analysis method is used to construct a graph structure, representing the features as a graph to illustrate the mutual influence of each modality. This is a necessary step before inputting into the GCN. First, the features of a modality extracted by n pre-trained models are assigned to n leaf vertices as vertex features of the leaf vertices. The vertex feature of the center vertex is a 0 vector with the same dimension as the leaf vertices. The center vertices of the m star graphs are then fully connected. That is, excluding the leaf vertices, the subgraph formed by the center vertices is an undirected complete graph.

[0144] After building the graph structure, use GCN for hierarchical propagation. Construct two layers of GCN and consider its hierarchical propagation rules. The calculation formula is as follows:

[0145]

[0146] in, is the adjacency matrix of the undirected graph G with self-connection added. N is the identity matrix. The purpose of adding self-connection is to consider the influence of node features in the process of aggregating features. W (l) Yes th The trainable weight matrix of the layer, ReLU(·)=max(0,·). Yes th The feature matrix after layer activation. H (0) =X, is the node feature initially passed in. In this way, each iteration will consider the node's neighbor information, allowing the model to gradually understand the local and global structure in the graph. Therefore, the central vertex can absorb the information of all nodes in the second iteration. After the feature fusion is completed, all attributes in the graph are considered. Graph-level classification means that given a graph dataset, the entire graph is classified according to the attributes in the graph structure. Graph-level classification uses the average pooling method, that is, the average value of the node features in the last layer of GCN is taken as the input of the classification layer. The calculation formula is as follows:

[0147]

[0148] Finally, x G The logits tensor x and the corresponding label y obtained by the classification layer, the loss function uses the cross entropy loss, and the calculation formula is as follows:

[0149]

[0150] Usually, in one round of propagation, n training samples are packaged. The model propagates forward for one round and obtains n C-dimensional logits tensors x = {x1 (c) , x2(c) , x3 (c) ,…,x n (c)}, C is the number of categories, and the weights of C categories are ω={ω1,ω2,ω3,…,ω c}, the labels of n samples are y = {y1, y2, y3, ..., y n},in Indicates the value of dimension j in the i-th sample, y i Represents the label corresponding to the i-th sample. Represents y i The weight corresponding to the class.

[0151] The ADAM (Adaptive Moment Estimation) optimizer is used to perform adaptive learning rate adjustments by combining momentum optimization. The first and second moments of the gradient are then used to calculate discrete universal learning rates for various parameters. ADAM is a simple and computationally efficient method that involves first-order gradients and minimizes memory requirements for stochastic optimization. After training, the trained GCN model is used for real-time bearing condition monitoring.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. Therefore, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A rolling bearing fault diagnosis device based on GCN-ResNet, characterized by: It includes data acquisition unit, data forwarding unit and data fusion analysis unit; The data acquisition unit is used to collect multimodal data from the sensor and transmit the data to the server; The data forwarding unit is responsible for the coordinated communication between the server and the client; The data fusion analysis unit is used to fuse and analyze multi-modal sensor data and to monitor the bearing status in real time; The fault diagnosis method based on the data fusion analysis unit includes the following steps: S1: Data cleaning: The collection time should not exceed three minutes; Clean the data stored locally, delete the data with insufficient vibration, and retain the data when the bearing is fully vibrating; S2: Data encoding into images: The cleaned data is encoded into images based on the GAF algorithm; the sensor vibration signal is converted into image data; including: Data type identification: Identify ten types of multimodal data collected by temperature and vibration sensors, including temperature modal data and triaxial modal data; triaxial modal data includes triaxial velocity, triaxial displacement, and triaxial acceleration; Data mapping: The temperature modal data is mapped to a grayscale image, and the triaxial modal data is mapped to an RGB image, ultimately obtaining four image data; S3: The four encoded image data are respectively subjected to feature extraction using ResNet152 and Wide_Resnet50-2. Each image data is extracted into two 1024-dimensional abstract feature representations. S4: Use the star topology analysis method to construct a graph structure and represent the features in the form of a graph to represent the mutual influence between the modes; S5: Establish a two-layer GCN model and train the model on the graph structure dataset. After the training is completed, use the model to monitor the bearing status in real time.

2. The device according to claim 1, characterized in that: The data acquisition unit includes a data transmission component and a data acquisition module; The data transmission component is used to collect vibration data generated by bearing rotation using the RS485 temperature and vibration sensor and transmit it to the data acquisition module via the RS485 to USB converter; The data acquisition module is used to realize the communication between the sensor device and the data acquisition part, and receives data from the data transmission component through the RS485 to USB device; The data acquisition module also forwards the received data to the host computer, and the host computer can send an inquiry request to the data acquisition module through the data forwarding module in the data forwarding unit, and the data acquisition module responds to the request.

3. The device according to claim 1, characterized in that: The data forwarding unit includes a network configuration module, a data forwarding module, a data sampling module, a data storage module, a data visualization display module, and a message mapping module; The network configuration module completes the interconnection configuration between the server network and the client network, ensures that the data forwarding module can send query instructions to the data acquisition module in real time, and enables the data acquisition module to respond to data to the data forwarding module immediately; After completing the network configuration, the data forwarding module can send a query instruction to the data acquisition module, and the data acquisition module will respond to the data forwarding module with the collected data; The data sampling module is used by the client to configure the data sampling rate of the server. It controls the rate at which the data forwarding module sends query instructions through three different sampling frequencies: fast, medium, and slow. The higher the sampling rate, the greater the amount of data collected per unit time. The data storage module is used to store the data obtained by sampling at different sampling frequencies locally on the client, and the stored data needs to be cleaned to filter out low-quality data for data fusion analysis; The data visualization module is used to dynamically visualize the data obtained by sampling at different sampling frequencies, and these data can be mapped to messages through dynamic line graphs; The message mapping module is used to map data into a dynamic line chart.

4. The device according to claim 1, characterized in that: The data fusion analysis unit includes a data cleaning module, a data conversion module, a feature extraction module, a feature fusion module, and a model training module; The data cleaning module is used to clean the data stored locally, remove low-quality data at the beginning and end of the vibration, and retain high-quality vibration data when the bearing is fully rotated; The data conversion module is used to convert the cleaned time series data into visual data; The feature extraction module uses ResNet152 and Wide_Resnet50-2 to extract features from the visual data converted by the data conversion module; The feature fusion module constructs a graph structure dataset to prepare for feature fusion training based on the GCN model for different features extracted by the feature extraction module; The model training module is trained using a graph structure dataset constructed by a feature fusion module.

5. The device according to claim 1, characterized in that: The specific steps of S1 are as follows: The first step is to divide the time series data into three equal parts and take the middle part for calculation; The second step is to calculate the lower margin of the middle part of the data based on the IQR method. The specific method is: Sort the middle data in ascending order according to the single mode. Taking the X-axis speed as an example, calculate the first quartile Q1 and the third quartile Q3. When the middle data is divided into 2n or 2n+1 values; The first quartile Q1 is the median of n values from small to large; The third quartile Q3 is the median of n values from large to small; The calculation formula for the interquartile range (IQR) is as follows: IQR=Q3-Q1 The formula for calculating the lower bound is as follows: lower_bound = Q1 - 1.5 × IQR Where lower_bound represents the lower boundary value; If the calculated lower boundary value is less than 0, then the lower boundary is changed to the smallest positive number greater than 0; then, follow this step to obtain the lower boundary value of each mode; The third step is to clean the data at both ends. The first part is cleaned in order from the beginning to the end of the time. If any modal value is less than the lower boundary of the mode, this row of data is deleted; when a row of data appears and all data are greater than the lower boundary value, the cleaning is stopped; the third part is cleaned in order from the end to the beginning of the time. If any modal value is less than the lower boundary of the mode, this row of data is deleted; when a row of data appears and all data are greater than the lower boundary value, the cleaning is stopped.

6. The device according to claim 1, characterized in that: The specific steps of S2 are as follows: The first step is to use the Min-Max scaling method to scale the cleaned time series data to [-1, 1]. The calculation formula is as follows: Among them, X is the value of the modal data, X min is the minimum value of a series of time series data of this mode, X max is the maximum value of a series of time series data of this mode; The second step is to map the time series data X′ processed by the Min-Max scaling method to polar coordinates. The specific steps are as follows: Time series data consists of two parts: the time series value and its corresponding timestamp. These two variables are mapped to angles and radii. Assuming that the time series consists of N timestamps and the corresponding value x, GAF uses arccos(x) to map the time series value to angles in the range [0, π]. Next, the time interval [0, 1] is divided into N equal parts, resulting in N+1 points {0, ..., i, ..., 1}. Zero is then discarded and these points are continuously associated with the time series. The functional relationship formula for all algorithm parameters is as follows: In the formula, x i is a single data of a series of time series data X′ just processed, such as x1 is the first data in X′, that is, i∈N; The third step is to obtain θ i ,r i Construct the Gram-like matrix G. The specific steps are as follows: The entire encoding is treated as a combination of bijective functions, which losslessly maps the time series to polar coordinates while preserving long-term dependencies through the r coordinate. Due to the limitations of the dot product in the two-dimensional polar coordinate space, the GAF method customizes a dot product-like operation; this operation only depends on the angle to explain the individual and joint information at two angles. The calculation formula is as follows: Where θ1 and θ2 are the angles after encoding x1 and x2; construct the following Gram-like matrix G, and the calculation formula is as follows: Successfully encode time series data into images through G; The fourth step is to classify different modes according to their correlation with other modes. Since the Gram-like matrix G constructed by a single mode is a one-dimensional n-order square matrix, it is regarded as a single-channel matrix storing grayscale images. In the industrial bearing fault classification task, it is stipulated that: (1) It is stipulated that single-modal data such as temperature, which has no strong correlation with other modes, is mapped into a single-channel grayscale image, and its channel is copied twice to obtain three copies of the three-channel matrix of the same data and input them into the ResNet152 and Wide_resnet50-2 models; (2) It is stipulated that the three-axis modal data is mapped into an RGB image; specifically, the Gram-like matrix constructed by the X-axis velocity is regarded as the R channel data, the Gram-like matrix constructed by the Y-axis velocity is regarded as the G channel data, and the Gram-like matrix constructed by the Z-axis velocity is regarded as the B channel data. The matrices with three separate channels are merged to obtain the storage form of the RGB image.

7. The device according to claim 1, characterized in that: The steps for extracting features using the ResNet152 model in S3 are: Because the feature extraction algorithm is based on the PyTorch deep learning framework, when the input image size is not 224×224, the image will be adaptively resized to 224×224; Step 1: The data passes through the first convolutional layer Conv1, which uses a 7×7 convolution kernel, meaning that each convolution operation processes a 7×7 pixel block. This convolutional layer generates 64 output channels, or 64 feature maps, with a final output size of 112×112. Step 2: The data passes through a 3×3 max pooling layer with a stride of 2 to reduce the spatial size of the feature map while retaining the salient features of each region. Next, the data passes through the second convolutional layer, Conv2, which constructs the bottleneck residual module, where each module consists of the following three convolutional layers: A 1×1 convolutional layer with 64 output channels for dimensionality reduction; A 3×3 convolutional layer with 64 output channels for feature extraction; A 1×1 convolutional layer with 256 output channels for dimensionality increase; This module is repeated 3 times; the final output size is 56×56; Step 3: The data passes through the third convolutional layer Conv3, which builds the bottleneck residual module, where each module consists of the following 3 convolutional layers: A 1×1 convolutional layer with 128 output channels for dimensionality reduction; A 3×3 convolutional layer with 128 output channels for feature extraction; A 1×1 convolutional layer with 512 output channels for dimensionality increase; This module is repeated 8 times; the final output size is 28×28; Step 4: The data passes through the fourth convolutional layer Conv4, which builds the bottleneck residual module, where each module consists of the following three convolutional layers: A 1×1 convolutional layer with 256 output channels for dimensionality reduction; A 3×3 convolutional layer with 256 output channels for feature extraction; A 1×1 convolutional layer with 1024 output channels for dimensionality increase; This module is repeated 36 times; the final output size is 14×14; Step 5: The data passes through the fourth convolutional layer Conv5, which builds the bottleneck residual module, where each module consists of the following 3 convolutional layers: A 1×1 convolutional layer with 512 output channels for dimensionality reduction; A 3×3 convolutional layer with 512 output channels for feature extraction; A 1×1 convolutional layer with an output channel of 2048 for dimensionality increase; This module is repeated 3 times; the final output size is 7×7; Step 6: The data passes through the average pooling layer to obtain 1024-dimensional abstract features; the model has a total of 152 layers; The difference between the Wide_ResNet50-2 model and the ResNet152 model is that the number of output channels of the 3×3 convolutional layer of the bottleneck residual block of the Wide_ResNet50-2 is twice that of the original ResNet50, and the model has a total of 50 layers; therefore, the steps for extracting features using the Wide_ResNet50-2 model are the same as those for the ResNet152 model, and the dimensions of the output features are the same.

8. The device according to claim 1, characterized in that: In S4, the star topology analysis method is used to construct a graph structure, and the features are represented in the form of a graph to represent the mutual influence between the modes; specifically, the following are included: 4.

1. Construct a single-modal star subgraph, connecting a central vertex to two leaf vertices, with no edges connecting the leaf vertices. According to the regulations, there is one temperature modal data set and three triaxial modal data sets, so four single-modal star subgraphs need to be constructed. 4.

2. Assign the two 1024-dimensional features of one modality extracted by the two pre-trained models to the two leaf vertices. Assign the zero vector with the same dimension as the leaf vertices as the vertex feature of the center vertex. Follow this step to assign features to the four subgraphs in turn. 4.

3. Perform a full connection operation on the central vertices of the four unimodal star-shaped subgraphs. That is, except for the leaf vertices, the subgraph formed by the central vertices is an undirected complete graph.

9. The device according to claim 1, characterized in that: In S5, building a two-layer GCN model includes the following steps: A. Set up a two-layer GCN model: Based on the GCN feature fusion algorithm, the model sets two GCN layers; B. Feature absorption of the first GCN layer: Each central vertex in the first GCN layer only absorbs the features of its leaf vertices; C. Feature absorption of the second GCN layer: In the second GCN layer, each central vertex not only absorbs the features of its leaf vertices, but also absorbs the features of other central vertices that have been fused with their leaf vertices. D. Calculate the layered propagation rules and calculate the feature updates of each layer of GCN; The calculation formula is as follows: in, is the adjacency matrix of the undirected graph G with self-connection added; I N is the identity matrix; W (l) Yes th The trainable weight matrix of the layer, ReLU(·)=max(0,·); Yes th The feature matrix after layer activation; H (0) =X, is the node feature initially input; E. Consider node neighbor information: Each iteration considers the node's neighbor information, allowing the model to gradually understand the local and global structure of the graph; F. The central vertex absorbs all node information: The central vertex can absorb information from all nodes in the second iteration; G. Feature fusion: After completing feature fusion, consider all attributes in the graph; H. Average pooling is used for image-level classification: Take the average value of the node features in the last layer of GCN as the input of the classification layer. The calculation formula is as follows: I. Use the cross entropy loss function to calculate the loss of the model: x G The logits tensor x and the corresponding label y obtained by the classification layer, the loss function uses the cross entropy loss, and the calculation formula is as follows: J. Packaging training samples: In one round of propagation, n training samples are packaged; the model propagates forward once and obtains n C-dimensional logits tensors x = {x1 (c) , x2 (c) , x3 (c) ,…,x n (c) }, C is the number of categories; K, classification weight and label: The weights of the C classes are ω={ω1,ω2,ω3,…,ω c }, the labels of n samples are y={y1,y2,y3,…,y n },in Represents the value of dimension j in the i-th sample, y i Represents the label corresponding to the i-th sample; Represents y i The weight corresponding to the class; L. Use the ADAM optimizer to perform adaptive learning rate adjustment by combining momentum optimization, and then use the first and second moments of the gradient to calculate the discrete universal learning rate of various parameters.

Images