Rolling bearing fault diagnosis method and device based on GCN-ResNet

Through the GCN-ResNet model and the Gram angle field algorithm, the multimodal data fusion problem is solved, the accuracy and real-time nature of bearing state monitoring are improved, and efficient fusion and accurate analysis of multimodal data are realized.

CN118936891AActive Publication Date: 2024-11-12LIAONING JINLU GENERAL AVIATION CO LTD

Patent Information

Application Number
CN202411284760.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-11-12
Estimated Expiration
2044-09-13

AI Technical Summary

Technical Problem

In an intelligent manufacturing environment, the heterogeneity and inconsistency of multimodal data lead to data conflicts and redundancy, which is difficult to effectively integrate, affecting the accuracy and real-time nature of equipment status monitoring.

Method used

Using the rolling bearing fault diagnosis method based on GCN-ResNet, the time series data is encoded as an image by using the Gram angle field algorithm, and features are extracted in combination with the ResNet152 and Wide_Resnet50-2 models, and features are performed through the GCN model. Fusion and training to achieve real-time monitoring of bearing status.

Benefits of technology

It improves the real-time and accuracy of bearing status monitoring, solves the problems of low accuracy of single-modal diagnosis and high model complexity, and realizes efficient fusion and accurate analysis of multimodal data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118936891A_ABST
    Figure CN118936891A_ABST
Patent Text Reader

Abstract

The invention relates to industrial bearing state monitoring, in particular to a rolling bearing fault diagnosis method and device based on GCN-ResNet. Aiming at the problems of poor equipment state diagnosis capability of data fusion, low equipment state diagnosis accuracy in a single mode, high model complexity and the like in an industrial Internet of Things application scene, the real-time performance and accuracy of bearing state monitoring are improved. The rolling bearing fault diagnosis device based on the GCN-ResNet comprises a data acquisition unit, a data forwarding unit and a data fusion analysis unit, the data acquisition unit is used for realizing acquisition of multi-modal data of the sensor and transmitting the data to the server; the data forwarding unit is responsible for cooperative communication between the server and the client; and the data fusion analysis unit is used for fusing and analyzing the data of the multi-mode sensor and monitoring the state of the bearing in real time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to industrial bearing condition monitoring, and more particularly to a rolling bearing fault diagnosis method and device based on GCN-ResNet. Background Art

[0002] At present, it has become a trend to promote the development of manufacturing industry towards digitalization, networking, intelligence, high-end and green. By promoting new industrialization and developing digital economy, the development of artificial intelligence technology will be accelerated to promote the transformation and upgrading of traditional manufacturing industry to intelligent manufacturing. Intelligent manufacturing obtains key data in real time through sensors and transmits it to the host computer for data fusion analysis. The purpose is to timely detect anomalies, optimize production processes, predict equipment failures, and thus improve production efficiency and product quality. The Industrial Internet of Things deeply integrates industrial manufacturing with technologies such as the Internet of Things, artificial intelligence, cloud computing and big data analysis, realizing the interconnection of equipment, production lines and management systems, and promoting the development of intelligent manufacturing.

[0003] In the intelligent manufacturing environment, the number of advanced manufacturing equipment and sensing devices deployed in industrial sites is increasing, the industrial network protocols are complex and diverse, and the data has multimodal characteristics such as large data volume, low value density, multi-source and multi-dimensional. Due to factors such as complex production environment and limited equipment energy consumption, the traditional single-modal analysis method will have key state data collection missing and contain a large amount of errors and redundant information, resulting in distortion of the actual state curve of the equipment and low data reliability, making it difficult to fully monitor the changing characteristics of the physical world. Multimodal data analysis methods take into account multiple aspects of information and can more accurately reflect the true state of the equipment, but multimodal data is difficult to effectively fuse, resulting in increased processing and analysis complexity, which in turn affects the accuracy of real-time monitoring and decision-making. In addition, the heterogeneity and inconsistency of multimodal data may cause data conflicts and redundancy, further increasing the difficulty of data cleaning and fusion. These problems make it more difficult to fully reflect the true state and changing characteristics of equipment through multimodal data in a complex industrial environment, bringing challenges to the implementation of intelligent manufacturing. Therefore, how to effectively integrate multimodal data, improve data accuracy and reliability, and then monitor the processing process in real time and ensure that the perception data maps the current physical state of the equipment with high quality has become an important issue that needs to be urgently addressed in the field of intelligent manufacturing. Summary of the invention

[0004] The present invention aims to solve the defects of the prior art and provide a rolling bearing fault diagnosis method and device based on GCN-ResNet. It improves the real-time and accuracy of bearing status monitoring in industrial Internet of Things application scenarios, such as poor equipment status diagnosis capability of data fusion, low accuracy of equipment status diagnosis with a single modality, and high model complexity.

[0005] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a rolling bearing fault diagnosis device based on GCN-ResNet, including a data acquisition unit, a data forwarding unit and a data fusion analysis unit; the data acquisition unit is used to realize the collection of multimodal data of the sensor and transmit the data to the server; the data forwarding unit is responsible for the collaborative communication between the server and the client; the data fusion analysis unit is used to fuse and analyze multimodal sensor data, and is used to monitor the bearing status in real time.

[0006] Furthermore, the data acquisition unit includes a data transmission component and a data acquisition module.

[0007] The data transmission component is used to collect vibration data generated by bearing rotation using an RS485 temperature and vibration sensor, and transmit the data to a data acquisition module via an RS485 to USB converter.

[0008] The data acquisition module is used to realize the communication between the sensor device and the data acquisition part, and receives the data from the data transmission component through the RS485 to USB device.

[0009] The data acquisition module also forwards the received data to the host computer, and the host computer can send an inquiry request to the data acquisition module through the data forwarding module in the data forwarding unit, and the data acquisition module responds to the request, thereby realizing the interaction between the data acquisition module and the data forwarding part to ensure the real-time performance of the sensor's multimodal data transmission.

[0010] Furthermore, the data forwarding unit includes a network configuration module, a data forwarding module, a data sampling module, a data storage module, a data visualization display module, and a message mapping module.

[0011] The network configuration module completes the interconnection configuration between the server network and the client network, ensures that the data forwarding module can send query instructions to the data acquisition module in real time, and enables the data acquisition module to respond to data to the data forwarding module immediately.

[0012] After completing the network configuration, the data forwarding module can send a query instruction to the data acquisition module, and the data acquisition module responds to the data forwarding module with the collected data, thereby realizing real-time communication between the server and the client.

[0013] The data sampling module is used by the client to configure the data sampling rate of the server, and controls the rate at which the data forwarding module sends query instructions through three different sampling frequencies: fast, medium, and slow. The higher the sampling rate, the greater the amount of data collected per unit time.

[0014] The data storage module is used to store data obtained by sampling at different sampling frequencies locally on the client, and the stored data needs to be cleaned to filter out low-quality data for data fusion analysis.

[0015] The data visualization module is used to dynamically visualize the data obtained by sampling at different sampling frequencies. These data can be message mapped through a dynamic line chart to timely reflect data changes.

[0016] The message mapping module is used to map the data into a dynamic line graph, which is convenient for researchers to observe the changes in vibration data in real time, and can also reflect the connection status between the server and the client and analyze the network connection status.

[0017] Furthermore, the data fusion analysis unit includes a data cleaning module, a data conversion module, a feature extraction module, a feature fusion module, and a model training module.

[0018] The data cleaning module is used to clean the data stored locally, remove low-quality data at the beginning and end stages of vibration, and retain high-quality vibration data when the bearing is fully rotated.

[0019] The data conversion module is used to convert the cleaned time series data into visual data.

[0020] The feature extraction module uses ResNet152 and Wide_Resnet50-2 to extract features from the visual data converted by the data conversion module.

[0021] The feature fusion module constructs a graph structure data set to prepare for feature fusion training based on the GCN (graph convolutional network) model for different features extracted by the feature extraction module.

[0022] The model training module is trained using a graph structure data set constructed by a feature fusion module to achieve the purpose of accurately classifying bearing fault types.

[0023] Furthermore, the fault diagnosis method is based on a data fusion analysis unit and specifically comprises the following steps:

[0024] S1: Data cleaning:

[0025] The collection time does not exceed three minutes to ensure that the working time of the faulty bearing is not too long.

[0026] The data stored locally is cleaned to delete the data with insufficient vibration and retain the data when the bearing is fully vibrating, so as to extract the characteristics of the vibration data under normal working conditions.

[0027] S2: Data encoding into images: The cleaned data is encoded into images based on the GAF (Gram Angular Field) algorithm; the sensor vibration signal is converted into image data to meet the requirements of feature extraction of the pre-trained model; including:

[0028] Data type identification: Identify the ten types of multimodal data collected by the temperature and vibration sensor, including temperature modal data and triaxial modal data; among them, triaxial modal data includes triaxial (X, Y, Z) velocity, triaxial displacement, and triaxial acceleration.

[0029] Data mapping: The temperature modal data is mapped to a grayscale image, and the three-axis modal data is mapped to an RGB image, and finally four image data are obtained.

[0030] S3: The four encoded image data are respectively subjected to feature extraction using ResNet152 and Wide_Resnet50-2. Each image data is extracted into two 1024-dimensional abstract feature representations.

[0031] S4: Use the star topology analysis method to construct a graph structure and represent the features in the form of a graph to represent the mutual influence between the modes.

[0032] S5: Establish a two-layer GCN (graph convolutional network) model and train the model on the graph structure data set. After the training is completed, the model is used to monitor the bearing status in real time.

[0033] Furthermore, since the acquisition time of the faulty bearing does not exceed three minutes, insufficient rotation of the bearing may occur each time sampling starts and ends. In this case, the data stored locally needs to be cleaned to delete the insufficient vibration data.

[0034] The first step is to divide the time series data into three equal parts and take the middle part for calculation.

[0035] The second step is to calculate the lower margin of the middle part of the data based on the IQR (interquartile range) method. The specific method is:

[0036] The middle part of the data is sorted in ascending order according to the single mode. Taking the X-axis speed as an example, the first quartile Q1 and the third quartile Q3 are calculated. When the middle data is divided into an even number (2n) or an odd number (2n+1) of values.

[0037] The first quartile Q1 is the median of n values ​​from small to large. The third quartile Q3 is the median of n values ​​from large to small.

[0038] The calculation formula for the interquartile range (IQR) is as follows:

[0039] IQR=Q3-Q1

[0040] The calculation formula for the lower boundary is as follows:

[0041] lower_bound = Q1-1.5×IQR

[0042] Where lower_bound represents the lower boundary value;

[0043] If the calculated lower boundary value is less than 0, the lower boundary is changed to the smallest positive number greater than 0; then, the lower boundary value of each mode is obtained according to this step.

[0044] The third step is to clean the data at both ends. The first part is cleaned in order from the beginning to the end of the time. If any modal value is less than the lower boundary of the mode, delete this row of data; when a row of data appears that satisfies all data greater than the lower boundary value, stop cleaning; the third part is cleaned in order from the end to the beginning of the time. If any modal value is less than the lower boundary of the mode, delete this row of data; when a row of data appears that satisfies all data greater than the lower boundary value, stop cleaning.

[0045] Furthermore, the specific steps of S2 are as follows:

[0046] The first step is to use the Min-Max scaling method to scale the cleaned time series data to [-1, 1]. The calculation formula is as follows:

[0047]

[0048] Among them, X is the value of the modal data, X min is the minimum value of a series of time series data of this mode, X max It is the maximum value of a series of time series data of this mode.

[0049] The second step is to map the time series data X′ processed by the Min-Max scaling method to polar coordinates. The specific steps are as follows:

[0050] Time series data consists of two parts: the value of the time series and its corresponding timestamp. These two variables are mapped to angle and radius. Assuming that the time series consists of N timestamps and the corresponding value x, GAF uses arccos(x) to map the value of the time series to angles. The range of angle values ​​is [0, π]. Next, the time interval [0, 1] is divided into N equal parts to obtain N+1 points {0, ..., i, ..., 1}. Then, 0 is discarded and these points are continuously associated with the time series. The functional relationship formula of all algorithm parameters is as follows:

[0051]

[0052] In the formula, x i It is a single data of a series of time series data X′ just processed, such as x1 is the first data in X′, that is, i∈N.

[0053] The third step is to obtain θ i , r i Construct the Gram-like matrix G. The specific steps are as follows:

[0054] The entire encoding is regarded as a combination of bijective functions, which losslessly maps the time series to polar coordinates while maintaining long-term dependencies through the r coordinates. Due to the limitations of the dot product in the two-dimensional polar coordinate space, the GAF method customizes a dot product-like operation. This operation depends only on the angle to explain the single and joint information at two angles. The calculation formula is as follows:

[0055]

[0056] Where θ1 and θ2 are the angles after encoding x1 and x2; then we can construct the following Gram-like matrix G, and the calculation formula is as follows:

[0057]

[0058] G successfully encodes time series data into images, making it easier to extract abstract features of the images using the pre-trained model.

[0059] The fourth step is to classify different modes according to their correlation with other modes. Since the Gram-like matrix G constructed by a single mode is a one-dimensional n-order square matrix, it is regarded as a single-channel matrix storing grayscale images. In the task of industrial bearing fault classification, it is stipulated that:

[0060] (1) It is stipulated that single-mode data such as temperature, which has no strong correlation with other modes, is mapped into a single-channel grayscale image, and its channel is copied twice to obtain three three-channel matrices of the same data and input them into the ResNet 152 and Wide_resnet50-2 models to match the input data type of the model.

[0061] (2) It is stipulated that for three-axis modal data, such as three-axis speed, this type of data with three modes and a strong correlation between the three modes is mapped into an RGB image; specifically, the Gram-like matrix constructed by the X-axis speed is regarded as the R channel data, the Gram-like matrix constructed by the Y-axis speed is regarded as the G channel data, and the Gram-like matrix constructed by the Z-axis speed is regarded as the B channel data, and the matrices with three separate channels are merged to obtain the storage form of the RGB image; in this way, this three-channel matrix can be input into the ResNet152 and Wide_resnet50-2 models to match the input data type of the model.

[0062] Furthermore, the steps for extracting features using the ResNet152 model in S3 are:

[0063] Because the feature extraction algorithm is based on the PyTorch deep learning framework, when the input image size is not 224×224, the image will be adaptively resized to 224×224.

[0064] Step 1: The data passes through the first convolution layer Conv1, which uses a 7×7 convolution kernel, which means that each convolution operation processes an area of ​​7×7 pixel blocks; this convolution layer generates 64 output channels, that is, 64 feature maps, and the final output size is 112×112.

[0065] Step 2: The data passes through a 3×3 maximum pooling layer with a stride of 2 to reduce the spatial size of the feature map and retain the salient features of each region; then, the data passes through the second convolutional layer Conv2, which constructs a bottleneck residual module, each of which consists of the following 3 convolutional layers.

[0066] A 1×1 convolutional layer with 64 output channels is used for dimensionality reduction.

[0067] A 3×3 convolutional layer with 64 output channels is used for feature extraction.

[0068] A 1×1 convolutional layer with an output channel number of 256 is used for dimensionality increase.

[0069] This module is repeated 3 times; the final output size is 56×56.

[0070] Step 3: The data passes through the third convolutional layer Conv3, which builds the bottleneck residual module, each of which consists of the following 3 convolutional layers:

[0071] A 1×1 convolutional layer with 128 output channels is used for dimensionality reduction.

[0072] A 3×3 convolutional layer with 128 output channels is used for feature extraction.

[0073] A 1×1 convolutional layer with an output channel number of 512 is used for dimensionality increase.

[0074] This module is repeated 8 times; the final output size is 28×28.

[0075] Step 4: The data passes through the fourth convolutional layer Conv4, which builds the bottleneck residual module, each of which consists of the following 3 convolutional layers:

[0076] A 1×1 convolutional layer with 256 output channels is used for dimensionality reduction.

[0077] A 3×3 convolutional layer with 256 output channels is used for feature extraction.

[0078] A 1×1 convolutional layer with an output channel number of 1024 is used for dimensionality increase.

[0079] This module is repeated 36 times; the final output size is 14×14.

[0080] Step 5: The data passes through the fourth convolutional layer Conv5, which builds the bottleneck residual module, each of which consists of the following 3 convolutional layers:

[0081] A 1×1 convolutional layer with 512 output channels is used for dimensionality reduction.

[0082] A 3×3 convolutional layer with 512 output channels is used for feature extraction.

[0083] A 1×1 convolutional layer with an output channel number of 2048 is used for dimensionality increase.

[0084] This module is repeated 3 times; the final output size is 7×7.

[0085] Step 6: The data passes through the average pooling layer to obtain 1024-dimensional abstract features; the model has a total of 152 layers;

[0086] The difference between the Wide_ResNet50-2 model and the ResNet152 model is that the number of output channels of the 3×3 convolution layer of the bottleneck residual block of the Wide_ResNet50-2 is twice that of the original ResNet50, and the model has a total of 50 layers; therefore, the steps for extracting features using the Wide_ResNet50-2 model are the same as those for extracting features using the ResNet152 model, and the dimensions of the output features are the same.

[0087] The difference is the network structure of the model, the number of layers, the number of channels and the weights of the model, and the value of the obtained features is different. Figure 4 shown.

[0088] Furthermore, in S4, the star topology structure analysis method is used to construct a graph structure, and the features are represented in the form of a graph to represent the mutual influence between the modes. This is a step that must be performed before inputting the GCN. Specifically, it includes:

[0089] 4.1. Construct a single-mode star-shaped subgraph, connecting a central vertex to two leaf vertices, with no edges connecting the two leaf vertices; according to regulations, there is 1 temperature modal data and 3 triaxial modal data, so 4 single-mode star-shaped subgraphs need to be constructed.

[0090] 4.2. Assign the two 1024-dimensional features of one modality extracted by the two pre-trained models to the two leaf vertices, and assign the 0 vector with the same dimension as the leaf vertex as the vertex feature of the center vertex to the center vertex. The four subgraphs are assigned features in turn according to this step.

[0091] 4.3. Perform a full connection operation on the central vertices of the four unimodal star-shaped subgraphs, that is, except for the leaf vertices, the subgraph formed by the central vertices is an undirected complete graph.

[0092] Furthermore, in S5, establishing a two-layer GCN model includes the following steps:

[0093] A. Set up a two-layer GCN model:

[0094] Based on the GCN feature fusion algorithm, the model sets two GCN layers.

[0095] B. Feature absorption of the first GCN layer:

[0096] Each central vertex in the first GCN layer only absorbs the features of its leaf vertices.

[0097] C. Feature absorption of the second GCN layer:

[0098] Each central vertex in the second GCN layer absorbs not only the features of its leaf vertices, but also the features of other central vertices that have been fused with their respective leaf vertices.

[0099] D. Calculate the layered propagation rules and calculate the feature updates of each layer of GCN.

[0100] The calculation formula is as follows:

[0101]

[0102] in, is the adjacency matrix of the undirected graph G with self-connection added. N is the identity matrix. W (l) Yes th The trainable weight matrix of the layer, ReLU(·)=max(0,·). Yes th The feature matrix after layer activation. (0) =X, is the node feature initially passed in.

[0103] E. Consider node neighbor information: Each iteration considers the node’s neighbor information, so that the model can gradually understand the local and global structures in the graph.

[0104] F. The central vertex absorbs all node information: The central vertex can absorb the information of all nodes in the second iteration.

[0105] G. Feature fusion: After completing feature fusion, all attributes in the graph are considered.

[0106] H. Average pooling is used for graph-level classification: the average value of the node features in the last layer of GCN is taken as the input of the classification layer. The calculation formula is as follows:

[0107]

[0108] I. Use the cross entropy loss function to calculate the loss of the model:

[0109] x G The logits tensor x and the corresponding label y obtained by the classification layer, the loss function uses the cross entropy loss, and the calculation formula is as follows:

[0110]

[0111] J. Packaging training samples:

[0112] In one round of propagation, n training samples are packaged. After one round of forward propagation, n C-dimensional logits tensors x = {x1 (c) , x2 (c) , x3 (c) , …, x n (c)}, C is the number of categories;

[0113] K, classification weight and label:

[0114] The weights of the C classes are ω = {ω1, ω2, ω3, …, ω c}, the labels of n samples are y = {y1, y2, y3, ..., y n},in represents the value of dimension j in the i-th sample, y i Represents the label corresponding to the i-th sample. Represents y i The weight corresponding to the class.

[0115] I. Use the ADAM (ADaptive Moment Estimation) optimizer to perform adaptive learning rate adjustment by combining momentum optimization, and then use the first and second moments of the gradient to calculate the discrete universal learning rate of various parameters.

[0116] Compared with the prior art, the present invention has beneficial effects.

[0117] 1. The present invention uses the Gram angle field to encode the time series data of industrial sensors. This data-driven method can convert the time series information into images losslessly and retain the long-term dependency of the sequence.

[0118] 2. The present invention does not require traditional signal processing technology or image processing means to extract features, thus avoiding the subjectivity of manual feature extraction.

[0119] 3. The present invention integrates computer vision into industrial processing state monitoring and machine fault diagnosis by using transfer learning (TL) methods. The method of extracting features using pre-trained models has a better representation of images, which can also solve the sample sparsity and generalization problem in industrial processing tasks with less data.

[0120] 4. The present invention uses GCN to better capture the hidden complex relationships between different modalities through nonlinear integration of features and adaptive weight learning. Moreover, the proposed method has very low computational overhead and very high accuracy, which solves the fundamental problem of high complexity and high computational overhead of the hybrid model. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. The protection scope of the present invention is not limited to the following description.

[0122] Figure 1 It is a system structure overview diagram of the present invention.

[0123] Figure 2 It is a schematic diagram of the system structure of the present invention.

[0124] Figure 3 It is a schematic diagram of the client data acquisition system of the present invention.

[0125] Figure 4 It is a schematic diagram of data fusion analysis of the present invention. DETAILED DESCRIPTION

[0126] In order to make the purpose, technical scheme and beneficial effects of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0127] A rolling bearing fault diagnosis method and device based on GCN-ResNet, the system structure overview is shown in the figure Figure 1As shown in the figure, it includes data acquisition part, data forwarding part and data fusion analysis part. The data acquisition part realizes the multi-modal data acquisition of the sensor to the server, the data forwarding part is responsible for the collaborative communication between the server and the client, and the data fusion analysis part is used to fuse and analyze the multi-modal sensor data for real-time monitoring of the bearing status.

[0128] Figure 2 The system structure diagram of the present invention is shown. A rolling bearing fault diagnosis device based on GCN-ResNet includes a data transmission component and a data acquisition module; the data acquisition part mainly includes that the data transmission component is responsible for collecting the vibration data of the servo motor controlling the bearing rotation through the RS485 temperature vibration sensor and sending the data to the data acquisition module through the RS485 to USB converter. The data acquisition module realizes the communication between the sensor device and the transmission module, and adopts the RS485 communication mechanism to ensure the real-time transmission of the sensor multimodal data. The network configuration module of the server and the client of the data forwarding part completes the interconnection configuration of the server network and the client network to ensure that the server and the client can communicate normally, the data sampling module realizes the data sampling rate of the server configured by the client, the data forwarding module forwards the server data to the client to realize real-time communication, the data storage module realizes the data storage locally on the client, the data visualization module dynamically visualizes the data, and completes the function of timely reflecting the data changes, and the message mapping module obtains the connection status between the server and the client and analyzes the network connection status. In the data fusion and analysis part, the data cleaning module cleans the data, removes the start and end stages of vibration, and retains the vibration data of the bearing when it is fully rotated; the data conversion module converts the collected time series data into visual data, the feature extraction module extracts the features of the converted visual data, the feature fusion module uses the GCN model to fuse the multimodal data collected by the sensor, and the model training module trains the graph structure data set with fused features to achieve the purpose of classifying the bearing fault types.

[0129] The acquisition equipment includes temperature and vibration sensors, RS485 to USB converter, Raspberry Pi, Raspberry Pi display, and host computer. The sensor is deployed on the vibration bearing to collect multimodal data, and the protocol is converted through the converter so that the Raspberry Pi can successfully receive the data, and then the Raspberry Pi data is forwarded to the host computer to achieve wireless network transmission. The Raspberry Pi display is used to monitor the communication between the Raspberry Pi and the sensor in real time. The host computer analyzes the real-time data, generates a visual curve, displays the dynamic changes of the real-time data, and stores the data for data fusion analysis.

[0130] Figure 3The client data acquisition system schematic diagram of the present invention is shown. A real-time data acquisition software based on PyQt, which collects multimodal data from sensors and displays them visually, monitors the connection status between the server and the client in real time, and provides feedback through human-computer interaction. First, enter the IP address of the Raspberry Pi, open the listening port number in the server data acquisition system on the Raspberry Pi, select the serial port number of the RS485 to USB converter, set the device baud rate, and connect the client to the server. The stable result shows the bearing vibration data in this state, the Z-axis speed spectrum shows the real-time parameters at different speeds, and the data visualization shows the real-time changes of the temperature, three-axis acceleration, three-axis speed, and three-axis displacement of the sensor in the stable state. The communication message shows the client sending the query instruction, returning the data in the sensor register, and the process of the Raspberry Pi responding through the wireless network.

[0131] Figure 4 The data fusion analysis schematic diagram of the present invention is shown, and the specific process is as follows:

[0132] First, the time series is scaled to [-1, 1] using the Min-Max scaling method.

[0133]

[0134] It is necessary to consider the value of the time series and its corresponding timestamp, which are represented by angle and radius respectively. Assuming that the time series consists of N timestamps and the corresponding value x, GAF uses arccos(x) to map the value to an angle, and the range of the angle value is [0, π]. Next, divide the time interval [0, 1] into N equal parts, we get N+1 points {0, ..., i, ..., 1}, then discard 0 and continuously associate these points to the time series. The functional relationship formula of all algorithm parameters is as follows:

[0135]

[0136] The entire encoding is a combination of bijective functions, which losslessly maps the time series to polar coordinates while maintaining long-term dependencies through the r coordinate. Due to the limitations of the dot product in the two-dimensional polar coordinate space, GAF customizes a dot product-like operation. This operation depends only on the angle to explain the single and joint information at two angles. The calculation formula is as follows:

[0137]

[0138] Where θ1 and θ2 are the angles after encoding x1 and x2. So we can construct the following Gram-like matrix G, and the calculation formula is as follows:

[0139]

[0140] The time series data is successfully encoded into an image through G. At this time, the image is used to extract features using ResNet152 and Wide_ResNet50-2 respectively.

[0141] The ResNet152 network structure is as follows Figure 4 As shown in (a), the first layer includes a convolution kernel of size 7×7, a channel number of 64, and a stride of 2. After a 3×3 maximum pooling layer with a stride of 2, the second layer includes 3 convolution kernels of size 1×1, a channel number of 64, a convolution kernel of size 3×3, a channel number of 64, a 1×1 convolution kernel, and a channel number of 256. The third layer includes 8 convolution kernels of size 1×1, a channel number of 128, a convolution kernel of size 3×3, a channel number of 128, a 1×1 convolution kernel, and a channel number of 256. The first layer is a convolutional layer with a convolution kernel and 512 channels. The fourth layer is a convolutional layer with 36 convolution kernels of size 1×1 and 256 channels, a convolution kernel of size 3×3 and 256 channels, and a convolution kernel of 1×1 and 1024 channels. The fifth layer is a convolutional layer with 3 convolution kernels of size 1×1 and 512 channels, a convolution kernel of size 3×3 and 512 channels, and a convolution kernel of 1×1 and 2048 channels. The last layer is the average pooling layer excluding the classification layer.

[0142] The Wide_ResNet50-2 network structure is as follows Figure 4 As shown in (b), the first layer includes a convolutional kernel of size 7×7, a channel number of 64, and a stride of 2. After a 3×3 maximum pooling layer with a stride of 2, the second layer includes 3 convolutional kernels of size 1×1, a channel number of 64, a convolutional kernel of size 3×3, a channel number of 128, a convolutional kernel of size 1×1, and a convolutional layer with a channel number of 256. The third layer includes 4 convolutional kernels of size 1×1, a channel number of 128, a convolutional kernel of size 3×3, a channel number of 256, and a convolutional kernel of size 1×1. The fourth layer is 6 convolution kernels of size 1×1 and the number of channels is 256, the convolution kernel of size 3×3 and the number of channels is 512, and the convolution kernel of size 1×1 and the number of channels is 1024. The fifth layer is 3 convolution kernels of size 1×1 and the number of channels is 512, the convolution kernel of size 3×3 and the number of channels is 1024, and the convolution kernel of size 1×1 and the number of channels is 2048. The last layer is the average pooling layer excluding the classification layer.

[0143] After extracting features using the two models, the star topology analysis method is used to construct a graph structure, and the features are represented in the form of a graph to represent the mutual influence of each mode. This is a necessary step before inputting GCN. First, the features of a mode extracted by n pre-trained models are assigned to n leaf vertices as vertex features of leaf vertices. The vertex feature of the central vertex is a 0 vector with the same dimension as the leaf vertex, and then the central vertices of the m star graphs are fully connected, that is, the subgraph formed by the central vertices except the leaf vertices is an undirected complete graph.

[0144] After building the graph structure, use GCN for hierarchical propagation. Construct two layers of GCN layers and consider its hierarchical propagation rules. The calculation formula is as follows:

[0145]

[0146] in, is the adjacency matrix of the undirected graph G with self-connection added. N is the identity matrix. The purpose of adding self-connection is to consider the influence of the node's own features in the process of aggregating features. W (l) Yes th The trainable weight matrix of the layer, ReLU(·)=max(0,·). Yes th The feature matrix after layer activation. (0) =X, is the node feature initially passed in. In this way, each iteration will consider the node's neighbor information, so that the model can gradually understand the local and global structure in the graph. Therefore, the central vertex can absorb the information of all nodes in the second iteration. When the feature fusion is completed, all attributes in the graph are considered. Graph-level classification means that given a graph data set, the entire graph is classified according to the attributes in the graph structure. Graph-level classification uses the average pooling method, that is, the average value of the node features in the last layer of GCN is taken as the input of the classification layer. The calculation formula is as follows:

[0147]

[0148] Finally, x G The logits tensor x and the corresponding label y obtained by the classification layer, the loss function uses the cross entropy loss, and the calculation formula is as follows:

[0149]

[0150] Usually, in one round of propagation, n training samples are packaged. After one round of forward propagation, the model obtains n C-dimensional logits tensors x = {x1 (c) , x2(c) , x3 (c) , …, x n (c)}, C is the number of categories, and the weights of C categories are ω={ω1,ω2,ω3,…,ω c}, the labels of n samples are y = {y1, y2, y3, ..., y n},in represents the value of dimension j in the i-th sample, y i Represents the label corresponding to the i-th sample. Represents y i The weight corresponding to the class.

[0151] The ADAM (ADaptive Moment Estimation) optimizer is used to perform adaptive learning rate adjustment by combining momentum optimization, and then the first and second moments of the gradient are used to calculate the discrete universal learning rate of various parameters. ADAM is a simple and computationally efficient method that involves first-order gradients and has minimal memory requirements for stochastic optimization. After training is completed, the trained GCN model is used for real-time bearing status detection.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features therein may be replaced by equivalents. Therefore, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A rolling bearing fault diagnosis device based on GCN-ResNet, Features: It includes a data collection unit, a data forwarding unit and a data fusion analysis unit; The data acquisition unit is used to acquire multimodal data from the sensor and transmit the data to the server; The data forwarding unit is responsible for the collaborative communication between the server and the client; The data fusion analysis unit is used to fuse and analyze multi-modal sensor data and to monitor the bearing status in real time.

2. The device according to claim 1, Features: The data acquisition unit includes a data transmission component and a data acquisition module; The data transmission component is used to collect vibration data generated by bearing rotation using an RS485 temperature and vibration sensor, and transmit the data to a data acquisition module via an RS485 to USB converter; The data acquisition module is used to realize the communication between the sensor device and the data acquisition part, and receives the data from the data transmission component through the RS485 to USB device; The data acquisition module also forwards the received data to the host computer, and the host computer can send an inquiry request to the data acquisition module through the data forwarding module in the data forwarding unit, and the data acquisition module responds to the request, thereby realizing the interaction between the data acquisition module and the data forwarding part to ensure the real-time performance of the sensor's multimodal data transmission.

3. The device according to claim 1, Features: The data forwarding unit includes a network configuration module, a data forwarding module, a data sampling module, a data storage module, a data visualization display module, and a message mapping module; The network configuration module completes the interconnection configuration between the server network and the client network, ensures that the data forwarding module can send query instructions to the data acquisition module in real time, and enables the data acquisition module to respond to data to the data forwarding module immediately; After completing the network configuration, the data forwarding module can send a query instruction to the data acquisition module, and the data acquisition module will respond to the data forwarding module with the collected data, thereby realizing real-time communication between the server and the client; The data sampling module is used for the client to configure the data sampling rate of the server, and controls the rate at which the data forwarding module sends the query instruction through three different sampling frequencies: fast, medium and slow. The higher the sampling rate, the greater the amount of data collected per unit time; The data storage module is used to store the data obtained by sampling at different sampling frequencies locally on the client, and the stored data needs to be cleaned to filter out low-quality data for data fusion analysis; The data visualization module is used to dynamically visualize the data obtained by sampling at different sampling frequencies. These data can be mapped through dynamic line graphs to reflect data changes in a timely manner. The message mapping module is used to map the data into a dynamic line graph, which is convenient for researchers to observe the changes in vibration data in real time, and can also reflect the connection status between the server and the client and analyze the network connection status.

4. The device according to claim 1, Features: The data fusion analysis unit includes a data cleaning module, a data conversion module, a feature extraction module, a feature fusion module, and a model training module; The data cleaning module is used to clean the data stored locally, remove low-quality data at the beginning and end of the vibration, and retain high-quality vibration data when the bearing is fully rotated; The data conversion module is used to convert the cleaned time series data into visual data; The feature extraction module uses ResNet152 and Wide_Resnet50-2 to extract features from the visual data converted by the data conversion module; The feature fusion module constructs a graph structure data set to prepare for feature fusion training based on the GCN model for different features extracted by the feature extraction module; The model training module is trained using a graph structure data set constructed by a feature fusion module to achieve the purpose of accurately classifying bearing fault types.

5. A rolling bearing fault diagnosis method based on GCN-ResNet, Features: The fault diagnosis method is based on the data fusion analysis unit of any one of claims 1 to 4, and specifically comprises the following steps: S1: Data cleaning: The acquisition time should not exceed three minutes to ensure that the faulty bearing does not work for too long; Clean the data stored locally, delete the data with insufficient vibration, and retain the data with sufficient vibration of the bearing, so as to extract the characteristics of the vibration data under normal working conditions; S2: Data encoding into images: The cleaned data is encoded into images based on the GAF algorithm; the sensor vibration signal is converted into image data to meet the requirements of feature extraction of the pre-trained model; including: Data type identification: Identify ten types of multimodal data collected by temperature and vibration sensors, including temperature modal data and triaxial modal data; triaxial modal data includes triaxial velocity, triaxial displacement, and triaxial acceleration; Data mapping: The temperature modal data is mapped to a grayscale image, and the triaxial modal data is mapped to an RGB image, and finally four image data are obtained; S3: The four encoded image data are respectively subjected to feature extraction using ResNet152 and Wide_Resnet50-2. Each image data is extracted into two 1024-dimensional abstract feature representations. S4: Use the star topology analysis method to construct a graph structure and represent the features in the form of a graph to represent the mutual influence between the modes; S5: Establish a two-layer GCN model and train the model on the graph structure data set. After the training is completed, the model is used to monitor the bearing status in real time.

6. The method according to claim 5, Features: The specific steps of S1 are as follows: Since the sampling time of the faulty bearing does not exceed three minutes, the bearing may not rotate sufficiently each time the sampling starts and ends. In this case, the data stored locally needs to be cleaned to delete the insufficient vibration data. The first step is to divide the time series data into three equal parts and take the middle part for calculation; The second step is to calculate the lower margin of the middle part of the data based on the IQR method. The specific method is: Sort the middle part of the data in ascending order according to the single mode. Taking the X-axis speed as an example, calculate the first quartile Q 1 and the third quartile Q 3 , when the intermediate data is divided into 2n or 2n+1 values; First quartile Q 1 is the median of n values ​​from small to large; The third quartile Q 3 is the median of n values ​​from large to small; The calculation formula for the interquartile range (IQR) is as follows: IQR / Q 3 -Q 1 The calculation formula for the lower boundary is as follows: lower_bound=Q 1 -1.5×IQR Where lower_bound represents the lower boundary value; If the calculated lower boundary value is less than 0, the lower boundary is changed to the smallest positive number greater than 0; then, the lower boundary value of each mode is obtained according to this step; The third step is to clean the data at both ends. The first part is cleaned in order from the beginning to the end of the time. If any modal value is less than the lower boundary of the mode, delete this row of data; when a row of data appears that satisfies all data greater than the lower boundary value, stop cleaning; the third part is cleaned in order from the end to the beginning of the time. If any modal value is less than the lower boundary of the mode, delete this row of data; when a row of data appears that satisfies all data greater than the lower boundary value, stop cleaning.

7. The method according to claim 5, Features: The specific steps of S2 are as follows: The first step is to use the Min-Max scaling method to scale the cleaned time series data to [-1, 1]. The calculation formula is as follows: Among them, X is the value of the modal data, X min is the minimum value of a series of time series data of this mode, X max It is the maximum value of a series of time series data of this mode; The second step is to map the time series data X′ processed by the Min-Max scaling method to polar coordinates. The specific steps are as follows: Time series data consists of two parts: the value of the time series and its corresponding timestamp. These two variables are mapped to angle and radius. Assuming that the time series consists of N timestamps and the corresponding value x, GAF uses arCCos(x) to map the value of the time series to angles, and the range of angle values ​​is [0, π]. Next, the time interval [0, 1] is divided into N equal parts to obtain N+1 points {0, ..., i, ..., 1}, and then 0 is discarded and these points are continuously associated with the time series. The functional relationship formula of all algorithm parameters is as follows: In the formula, x i is a single data of a series of time series data X' just processed, such as x 1 is the first data in X′, i∈N; The third step is to obtain θ i , r i Construct the Gram-like matrix G. The specific steps are as follows: The entire encoding is regarded as a combination of bijective functions, which losslessly maps the time series to polar coordinates while maintaining long-term dependencies through the r coordinates. Due to the limitations of the dot product in the two-dimensional polar coordinate space, the GAF method customizes a dot product-like operation. This operation depends only on the angle to explain the single and joint information at two angles. The calculation formula is as follows: where θ 1 and θ 2 is x 1 and x 2 The encoded angle; then we can construct the following Gram-like matrix G, and the calculation formula is as follows: G successfully encodes time series data into images, making it easier to extract abstract features of images using pre-trained models. The fourth step is to classify different modes according to their correlation with other modes. Since the Gram-like matrix G constructed by a single mode is a one-dimensional n-order square matrix, it is regarded as a single-channel matrix storing grayscale images. In the task of industrial bearing fault classification, it is stipulated that: (1) It is stipulated that single-mode data such as temperature, which has no strong correlation with other modes, is mapped into a single-channel grayscale image, and its channel is copied twice to obtain three three-channel matrices of the same data and input them into the ResNet152 and Wide_resnet50-2 models to match the input data type of the model; (2) It is stipulated that for three-axis modal data, such as three-axis speed, this type of data with three modes and a strong correlation between the three modes is mapped into an RGB image; specifically, the Gram-like matrix constructed by the X-axis speed is regarded as the R channel data, the Gram-like matrix constructed by the Y-axis speed is regarded as the G channel data, and the Gram-like matrix constructed by the Z-axis speed is regarded as the B channel data, and the matrices with three separate channels are merged to obtain the storage form of the RGB image; in this way, this three-channel matrix can be input into the ResNet152 and Wide_resnet50-2 models to match the input data type of the model.

8. The method according to claim 1, Features: The steps to extract features using the ResNet152 model in S3 are: Because the feature extraction algorithm is based on the PyTorch deep learning framework, when the input image size is not 224×224, the image will be adaptively resized to 224×224; Step 1: The data passes through the first convolution layer Conv1, which uses a 7×7 convolution kernel, which means that each convolution operation processes a 7×7 pixel block area; this convolution layer generates 64 output channels, that is, 64 feature maps, and the final output size is 112×112; Step 2: The data passes through a 3×3 max pooling layer with a stride of 2 to reduce the spatial size of the feature map and retain the salient features of each region; then, the data passes through the second convolutional layer Conv2, which constructs the bottleneck residual module, each of which consists of the following three convolutional layers: A 1×1 convolutional layer with 64 output channels for dimensionality reduction; A 3×3 convolutional layer with 64 output channels for feature extraction; A 1×1 convolutional layer with 256 output channels for dimensionality increase; This module is repeated 3 times; the final output size is 56×56; Step 3: The data passes through the third convolutional layer Conv3, which builds the bottleneck residual module, each of which consists of the following 3 convolutional layers: A 1×1 convolutional layer with 128 output channels for dimensionality reduction; A 3×3 convolutional layer with 128 output channels for feature extraction; A 1×1 convolutional layer with an output channel number of 512 for dimensionality increase; This module is repeated 8 times; the final output size is 28×28; Step 4: The data passes through the fourth convolutional layer Conv4, which builds the bottleneck residual module, each of which consists of the following 3 convolutional layers: A 1×1 convolutional layer with 256 output channels for dimensionality reduction; A 3×3 convolutional layer with 256 output channels for feature extraction; A 1×1 convolutional layer with 1024 output channels for dimensionality increase; This module is repeated 36 times; the final output size is 14×14; Step 5: The data passes through the fourth convolutional layer Conv5, which builds the bottleneck residual module, each of which consists of the following 3 convolutional layers: A 1×1 convolutional layer with 512 output channels for dimensionality reduction; A 3×3 convolutional layer with 512 output channels for feature extraction; A 1×1 convolutional layer with an output channel number of 2048 for dimensionality increase; This module is repeated 3 times; the final output size is 7×7; Step 6: The data passes through the average pooling layer to obtain 1024-dimensional abstract features; the model has a total of 152 layers; The difference between the Wide_ResNet50-2 model and the ResNet152 model is that the number of output channels of the 3×3 convolution layer of the bottleneck residual block of the Wide_ResNet50-2 is twice that of the original ResNet50, and the model has a total of 50 layers; therefore, the steps for extracting features using the Wide_ResNet50-2 model are the same as those for extracting features using the ResNet152 model, and the dimensions of the output features are the same.

9. The method according to claim 1, Features: In S4, the star topology structure analysis method is used to construct a graph structure, and the features are represented in the form of a graph to represent the mutual influence between the modes. This is a step that must be performed before inputting into GCN. Specifically, it includes: 4.

1. Construct a single-mode star-shaped subgraph, connect a central vertex to two leaf vertices, and there is no edge connecting the two leaf vertices; according to regulations, there is 1 temperature modal data and 3 triaxial modal data, so 4 single-mode star-shaped subgraphs need to be constructed; 4.

2. Assign the two 1024-dimensional features of one modality extracted by the two pre-trained models to the two leaf vertices, and assign the 0 vector with the same dimension as the leaf vertex as the vertex feature of the center vertex. The four subgraphs are assigned features in turn according to this step; 4.

3. Perform a full connection operation on the central vertices of the four unimodal star-shaped subgraphs, that is, except for the leaf vertices, the subgraph formed by the central vertices is an undirected complete graph.

10. The method according to claim 1, Features: In S5, building a two-layer GCN model includes the following steps: A. Set up a two-layer GCN model: Based on the GCN feature fusion algorithm, the model sets two GCN layers; B. Feature absorption of the first GCN layer: Each central vertex in the first GCN layer only absorbs the features of its leaf vertices; C. Feature absorption of the second GCN layer: Each central vertex in the second GCN layer not only absorbs the features of its leaf vertices, but also absorbs the features of other central vertices that have merged their respective leaf vertices; D. Calculate the layered propagation rules and calculate the feature updates of each layer of GCN; The calculation formula is as follows: in, is the adjacency matrix of the undirected graph G with self-connection added; I N is the identity matrix; W (l) Yes th The trainable weight matrix of the layer, ReLU(·)=max(0,·); Yes th The feature matrix after layer activation; H (0) =X, is the node feature initially passed in; E. Consider node neighbor information: Each iteration considers the neighbor information of the node, so that the model can gradually understand the local and global structure of the graph; F. The central vertex absorbs all node information: The central vertex can absorb the information of all nodes in the second iteration; G. Feature fusion: After completing feature fusion, consider all attributes in the graph; H. Average pooling is used for image-level classification: Take the average value of the node features in the last layer of GCN as the input of the classification layer. The calculation formula is as follows: I. Use the cross entropy loss function to calculate the loss of the model: x G The logits tensor x and the corresponding label y obtained by the classification layer, the loss function uses the cross entropy loss, and the calculation formula is as follows: J. Packaging training samples: In one round of propagation, n training samples are packaged; the model is forward propagated once, and n C-dimensional logits tensors x = {x 1 (c) , x 3 (c) , x 3 (c) , …, x n (c) }, C is the number of categories; K, classification weight and label: The weight of C classes is ω={ω 1 ,ω 2 ,ω 3 ,…,ω c }, the labels of n samples are y = {y 1 ,y 2 ,y 3 , …, y n },in represents the value of dimension j in the i-th sample, y i Represents the label corresponding to the i-th sample; Represents y i The weight corresponding to the class; I. Use the ADAM optimizer to perform adaptive learning rate adjustment by combining momentum optimization, and then use the first and second moments of the gradient to calculate the discrete universal learning rate of various parameters.

Citation Information

Patent Citations

  • User classification method and device

    CN111639696A

  • Refrigeration equipment fault diagnosis method based on multi-modal feature fusion neural network

    CN114970605A

  • Wheeled robot intelligent fault diagnosis method and system based on graph convolutional network

    CN115099268A

  • Multi-modal data fusion bearing fault diagnosis method, medium and equipment

    CN117370929A

  • State monitoring and fault diagnosis system for motor rolling bearing

    CN117928642A

Cited By

  • Intelligent operation and maintenance management method and system based on cloud platform

    CN120470469A

  • Bearing intelligent fault diagnosis method fusing sound and vibration signals

    CN121453400A