Harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion

Abstract

The invention discloses a harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion, and relates to the field of collaborative robot joint fault diagnosis. In order to solve the problem that an existing harmonic reducer fault diagnosis method is low in diagnosis rate, the invention designs a harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion, and the method comprises the steps: collecting the vibration, torque, temperature and other multi-sensor original data of a harmonic reducer in different fault modes; one-dimensional time-domain signals of a plurality of sensors are converted into two-dimensional time-frequency images through continuous wavelet transform, a fault feature time-frequency graph data set of the harmonic reducer is constructed, feature selection and noise suppression are performed through a residual shrinkage unit in a DRSN-SE algorithm, the feature representation capability is enhanced by using an SE-Net attention mechanism, and the fault feature time-frequency graph data set of the harmonic reducer is constructed. And fusing the features of the multi-source heterogeneous data to obtain final fused features, and outputting a fault diagnosis result through a classification head. According to the invention, the problem of poor reliability of single fault data can be effectively solved, and high-precision fault diagnosis and classification identification of the harmonic reducer are realized.

Description

Technical Field

[0001] The present invention belongs to the field of big data learning models, and in particular relates to a harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion. Background Art

[0002] As the core component of the robot joint, the operating status of the intelligent harmonic reducer has an important impact on the performance and life of the entire system. Due to factors such as complex working environment and variable load, harmonic reducers are prone to faults such as flexible wheel wear and fracture, rigid wheel wear, and bearing pitting, which lead to serious consequences such as personal injury, economic losses, etc. due to robot failure. At present, the fault detection and performance testing of harmonic reducers are mainly carried out by workshop technicians using vibration analyzers to complete comparison tests for troubleshooting, which is inefficient and cannot be integrated into the robot joint system; research on harmonic reducer fault diagnosis is still very scarce, and it is impossible to monitor potential faults in real time and reduce the losses caused by faults. Therefore, using the signals during the operation of the harmonic reducer for intelligent fault diagnosis and analysis has important guiding significance for improving the working performance of the robot.

[0003] During the operation of the harmonic reducer, it is usually affected by the coupling of external systems such as the motor and the load. Factors such as noise signals and harmonic signals make it difficult for the characteristic information of a single signal to reflect the actual fault state of the harmonic reducer, greatly reducing the recognition effect of the fault diagnosis system. Therefore, it is necessary to integrate information from multiple different types of sensors to complement the features of multi-source heterogeneous data and improve the robustness of the fault diagnosis system. Summary of the invention

[0004] In view of the problems that the characteristic information of a single sensor signal of a harmonic reducer is weak, early fault identification is difficult, the multi-source heterogeneous data fusion mechanism is unclear, and different fault mode data sets are missing, the present invention proposes a harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion.

[0005] The specific technical solution adopted by the present invention comprises the following steps:

[0006] A harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion comprises the following steps:

[0007] S1. Through the data sensors arranged on the harmonic reducer, the one-dimensional time domain signals of the vibration, torque and temperature of the harmonic reducer under different fault modes are collected. The three one-dimensional time domain signals collected under each fault mode constitute a data sample in the multi-source heterogeneous data set;

[0008] S2. Preprocess each data sample in the multi-source heterogeneous data, convert the one-dimensional time domain signals of vibration, torque and temperature into two-dimensional time-frequency domain feature maps by continuous wavelet transform, thereby converting each data sample into a training sample, and constructing a training sample set with fault mode labels;

[0009] S3. The harmonic reducer fault diagnosis model is trained using the training sample set; the input of the harmonic reducer fault diagnosis model is the two-dimensional time-frequency domain feature map of the three signals of vibration, torque and temperature. The two-dimensional time-frequency domain feature maps of the three signals are first subjected to noise suppression and feature selection by a deep residual shrinkage network, and then the important features are enhanced by the SE-Net module. Finally, the three enhanced features are fused at the feature level and the fault prediction label of the harmonic reducer is output through the prediction head;

[0010] S4. For harmonic reducers with unknown fault modes, the one-dimensional time domain signals of vibration, torque and temperature are obtained and converted into two-dimensional time-frequency domain feature maps respectively through the continuous wavelet transform method, and then input into the trained harmonic reducer fault diagnosis model to predict the fault mode of the harmonic reducer.

[0011] Preferably, the data sensors arranged on the harmonic reducer include an acceleration sensor for detecting vibration signals, a torque sensor for detecting the output torque of the harmonic reducer, and a temperature sensor for detecting the operating temperature of the harmonic reducer.

[0012] Preferably, the vibration sensor is adsorbed on the outer surface of the rigid wheel by magnetic attraction; the torque sensor is installed at the output end of the harmonic reducer; the temperature sensor uses a temperature probe and is attached to the surface of the harmonic reducer to measure the real-time operating temperature.

[0013] Preferably, the failure modes covered by the data samples in the multi-source heterogeneous data set include six types: no fault, rigid wheel wear fault, flexible wheel wear fault, flexible wheel pitting fault, flexible wheel crack fault, and bearing pitting fault.

[0014] Preferably, for each one-dimensional time domain signal collected by the data sensor, it needs to be resampled and minimum-maximum normalized, then divided into signal segments of fixed length and converted into a two-dimensional time-frequency feature map through continuous wavelet transform. The two-dimensional time-frequency feature map obtained by continuous wavelet transform is resampled by interpolation to meet the input requirements of the harmonic reducer fault diagnosis model.

[0015] Preferably, the harmonic reducer fault diagnosis model includes three branches with the same structure, each sensor signal corresponds to a branch, each branch is composed of an input layer, two convolution layers, a maximum pooling layer, three two-dimensional residual shrinkage unit modules, a global average pooling layer and a SE-Net module cascaded in sequence, the input layer of the first branch is used to input the two-dimensional time-frequency domain feature map of the vibration signal, the input layer of the second branch is used to input the two-dimensional time-frequency domain feature map of the torque signal, and the input layer of the third branch is used to input the two-dimensional time-frequency domain feature map of the harmonic temperature signal. After passing through the SE-Net module, each of the three branches obtains an enhanced feature; the enhanced features output by the three branches are fused through the feature fusion layer, and then output the classification label of the fault mode through a prediction head composed of a fully connected layer and a Softmax layer.

[0016] Preferably, in each of the two-dimensional residual shrinkage unit modules, the input feature map of the module first passes through two two-dimensional convolutional layers to obtain an intermediate feature map, and the intermediate feature map is subjected to absolute value operation and global average pooling to obtain a feature vector Z; the feature vector Z passes through two fully connected layers and is output through a Sigmoid function to obtain a scaling parameter a for each channel, all channels of the feature vector Z are scaled by their respective scaling parameters to obtain a threshold vector, and the intermediate feature map is soft-thresholded using the threshold vector to eliminate noise-related information in the input feature map; finally, the feature map after soft threshold processing is residually connected to the input feature map and passed to the next cascaded network layer through a ReLU function.

[0017] Preferably, in each branch of the harmonic reducer fault diagnosis model, the size of the two-time-frequency feature map input by the input layer is 3×256×256; the filter sizes of the two convolutional layers are 3 and 16 respectively, the convolution kernel size is 3×3, and the stride is 2; the convolution kernel size of the maximum pooling layer is 2×2, and the stride is 2; the size of the three filters in the first residual contraction unit is 32, the convolution kernel size is 3×3, and the stride is 1; the size of the three filters in the second residual contraction unit is 48, the convolution kernel size is 3×3, and the stride is 1; the size of the three filters in the third residual contraction unit is 64, the convolution kernel size is 3×3, and the stride is 1; the feature map size of the input global average pooling layer is 64×4×4, and the output size of the global average pooling layer is a tensor of 64×1×1.

[0018] Preferably, in the feature fusion layer, three enhanced features are fused at feature level through feature splicing operation.

[0019] Preferably, when the harmonic reducer fault diagnosis model is trained using the training sample set, it is necessary to pre-divide it into a training set and a validation set according to a preset ratio.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1) The present invention establishes an intelligent fault diagnosis data set for harmonic reducers by collecting multi-source heterogeneous data such as vibration, torque and temperature of harmonic reducers under different fault modes. This eliminates the need for technicians to manually troubleshoot fault modes, reduces the impact of external systems such as motors and loads on fault diagnosis, and improves the robustness of the harmonic reducer fault diagnosis system.

[0022] 2) The multi-source heterogeneous time domain data sets under different fault modes are converted into time-frequency domain feature maps through continuous wavelet transform. Based on the DRSN-SE model, the noise of the original signal feature maps in multiple branches is suppressed, and feature enhancement at different levels is achieved, thereby improving the classification and recognition performance of the fault system.

[0023] 3) The DRSN-SE model for multi-source heterogeneous data fusion can be integrated into the online monitoring system of robot joints to monitor the potential faults of the harmonic reducer in real time and reduce the losses caused by the faults.

[0024] The above description briefly summarizes the technical solution of the present invention. In order to make the features, solutions and advantages of the present invention easier to understand and use, a detailed description is given below through a specific implementation scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The figure is a flowchart of the steps of a harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion.

[0026] Figure 2 Schematic diagram of a data acquisition system in an embodiment of the present invention.

[0027] Figure 3 This is a DRSN-SE fault diagnosis model based on multi-source heterogeneous data fusion in an embodiment of the present invention.

[0028] Figure 4 2D-RSUB unit structure diagram in an embodiment of the present invention.

[0029] Explanation of symbols in the figure: driving servo motor 1, first encoder 2, input end torque sensor 3, vibration sensor 4, temperature sensor 5, harmonic reducer 6, output end torque sensor 7, second encoder 8, magnetic powder brake 9. DETAILED DESCRIPTION

[0030] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0031] In the description of the present invention, it is to be understood that when an element is considered to be "connected" to another element, it may be directly connected to the other element or indirectly connected, that is, there are intermediate elements. On the contrary, when an element is said to be "directly" connected to another element, there are no intermediate elements.

[0032] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0033] In a preferred embodiment of the present invention, a harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion is provided. Figure 1 As shown, the specific steps include S1 to S4. The specific implementation of each step is described below.

[0034] S1. Through the data sensors arranged on the harmonic reducer, three signals of vibration, torque and temperature of the harmonic reducer under different fault modes are collected, wherein the signals collected by different data sensors are all one-dimensional time domain signals under one-dimensional time series.

[0035] The specific form of the above data sensor can be adjusted according to actual needs. In the present invention, the data sensor arranged on the harmonic reducer may include an acceleration sensor for detecting vibration signals, a torque sensor for detecting the output torque of the harmonic reducer, and a temperature sensor for detecting the operating temperature of the harmonic reducer.

[0036] The specific arrangement positions of the above data sensors can also be adjusted according to actual needs. In order to facilitate accurate data collection, in the embodiment of the present invention, it is necessary to build a harmonic reducer multi-sensor data acquisition test platform to obtain the one-dimensional time domain signals of the three signals of the above harmonic reducer under different fault modes. Figure 2As shown, the basic components of the test platform include a driving servo motor 1, a first encoder 2, an input torque sensor 3, a vibration sensor 4, a temperature sensor 5, a harmonic reducer 6, an output torque sensor 7, a second encoder 8, and a magnetic powder brake 9, which are sequentially as follows from left to right: a driving servo motor 1, a first encoder 2, an input torque sensor 3, a harmonic reducer 6, an output torque sensor 7, a second encoder 8, and a magnetic powder brake 9. In the embodiment of the present invention, the temperature sensor 5 uses an acceleration sensor in the vertical direction of the surface of the harmonic reducer rigid wheel to collect vibration signals during the operation of the harmonic reducer under different fault modes, and the temperature sensor 5 uses a K-type thermocouple in the form of a temperature probe attached to measure the temperature of the surface of the harmonic reducer 6. The input torque sensor 3 and the output torque sensor 7 can respectively capture the real-time torque changes of the harmonic reducer using the torque sensors at the input and output ends. In the embodiment of the present invention, the one-dimensional time domain signal of the torque is recorded by the output torque sensor 7 installed at the output end of the harmonic reducer 6. The speed of the harmonic reducer 6 on the test platform is set to 2400 rpm, and the load is set to 80 Nm.

[0037] Based on this test platform, harmonic reducers with different fault modes can be simulated, and then one-dimensional time domain signals of three different signals can be collected. The three one-dimensional time domain signals collected under each fault mode constitute a data sample in a multi-source heterogeneous data set, thereby forming multi-source heterogeneous data and used to construct model training samples.

[0038] In the embodiment of the present invention, the fault modes covered by the data samples in the multi-source heterogeneous data set include: no fault (i.e. normal operation), rigid wheel wear fault, flexible wheel wear fault, flexible wheel pitting fault, flexible wheel crack fault, and bearing pitting fault. Among them, the bearing pitting is divided into inner and outer ring faults, which are processed by electric spark cutting, with a width of 1mm, a length of 3mm, and a depth of 1mm.

[0039] Under 6 different fault modes, the corresponding vibration signals, torque signals, and temperature signals are collected, and the fault diagrams of the harmonic reducer in the six fault modes are recorded. During the sampling process, the data acquisition card acquisition frequency of the vibration signal is set to 25600Hz, the sampling frequency of the torque sensor is set to 2560Hz, the temperature sensor is 10Hz, the sampling time is 10 minutes, and a sliding window of 12 seconds is selected. The signals collected by the torque sensor and the temperature sensor are resampled, and the sampling points of the vibration signal are unified to 307200 points.

[0040] S2: Preprocess each data sample in the multi-source heterogeneous data, convert the one-dimensional time domain signals of vibration, torque and temperature into two-dimensional time-frequency domain feature maps through the continuous wavelet transform method, so as to convert each data sample into a training sample and construct a training sample set with fault mode labels.

[0041] The one-dimensional time domain signal collected by each data sensor in the above test platform has been resampled in advance to ensure the same number of sampling points. In order to construct a two-dimensional time-frequency feature map that meets the model input, it is also necessary to perform minimum-maximum normalization on the resampled one-dimensional time domain signal, then divide it into fixed-length signal segments and convert it into a two-dimensional time-frequency feature map through continuous wavelet transform, and finally resample it through interpolation to make it meet the input requirements of the harmonic reducer fault diagnosis model.

[0042] In this implementation step, the expression of the signal to be analyzed is assumed to be f(t), and the signal is subjected to minimum-maximum normalization processing to obtain the normalized signal f norm The formula for (t) is as follows:

[0043]

[0044] Among them, f max is the maximum value of the signal, f min is the minimum value of the signal.

[0045] The normalized multi-source heterogeneous data can be converted into a two-dimensional time-frequency domain feature map through continuous wavelet transform. Continuous wavelet transform belongs to the existing technology and is briefly described below.

[0046] Continuous wavelet transform obtains the spectrum information of the signal at different scales and positions by convolving the signal with a set of wavelet basis functions. The calculation formula of wavelet transform is:

[0047]

[0048] The formula for continuous wavelet transform is defined as:

[0049]

[0050] Among them, a, b are the parameters of continuous transformation, a is the scale parameter, b is the translation parameter, f(t) is the original signal, which is a continuous time signal, ψ * is the complex conjugate of the wavelet function.

[0051] In the embodiment of the present invention, Morlet is selected as the wavelet basis function, and the expression of the Morlet wavelet basis function is:

[0052]

[0053] Among them, t is the time variable, ω0 is the frequency parameter that controls the frequency of the wavelet basis function, and j is the imaginary unit. The time variable is set to 256, a and b are set to 2 and 5 respectively, and the frequency parameter is set to 3.

[0054] The two-dimensional time-frequency domain feature map of each data sample in the above-mentioned multi-source heterogeneous data set after continuous wavelet transform can be used to construct training samples. Each training sample needs to contain the two-dimensional time-frequency domain feature map of three signals: vibration, torque and temperature, and also needs to have the fault mode labels when collecting these three signals. Thus, all training samples can constitute a training sample set with fault mode labels. In an embodiment of the present invention, the training set and the validation set can be divided with reference to the conventional model training method to obtain the optimal model. In an embodiment of the present invention, in addition to dividing the training set and the validation set, a part of the training samples are also divided to verify the final performance of the model. Therefore, it can be divided into the training set, validation set and test set of the network model in a ratio of 7:2:1. Of course, the specific division method can be adjusted with reference to the conventional model training method.

[0055] S3. Use the above training sample set to train the harmonic reducer fault diagnosis model; the input of the harmonic reducer fault diagnosis model is the two-dimensional time-frequency domain feature map of three signals: vibration, torque and temperature. The two-dimensional time-frequency domain feature maps of the three signals are first subjected to noise suppression and feature selection through a deep residual shrinkage network, and then the important features are enhanced through the SE-Net module. Finally, the three enhanced features are fused at the feature level and the fault prediction label of the harmonic reducer is output through the prediction head.

[0056] In the harmonic reducer fault diagnosis model constructed by the present invention, it is necessary to input the preprocessed different two-dimensional time-frequency domain feature maps into the DRSN-SE model and extract features through the DRSN and SE-Net modules respectively. Finally, the features extracted by the three channels are fused at the feature level to finally realize multi-source heterogeneous data fusion under different fault modes.

[0057] like Figure 3As shown in the figure, the above harmonic reducer fault diagnosis model includes three branches with the same structure, each sensor signal corresponds to a branch, and each branch is composed of an input layer (Signal), two convolutional layers (Conv-2D), a maximum pooling layer (Maxpooling), three two-dimensional residual shrinkage units (2D-RSUB) modules, a global average pooling layer (Global Average Pooling) and a SE-Net module cascaded in sequence. The input layer (Signal 1) of the first branch is used to input the two-dimensional time-frequency domain feature map of the vibration signal, the input layer (Signal 2) of the second branch is used to input the two-dimensional time-frequency domain feature map of the torque signal, and the input layer (Signal 3) of the third branch is used to input the two-dimensional time-frequency domain feature map of the harmonic temperature signal. After passing through the SE-Net module, each of the three branches obtains an enhanced feature; the enhanced features output by the three branches are fused through the feature fusion layer (Fusion), and then output the classification label of the fault mode through the prediction head composed of a fully connected layer (Fully Connected) and a Softmax layer.

[0058] The specific parameters of each network layer in the above network can be adjusted as needed. In an embodiment of the present invention, in each branch, the size of the two-time-frequency feature map input by the input layer is 3×256×256; the filter sizes of the two convolutional layers are 3 and 16 respectively, the convolution kernel size is 3×3, and the stride is 2; the convolution kernel size of the maximum pooling layer is 2×2, and the stride is 2; the size of the three filters in the first residual shrinkage unit (RSBU) is 32, the convolution kernel size is 3×3, and the stride is 1; the size of the three filters in the second RSBU is 48, the convolution kernel size is 3×3, and the stride is 1; the size of the three filters in the third RSBU is 64, the convolution kernel size is 3×3, and the stride is 1; the feature map size of the input global average pooling layer is 64×4×4, and the output size of the global average pooling layer is a tensor of 64×1×1.

[0059] The above DRSN contains multiple RSBUs, which integrate the soft threshold function and attention mechanism, and can be used for feature learning methods of strong noise or high redundant data. The soft threshold function can adaptively remove noise and redundant information in the signal and enhance the learning ability of features. The expression of the soft threshold function is:

[0060]

[0061] Among them, x represents the input feature, y represents the output feature, and τ is the threshold.

[0062] Traditional RSBU is usually designed for one-dimensional time series signals. To adapt to the input samples, the one-dimensional convolution in RSBU is changed to a two-dimensional convolution. Figure 4As shown in the figure, in the two-dimensional residual shrinkage unit 2D-RSUB module, the input feature map of the module first passes through two two-dimensional convolutional layers (with BN operation and ReLu activation function) to obtain an intermediate feature map, and the intermediate feature map passes through an absolute value operation (Absloute) and a global average pooling (GAP) to obtain a C×1×1 dimensional feature vector Z; the feature vector Z passes through two fully connected layers (the second fully connected layer has a BN operation and a ReLu activation function) to obtain a C×1×1 dimensional feature vector k, and the feature vector k is output through the Sigmoid function to obtain the scaling parameters of each channel (a total of C channels, each channel has a scaling parameter a c , whereby the Sigmoid function outputs a C×1×1 dimensional scaling parameter vector), all channels of the feature vector Z are scaled by their respective scaling parameters to obtain a threshold vector τ, and the threshold vector τ is used to perform soft threshold processing on the intermediate feature map to eliminate noise-related information in the input feature map; finally, the feature map after soft threshold processing is residually connected to the input feature map (Identity shortcut) and then passed to the next cascaded network layer through the ReLU function.

[0063] The one-dimensional convolution in the above 2D-RSUB module is changed to a two-dimensional convolution, but the main network layers are similar to the traditional RSBU, such as Absloute, scaling parameter calculation, and soft threshold processing. The specific calculation principle can be found in the traditional RSBU.

[0064] In the branch of the above 2D-RSUB module, the core of noise reduction lies in the soft threshold processing based on the scaling parameter. After the input features pass through two fully connected layers, the scaling parameter a is obtained through the output of the Sigmoid function, and the expression is:

[0065]

[0066] Among them, k c represents the cth channel in the feature vector k output by the second FC layer, a c Represents the c-th channel of the scaling parameter a.

[0067] The calculation method of the threshold is expressed as:

[0068] τ c =a c ·Z

[0069] Among them, τ c is the threshold of the cth channel of the feature map, and τ for all channels c Constitute the threshold vector τ; Z is the output feature of the convolutional layer.

[0070] In the sample set input to the network model in the present invention, the noise content of each sample is different, and the residual shrinkage unit can automatically learn a set of thresholds according to the input samples, and each sample generates its own independent threshold according to its own noise content to eliminate noise-related information. The final output is passed to the next layer of the network through the ReLU function, which can accelerate the training process and improve computational efficiency.

[0071] The above-mentioned SE-Net (Squeeze Excitation Network) module belongs to the existing technology and includes a convolution layer, a global average pooling layer, two fully connected layers, a Sigmoid activation function and a Scale unit. In the first convolution layer, Transformation is performed. tr Convolution operation, construct feature map U, F tr The expression is

[0072]

[0073] X=[x 1 ,x 2 ,…,x C′ ]

[0074] in, * indicates convolution, is a 2D convolution kernel, V c The single channel that is applied will be applied to X.

[0075] In order to utilize channel dependencies, channel statistics are generated through global average pooling, and the W×H×C feature map containing global information is compressed into a 1×1×C feature vector Z.

[0076]

[0077] Among them, z c is the Cth element of Z.

[0078] The present invention uses aggregated information to fully capture channel dependencies, adopts a gating mechanism consisting of two fully connected layers, and uses channel compression, nonlinear activation function, and channel number recovery. Then, a Sigmoid activation function is used to obtain a weight s with a dimension of 1×1×C to characterize the weight of the channel in the feature map U. The expression of s is:

[0079] s=F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z))

[0080] Where δ refers to the ReLU function, r represents the scaling factor.

[0081] Finally, the attention weights obtained above are weighted to the features of each channel through the Scale operation to obtain the final output of the SE-Net module.

[0082]

[0083] in, F scale (u c ,s c ) refers to the feature map With scalar s c Channel multiplication.

[0084] In the global average pooling layer, the size of the input feature map is 64×1×1. The features of each channel are averaged to generate a 1×1×64 feature map. The number of channels C input to the first fully connected layer is 64. Assuming the scaling factor is r=16, the output channels after the first fully connected layer are 4. After the ReLU function and the second fully connected layer, the number of channels output at the Scale layer is 64.

[0085] After the data of the three modalities pass through the same network structure, feature splicing operation is performed in the feature fusion layer to splice and integrate the features of the three branches, and feature classification and recognition are realized through the Softmax layer.

[0086] The above DRSN-SE model is completed in Python programming language under the Pytorch deep learning model framework. In the training stage, the learning rate of the model is set to 0.001, and the optimization algorithm used is the Adam algorithm. During training, the batch size is set to 90 and the number of epochs is set to 40. In the model, the parameters of each neural network layer are shown in Table 1, where the first number in the brackets in the first column represents the number of channels, the middle number represents the size of the convolution kernel, and the last number represents the step size of the operation step; the second column represents the number of each unit, and the third column represents the output size of different layers, in the format of number of channels × width × height.

[0087] Table 1 DRSN-SE model parameter settings

[0088]

[0089]

[0090] The constructed dataset is used to train and verify the DRSN-SE model, the trained model is tested and the optimal model parameters are saved.

[0091] S4. For harmonic reducers with unknown fault modes, the one-dimensional time domain signals of vibration, torque and temperature can be obtained and converted into two-dimensional time-frequency domain feature maps respectively through the continuous wavelet transform method, and then input into the trained harmonic reducer fault diagnosis model to predict the fault mode of the harmonic reducer.

[0092] In the embodiment of the present invention, the data of the test set is used for the performance verification of the DRSN-SE model, which shows that the model can better realize the fault diagnosis of the harmonic reducer, and finally the parameters of the trained model are saved. The model can be integrated into the data processing equipment of the robot joint integration module, so that the model can integrate the fault warning function on other devices.

[0093] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion, characterized in that: The steps include: S1. Through the data sensors arranged on the harmonic reducer, the one-dimensional time domain signals of the vibration, torque and temperature of the harmonic reducer under different fault modes are collected. The three one-dimensional time domain signals collected under each fault mode constitute a data sample in the multi-source heterogeneous data set; S2. Preprocess each data sample in the multi-source heterogeneous data, convert the one-dimensional time domain signals of vibration, torque and temperature into two-dimensional time-frequency domain feature maps by continuous wavelet transform, thereby converting each data sample into a training sample, and constructing a training sample set with fault mode labels; S3. The harmonic reducer fault diagnosis model is trained using the training sample set; the input of the harmonic reducer fault diagnosis model is the two-dimensional time-frequency domain feature map of the three signals of vibration, torque and temperature. The two-dimensional time-frequency domain feature maps of the three signals are first subjected to noise suppression and feature selection by a deep residual shrinkage network, and then the important features are enhanced by the SE-Net module. Finally, the three enhanced features are fused at the feature level and the fault prediction label of the harmonic reducer is output through the prediction head; S4. For harmonic reducers with unknown fault modes, the one-dimensional time domain signals of vibration, torque and temperature are obtained and converted into two-dimensional time-frequency domain feature maps respectively through the continuous wavelet transform method, and then input into the trained harmonic reducer fault diagnosis model to predict the fault mode of the harmonic reducer.

2. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The data sensors arranged on the harmonic reducer include an acceleration sensor for detecting vibration signals, a torque sensor for detecting the output torque of the harmonic reducer, and a temperature sensor for detecting the operating temperature of the harmonic reducer.

3. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 2 is characterized in that: The vibration sensor is adsorbed on the outer surface of the rigid wheel by magnetic attraction; the torque sensor is installed at the output end of the harmonic reducer; the temperature sensor adopts a temperature probe and is attached to the surface of the harmonic reducer to measure the real-time operating temperature.

4. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The failure modes covered by the data samples in the multi-source heterogeneous data set include six types: no fault, rigid wheel wear fault, flexible wheel wear fault, flexible wheel pitting fault, flexible wheel crack fault, and bearing pitting fault.

5. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: For each one-dimensional time domain signal collected by the data sensor, it needs to be resampled and minimum-maximum normalized, then divided into signal segments of fixed length and converted into a two-dimensional time-frequency feature map through continuous wavelet transform. The two-dimensional time-frequency feature map obtained by continuous wavelet transform is resampled through interpolation to make it meet the input requirements of the harmonic reducer fault diagnosis model.

6. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The harmonic reducer fault diagnosis model comprises three branches with the same structure, each sensor signal corresponds to one branch, each branch is formed by sequentially cascading an input layer, two convolution layers, a maximum pooling layer, three two-dimensional residual shrinkage unit modules, a global average pooling layer and an SE-Net module, the input layer of the first branch is used to input a two-dimensional time-frequency domain feature map of a vibration signal, the input layer of the second branch is used to input a two-dimensional time-frequency domain feature map of a torque signal, and the input layer of the third branch is used to input a two-dimensional time-frequency domain feature map of a harmonic temperature signal, and each of the three branches obtains an enhanced feature after passing through the SE-Net module; The enhanced features output by the three branches are fused through the feature fusion layer and then pass through the prediction head composed of a fully connected layer and a Softmax layer to output the classification label of the fault mode.

7. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 6 is characterized in that: In each of the two-dimensional residual shrinkage unit modules, the input feature map of the module first passes through two two-dimensional convolutional layers to obtain an intermediate feature map, and the intermediate feature map is subjected to absolute value operation and global average pooling to obtain a feature vector Z; After the feature vector Z passes through two fully connected layers, the scaling parameter a of each channel is obtained by the Sigmoid function output. All channels of the feature vector Z are scaled by their respective scaling parameters to obtain a threshold vector. The threshold vector is used to perform soft threshold processing on the intermediate feature map to eliminate noise-related information in the input feature map. Finally, the feature map after soft threshold processing is connected to the residual of the input feature map and passed to the next cascade network layer through the ReLU function.

8. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 6 is characterized in that: In each branch of the harmonic reducer fault diagnosis model, the size of the two-time-frequency feature map input by the input layer is 3×256×256; the filter sizes of the two convolutional layers are 3 and 16 respectively, the convolution kernel size is 3×3, and the stride is 2; the convolution kernel size of the maximum pooling layer is 2×2, and the stride is 2; the size of the three filters in the first residual contraction unit is 32, the convolution kernel size is 3×3, and the stride is 1; the size of the three filters in the second residual contraction unit is 48, the convolution kernel size is 3×3, and the stride is 1; the size of the three filters in the third residual contraction unit is 64, the convolution kernel size is 3×3, and the stride is 1; the feature map size of the input global average pooling layer is 64×4×4, and the output size of the global average pooling layer is a tensor of 64×1×1.

9. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 1, characterized in that: In the feature fusion layer, the three enhanced features are fused at feature level through feature concatenation operation.

10. The harmonic reducer fault diagnosis method based on multi-source heterogeneous data fusion according to claim 1, characterized in that: When the harmonic reducer fault diagnosis model is trained using the training sample set, it is necessary to divide it into a training set and a validation set in advance according to a preset ratio.

Images