Industrial robot fault diagnosis method, system, device and storage medium

Abstract

The present invention discloses an industrial robot fault diagnosis method, system, device and storage medium, including: when a fault diagnosis request is detected, determining the industrial robot corresponding to the fault diagnosis request; extracting operation features from the periodic operation data corresponding to the industrial robot; forming a feature sample pair with the operation features and the fault feature samples; inputting the feature sample pairs into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pairs; comparing the feature similarity with the preset similarity threshold, and determining the fault category corresponding to the industrial robot according to the comparison result, solving the problem that the existing deep learning model fault diagnosis technology uses a large amount of data to train the model, and diagnoses the industrial robot fault signal after the training is completed, but the industrial robot fault data is difficult to obtain, resulting in low generalization ability of the trained model and difficulty in adapting to different usage scenarios. The present invention uses a twin neural network structure to realize automatic feature extraction and fault identification.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial robots, and in particular to an industrial robot fault diagnosis method, system, equipment and storage medium. Background Art

[0002] Industrial robots are highly automated modern manufacturing machinery and equipment, consisting of multiple technologies such as machinery and artificial intelligence. Industrial robots are widely used in the field of production and manufacturing, gradually replacing manual labor to complete assembly line work, improving productivity, product quality and production safety, and have become an indispensable automation equipment in the 21st century.

[0003] However, once an industrial robot fails, the entire production cycle may be delayed at best, or even cause casualties. In the existing technology, when faced with industrial robot failures, experienced and skilled workers are usually relied on to perform fault diagnosis on site. However, fault diagnosis that relies solely on experience can no longer keep up with the increasing complexity of the system. Blind inspection and repair lead to increased maintenance costs and low efficiency.

[0004] To this end, the existing deep learning model fault diagnosis technology uses a large amount of data to train the model. After the training is completed, the industrial robot fault signals can be diagnosed. However, industrial robot fault data is difficult to obtain under natural conditions, resulting in low generalization ability of the trained model and difficulty in quickly adapting to different usage scenarios. Summary of the invention

[0005] The present invention provides an industrial robot fault diagnosis method, system, device and storage medium, which solves the technical problem that the existing deep learning model fault diagnosis technology uses a large amount of data to train the model. After the training is completed, the industrial robot fault signal can be diagnosed, but the industrial robot fault data is difficult to obtain under natural conditions, resulting in the low generalization ability of the trained model and difficulty in quickly adapting to different usage scenarios.

[0006] A first aspect of the present invention provides an industrial robot fault diagnosis method, the method comprising:

[0007] When a fault diagnosis request is detected, determining the industrial robot corresponding to the fault diagnosis request;

[0008] Extracting operation features from periodic operation data corresponding to the industrial robot;

[0009] The operation feature and the fault feature sample form a feature sample pair;

[0010] Inputting the feature sample pair into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair;

[0011] The feature similarity is compared with a preset similarity threshold, and the fault category corresponding to the industrial robot is determined according to the comparison result.

[0012] Optionally, the step of extracting operation features from the periodic operation data corresponding to the industrial robot includes:

[0013] Obtaining feedback speed from periodic operation data corresponding to the industrial robot;

[0014] Selecting an acceleration operation stage; wherein the feedback speed in the acceleration operation stage is greater than a predetermined threshold;

[0015] Obtaining feedback torque or feedback current corresponding to the acceleration operation stage;

[0016] The time-frequency domain features of the feedback torque or the feedback current in the time-frequency domain are extracted as operation features.

[0017] Optionally, the target twin neural network includes a convolution layer, a pooling layer, a multi-scale convolution layer, a global maximum pooling layer and a fully connected layer, and the step of inputting the feature sample pair into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair includes:

[0018] Inputting the operation feature and the fault feature sample in the feature sample pair into the convolution layer respectively, performing convolution on the operation feature and the fault feature sample respectively through the convolution layer, generating first convolution features corresponding to the operation feature and the fault feature sample respectively, and transmitting the first convolution features to the pooling layer;

[0019] Performing dimension reduction on the first convolutional feature through the pooling layer to generate a first pooling feature and transmitting it to the multi-scale convolutional layer;

[0020] Using the multi-scale convolution layer to convolve the first pooling features respectively using multiple convolution kernels, generating multiple second convolution features and transmitting them to the global maximum pooling layer;

[0021] Performing a global maximum pooling operation on all the second convolutional features through the global maximum pooling layer to generate multiple second pooling features and transmitting them to the fully connected layer;

[0022] Classifying all the second pooling features through the fully connected layer to obtain target features;

[0023] The Euclidean distance between the target features is calculated to obtain the feature similarity corresponding to the feature sample pair.

[0024] Optionally, the target twin neural network further includes a multi-head attention mechanism layer, and before the step of performing a global maximum pooling operation on all the second convolutional features through the global maximum pooling layer to generate multiple second pooling features and transmitting them to the fully connected layer, it also includes:

[0025] Inputting the second convolutional feature into the multi-head attention mechanism layer;

[0026] Converting the second convolutional features into multiple input vectors through the multi-head attention mechanism layer;

[0027] Mapping the plurality of input vectors to a plurality of subspaces to perform scaled dot product attention calculations, and generating a plurality of scaled dot product attention results;

[0028] All the scaled dot product attention results are concatenated to generate a new second convolutional feature.

[0029] Optionally, the step of comparing the feature similarity with a preset similarity threshold and determining the fault category corresponding to the industrial robot according to the comparison result includes:

[0030] Comparing the feature similarity with a preset similarity threshold;

[0031] If the feature similarity is greater than the similarity threshold, determining that the fault category corresponding to the industrial robot is the fault category associated with the fault feature sample;

[0032] If the feature similarity is less than or equal to the similarity threshold, then in response to the inputted new fault category, the fault category corresponding to the industrial robot is determined as the new fault category.

[0033] Optionally, before the step of responding to the input of the newly added fault category and determining the fault category corresponding to the industrial robot as the newly added fault category, the method further includes:

[0034] Acquire remaining samples except the fault feature samples from a preset sample set as new fault feature samples;

[0035] Jump to the step of combining the operation feature and the fault feature sample into a feature sample pair until the remaining samples do not exist in the sample set.

[0036] Optionally, it also includes:

[0037] Construct an initial twin neural network using preset model parameters;

[0038] Inputting the preset feature training set into the initial twin neural network in sequence, and outputting the training results respectively;

[0039] Using a preset loss function combined with all the training results, the cross entropy corresponding to the initial twin neural network is calculated;

[0040] When the cross entropy is greater than a preset training threshold, the model parameters are adjusted according to a preset gradient, and the step of sequentially inputting the preset feature training set into the initial twin neural network and outputting the training results is skipped;

[0041] When the cross entropy is less than or equal to the training threshold, the initial twin neural network at the current moment is determined as the target twin neural network.

[0042] A second aspect of the present invention provides an industrial robot fault diagnosis system, comprising:

[0043] A fault diagnosis request module, configured to, when a fault diagnosis request is detected, determine the industrial robot corresponding to the fault diagnosis request;

[0044] An operation feature extraction module, used to extract operation features from periodic operation data corresponding to the industrial robot;

[0045] A feature sample pair module, used for combining the operation feature and the fault feature sample into a feature sample pair;

[0046] A feature similarity module, used to input the feature sample pair into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair;

[0047] The feature similarity comparison module is used to compare the feature similarity with a preset similarity threshold, and determine the fault category corresponding to the industrial robot according to the comparison result.

[0048] A third aspect of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the industrial robot fault diagnosis method as described in any one of the above items.

[0049] A fourth aspect of the present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed, the industrial robot fault diagnosis method as described in any one of the above items is implemented.

[0050] It can be seen from the above technical solutions that the present invention has the following advantages:

[0051] The present invention detects fault diagnosis requests in real time to obtain fault diagnosis request information in the first time. When a fault diagnosis request is detected, the fault diagnosis request is parsed to determine the industrial robot corresponding to the fault diagnosis request. At this time, the operation features are extracted from the periodic operation data corresponding to the industrial robot, and the extracted operation features and fault feature samples are combined into feature sample pairs, and the feature sample pairs are input into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pairs, and the feature similarity is compared with the preset similarity threshold. According to the comparison result, the fault category corresponding to the industrial robot is determined, thereby solving the technical problem that the existing deep learning model fault diagnosis technology uses a large amount of data to train the model, and the industrial robot fault signal can be diagnosed after the training is completed, but the industrial robot fault data is difficult to obtain under natural conditions, resulting in low generalization ability of the trained model and difficulty in quickly adapting to different usage scenarios. The present invention uses a twin neural network structure combined with a new intelligent fault diagnosis model of a subnetwork to realize automatic feature extraction and fault identification. The twin neural network structure effectively expands the number of model training times, deeply mines the relationship between different samples, avoids problems such as network underfitting caused by insufficient sample number, and achieves better fault diagnosis effect in small sample conditions, and is better applied to actual production and manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0053] Figure 1 A flowchart of a method for diagnosing a fault in an industrial robot provided in Embodiment 1 of the present invention;

[0054] Figure 2 A flowchart of a method for diagnosing an industrial robot fault provided in Embodiment 2 of the present invention;

[0055] Figure 3 A schematic diagram of the root mean square of the feedback torque of a complete cycle provided by an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of the root mean square of the feedback torque during the acceleration operation phase provided by an embodiment of the present invention;

[0057] Figure 5 Schematic diagram of the structure of the scaled dot product attention mechanism (left) and the multi-head attention mechanism (right) provided in the embodiments of the present invention;

[0058] Figure 6 A schematic diagram of the structure of a twin neural network model provided in an embodiment of the present invention;

[0059] Figure 7 This is a structural block diagram of an industrial robot fault diagnosis system provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0060] The embodiments of the present invention provide an industrial robot fault diagnosis method, system, device and storage medium, which are used to solve the technical problem that the existing deep learning model fault diagnosis technology uses a large amount of data to train the model. After the training is completed, the industrial robot fault signal can be diagnosed, but the industrial robot fault data is difficult to obtain under natural conditions, resulting in the low generalization ability of the trained model and difficulty in quickly adapting to different usage scenarios.

[0061] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1 , Figure 1 A flowchart of the steps of an industrial robot fault diagnosis method provided in Embodiment 1 of the present invention.

[0063] The present invention provides an industrial robot fault diagnosis method, comprising the following steps:

[0064] Step 101: When a fault diagnosis request is detected, determine the industrial robot corresponding to the fault diagnosis request.

[0065] It should be noted that fault diagnosis request refers to a technical request to "see a doctor" for an industrial robot. Fault diagnosis is a technology that understands and grasps the status of an industrial robot during use, determines whether it is normal or abnormal as a whole or in part, detects faults and their causes at an early stage, and can predict fault development trends.

[0066] Industrial robots refer to highly automated modern manufacturing machinery and equipment. The most common industrial robots are industrial robot arms. In the actual production process, industrial robot arms often periodically repeat a set of actions, such as grinding, welding, spraying, etc. Therefore, the corresponding operating signals of industrial robots will also show periodic changes.

[0067] In a specific implementation, the fault diagnosis request is detected in real time to obtain the fault diagnosis request information in the first place. When the fault diagnosis request is detected, the information carried by the fault diagnosis request is parsed to determine the industrial robot corresponding to the fault diagnosis request from multiple industrial robots.

[0068] Step 102: extracting operation features from periodic operation data corresponding to the industrial robot.

[0069] It should be noted that periodic operation data refers to the data generated by an industrial robot periodically repeating a set of actions in the actual production process.

[0070] Motion characteristics refer to the characteristics of the actions performed by industrial robots during corresponding production operations.

[0071] In this embodiment, the operation characteristics of the industrial robot when performing an action are obtained from a set of periodic operation data corresponding to the industrial robot according to the law of the industrial robot operation data. Since the operation characteristics are obvious when the industrial robot fails, the accuracy of fault diagnosis can be effectively improved.

[0072] Step 103: Combine the operation feature and the fault feature samples into a feature sample pair.

[0073] It should be noted that fault feature samples refer to individual fault features stored in the fault database. Specifically, the fault feature samples may include fault information such as the industrial robot switch tripping or being unable to close, the industrial robot inverter frequently reporting faults, the industrial robot maintenance controller output and input points being abnormal or unable to operate normally, and the industrial robot making abnormal noises. The faults are classified according to the fault conditions of each fault feature sample.

[0074] In this embodiment, the acquired operation characteristics and the fault characteristic samples stored in the fault database are combined into characteristic sample pairs.

[0075] Step 104: input the feature sample pairs into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pairs.

[0076] It should be noted that the twin neural network refers to a pattern recognition method based on sample similarity, which uses two weight-sharing sub-networks to simultaneously receive two input samples, and the output result is the similarity of the two input samples. The target twin neural network uses a one-dimensional convolutional neural network (CNN) plus a multi-scale convolutional layer and a multi-head attention mechanism layer to form a twin sub-network (1D-MSCNN) to construct a twin neural network.

[0077] In the specific implementation, the operating features and fault feature samples in the feature sample pair are respectively input into the preset target twin neural network, and the operating features and fault feature samples are subjected to dimensionality reduction processing through the target twin neural network. The Euclidean distance between the operating features and the fault feature samples is calculated in the low-dimensional space to provide a data basis for similarity comparison, thereby obtaining the similarity between the operating features and the fault feature samples.

[0078] Step 105 : compare the feature similarity with a preset similarity threshold, and determine the fault category corresponding to the industrial robot according to the comparison result.

[0079] In this embodiment, the Euclidean distance is used as a metric to calculate the similarity between the operation feature and the fault feature sample to obtain the feature similarity. The preset similarity threshold is set by the technician according to actual needs, and the embodiment of the present invention does not limit this.

[0080] In a specific implementation, the feature similarity is compared with a preset similarity threshold, and the fault category corresponding to the industrial robot is determined according to the comparison result.

[0081] The present invention detects fault diagnosis requests in real time to obtain fault diagnosis request information in the first time. When a fault diagnosis request is detected, the fault diagnosis request is parsed to determine the industrial robot corresponding to the fault diagnosis request. At this time, the operation features are extracted from the periodic operation data corresponding to the industrial robot, and the extracted operation features and fault feature samples are combined into feature sample pairs, and the feature sample pairs are input into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pairs, and the feature similarity is compared with the preset similarity threshold. According to the comparison result, the fault category corresponding to the industrial robot is determined, thereby solving the technical problem that the existing deep learning model fault diagnosis technology uses a large amount of data to train the model, and the industrial robot fault signal can be diagnosed after the training is completed, but the industrial robot fault data is difficult to obtain under natural conditions, resulting in low generalization ability of the trained model and difficulty in quickly adapting to different usage scenarios. The present invention uses a twin neural network structure combined with a new intelligent fault diagnosis model of a subnetwork to realize automatic feature extraction and fault identification. The twin neural network structure effectively expands the number of model training times, deeply mines the relationship between different samples, avoids problems such as network underfitting caused by insufficient sample number, and achieves better fault diagnosis effect in small sample conditions, and is better applied to actual production and manufacturing.

[0082] See also Figure 2 , Figure 2 A flowchart of a method for diagnosing an industrial robot fault provided in Embodiment 2 of the present invention;

[0083] Step 201: When a fault diagnosis request is detected, determine the industrial robot corresponding to the fault diagnosis request.

[0084] It should be noted that the fault diagnosis request information can be initiated by the user, or the system can automatically detect the fault information of the industrial robot. Specifically, when the user wants to know the operation status of a specific industrial robot and whether there is a fault, he can directly send a fault diagnosis request to the industrial robot. Or when the system detects fault data, it can directly initiate a fault diagnosis request to automatically detect the industrial robot corresponding to the fault data, so as to perform fault diagnosis on the industrial robot.

[0085] Step 202: Obtain feedback speed from the periodic operation data corresponding to the industrial robot.

[0086] It should be noted that the periodic operation data of the industrial robot includes feedback speed, feedback torque and feedback current. The feedback speed is the speed change when the industrial robot performs an action or stops performing an action.

[0087] Step 203: Select an accelerated operation phase; wherein the feedback speed in the accelerated operation phase is greater than a predetermined threshold.

[0088] It should be noted that the acceleration operation phase refers to a phase in which the industrial robot accelerates its operation speed when performing an operation action. The acceleration operation phase is selected from the periodic operation data, and the feedback speed in the acceleration operation phase is greater than a predetermined threshold.

[0089] In a specific embodiment, when the feedback speed is greater than a predetermined threshold, it indicates that the industrial robot is performing an operation; when the feedback speed is less than the predetermined threshold, it indicates that the industrial robot is not performing an operation. Specifically, the predetermined threshold is set by technicians according to actual needs, and the embodiment of the present invention does not limit this.

[0090] Step 204: Obtain feedback torque or feedback current corresponding to the acceleration operation stage.

[0091] It should be noted that the feedback torque or feedback current corresponding to the acceleration operation stage is obtained from the periodic operation data.

[0092] Step 205: extracting the time-frequency domain features of the feedback torque or feedback current in the time-frequency domain as the operation features.

[0093] It should be noted that the EMD feature extraction method is used to extract the time-frequency domain features of the feedback torque or feedback current in the time-frequency domain as the operation features.

[0094] In a specific embodiment, the feedback torque in the periodic operation data changes periodically with the operation time of the industrial robot, and the root mean square value of the feedback torque of a complete cycle is calculated, such as Figure 3As shown in the figure, it is difficult to intuitively see whether there is a fault in the operating characteristics by using a visual chart to represent the RMS value of the feedback torque of the complete cycle. Figure 4 As shown, a visual chart is used to represent the root mean square value of the feedback torque in the acceleration operation stage. It can be intuitively seen that there is a fault in the operation characteristics of the acceleration operation stage. It can be seen that the operation characteristics extracted in the acceleration operation stage are more distinguishable between normal operation data and fault data. Therefore, the feedback torque or feedback current corresponding to the acceleration operation stage is obtained from the periodic operation data, and the time-frequency domain characteristics of the feedback torque or feedback current in the time-frequency domain are extracted as the operation characteristics.

[0095] Step 206: Combine the operation feature and the fault feature samples into a feature sample pair.

[0096] In the embodiment of the present invention, the specific implementation process of step 206 is similar to step 103 and will not be repeated here.

[0097] Step 207: input the feature sample pair into the preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair.

[0098] Optionally, the target twin neural network includes a convolution layer, a pooling layer, a multi-scale convolution layer, a global maximum pooling layer and a fully connected layer, and step 207 may include the following steps S11-S16:

[0099] S11, inputting the operation feature and the fault feature samples in the feature sample pair into the convolution layer respectively, convolving the operation feature and the fault feature samples respectively through the convolution layer, generating first convolution features corresponding to the operation feature and the fault feature samples respectively and transmitting them to the pooling layer;

[0100] S12, reducing the dimension of the first convolution feature through the pooling layer, generating the first pooling feature and transmitting it to the multi-scale convolution layer;

[0101] S13, using a multi-scale convolution layer to convolve the first pooling features with multiple convolution kernels, generating multiple second convolution features and transmitting them to the global maximum pooling layer;

[0102] S14, performing a global maximum pooling operation on all second convolutional features through a global maximum pooling layer, generating multiple second pooling features and transmitting them to a fully connected layer;

[0103] S15, classify all the second pooling features through the fully connected layer to obtain the target features;

[0104] S16. Calculate the Euclidean distance between target features to obtain the feature similarity corresponding to the feature sample pair.

[0105] It should be noted that the convolution kernel of the convolution layer is used to perform convolution operations on the operating features and fault feature samples in the feature sample pair, so as to extract the first convolution features corresponding to the operating features and fault feature samples, respectively, and transmit the first convolution features corresponding to the operating features and fault feature samples to the pooling layer to generate the first pooling features corresponding to the operating features and fault feature samples. The first convolution features corresponding to the operating features and fault feature samples are reduced in dimension through the pooling layer to improve the receptive field of the first pooling features, and the first pooling features corresponding to the operating features and fault feature samples are transmitted to the multi-scale convolution layer.

[0106] In a specific embodiment, the multi-scale convolution layer is composed of three parts, and the convolution kernel scale used in each part is different from the other convolution kernel scales. In this way, multi-scale features can be extracted from the output of the previous layer, and then the feature vectors obtained by convolution of each branch are concatenated as the input of the next layer. The multi-scale convolution layer uses multiple convolution kernels to convolve the first pooling features respectively, which can further fully extract the first pooling features, generate multiple second convolution features and transmit them to the global maximum pooling layer.

[0107] In this embodiment, a global maximum pooling operation is performed on all second convolutional features through a maximum pooling layer, and the dimension of all second convolutional features is further reduced, the amount of calculation is reduced, and the invariance of the second convolutional features is strengthened. The generated multiple second pooling features are transmitted to the fully connected layer, and the target features corresponding to the operation features and the fault feature samples are obtained through classification by the fully connected layer, and the Euclidean distance between the target features is calculated in a low-dimensional space to determine the similarity corresponding to the operation features and the fault feature samples.

[0108] Optionally, the target twin neural network further includes a multi-head attention mechanism layer, and before executing step S14, the following steps S131-S134 are also included:

[0109] S131, input the second convolutional feature into the multi-head attention mechanism layer;

[0110] S132, converting the second convolutional feature into multiple input vectors through a multi-head attention mechanism layer;

[0111] S133, mapping multiple input vectors to multiple subspaces to perform scaled dot product attention calculations, and generating multiple scaled dot product attention results;

[0112] S134. Concatenate all scaled dot product attention results to generate a new second convolutional feature.

[0113] It should be noted that in order to improve the accuracy and efficiency of industrial robot fault diagnosis, this embodiment uses Figure 5The multi-head attention mechanism shown optimizes the 1D-MSCNN model. The multi-head attention mechanism layer refers to a mechanism that links different positions of the input sequence with the computational representation. The second convolutional feature is input into the multi-head attention mechanism layer, and the second convolutional feature is converted into multiple input vectors through the multi-head attention mechanism layer, which are the query vector Q, the key vector K and the value vector V, respectively. The query vector Q and the key vector K have the same numerical value, and the dependency relationship between the data can be obtained. Specifically, the embodiment of the present invention uses a scaled dot-product attention mechanism (Scaled Dot-Product attention), and the specific calculation formula is as follows:

[0114] (1)

[0115] In the formula , , , Indicates length, Indicates the dimension. The dot product result is divided by the scaling factor Adjustment is performed to avoid the gradient vanishing problem caused by very small gradient values ​​during gradient calculation, which will affect the training of the model. In this way, the attention score of Q on K is obtained after the softmax function, and finally the dot product calculation with V is performed to capture the content that is being paid attention to in K.

[0116] In a specific embodiment, the query vector Q, the key vector K and the value vector V are linearly mapped to different subspaces h times, and the scaled dot product attention calculation is performed in parallel on the three vectors in each subspace. Each subspace is called a "head". For example, the calculation process of the i-th head is as follows:

[0117] (2)

[0118] (3)

[0119] (4)

[0120] (5)

[0121] in, , , is the mapping matrix weight, is the mapping dimension.

[0122] After computing each scaled dot product attention independently, their outputs are concatenated to capture feature information in all subspaces and fed into another linear projection to obtain the final The output of the dimension generates a new second convolution feature. The specific calculation formula is as follows:

[0123] (6)

[0124] in, is the output weight matrix, which can restore the output dimension size to the dimension of the initial input. The input vector dimensions in each head are evenly distributed from the original input, with the goal of making the computation of multi-head attention close to that of a single scaled dot-product attention.

[0125] Step 208: compare the feature similarity with a preset similarity threshold, and determine the fault category corresponding to the industrial robot according to the comparison result.

[0126] Optionally, step 208 may include the following steps S21-S23:

[0127] S21. Compare the feature similarity with a preset similarity threshold.

[0128] S22. If the feature similarity is greater than the similarity threshold, it is determined that the fault category corresponding to the industrial robot is the fault category associated with the fault feature sample.

[0129] S23. If the feature similarity is less than or equal to the similarity threshold, in response to the input of the newly added fault category, the fault category corresponding to the industrial robot is determined as the newly added fault category.

[0130] It should be noted that the feature similarity is compared with a preset similarity threshold. Specifically, the preset similarity threshold is set by technical personnel according to actual needs, and the embodiment of the present invention does not limit this. If the feature similarity is greater than the similarity threshold, the fault category corresponding to the industrial robot is consistent with the fault category associated with the fault feature sample, so the fault category corresponding to the industrial robot is determined to be the fault category associated with the fault feature sample.

[0131] In a specific embodiment, if the feature similarity is less than or equal to the similarity threshold, the fault category corresponding to the industrial robot is inconsistent with the fault category associated with the fault feature sample, and manual confirmation of the fault status is required, analyzing the fault category, and setting the fault category as a new fault category.

[0132] Before executing the step of responding to the input of the newly added fault category and determining the fault category corresponding to the industrial robot as the newly added fault category, the following steps S231-S232 are also included:

[0133] S231, obtaining the remaining samples except the fault feature samples from the preset sample set as new fault feature samples;

[0134] S232, jump to execute the step of combining the operation feature and the fault feature samples into feature sample pairs until there are no remaining samples in the sample set.

[0135] It should be noted that the sample set refers to a set of known fault feature samples and fault types associated with the fault feature samples.

[0136] In this embodiment, before the fault state is manually confirmed and the fault category is analyzed, the remaining samples except the fault feature samples are obtained from the preset sample set as new fault feature samples. If the feature similarity corresponding to the feature sample pair is less than or equal to the similarity threshold, the step of forming a feature sample pair with the operating feature and the fault feature sample is repeated until there are no remaining samples in the sample set. It can be seen that the fault category corresponding to the operating feature is a new unknown fault, and it is necessary to manually confirm the fault state and analyze the fault category. Specifically, the preset sample set is set according to the fault feature samples actually obtained and the fault category associated with the fault feature samples, and the embodiment of the present invention does not limit this.

[0137] The method further comprises the following steps S31-S35:

[0138] S31, constructing an initial twin neural network using preset model parameters;

[0139] S32, inputting the preset feature training set into the initial twin neural network in sequence, and outputting the training results respectively;

[0140] S33. Use the preset loss function combined with all training results to calculate the cross entropy corresponding to the initial twin neural network.

[0141] S34. When the cross entropy is greater than the preset training threshold, the model parameters are adjusted according to the preset gradient, and the preset feature training set is input into the initial twin neural network in sequence, and the training results are output respectively.

[0142] S35. When the cross entropy is less than or equal to the training threshold, the initial twin neural network at the current moment is determined as the target twin neural network.

[0143] It should be noted that the twin neural network refers to a pattern recognition method based on sample similarity. This method uses two weight-sharing sub-networks to simultaneously receive two input samples, and the output result is the similarity of the two input samples. Figure 6 As shown in Figure 1, the sub-network structure of the Siamese Network (SN) includes a convolutional layer, a pooling layer, three multi-scale convolutional layers, a multi-head attention mechanism, a global maximum pooling layer, and a fully connected layer. The feature training set refers to the feature data samples used for model fitting.

[0144] In this embodiment, the preset feature training set is input into the initial twin neural network. When the number of training samples in the feature training set is n, two training samples are input into the twin neural network for each training. n 2 Therefore, in the case of small samples, the twin neural network can expand the number of simulated training times, deeply explore the relationship between different samples, avoid problems such as network underfitting due to insufficient sample number, and better solve the difficulties of industrial robot fault diagnosis under small samples in actual production.

[0145] In a specific embodiment, a training sample pair of a preset feature training set is randomly selected ( , ) are respectively input into the two identical sub-neural network modules of the initial twin neural network. The two sub-neural network modules have shared parameter weights w and bias b. The low-dimensional features corresponding to the training samples are extracted from the two training samples through the weight-sharing sub-networks. The Euclidean distance is used as a metric in the low-dimensional space to calculate the similarity of the low-dimensional features corresponding to the two training samples. Specifically, the formula for using the Euclidean distance as a metric is as follows:

[0146] (7)

[0147] Where: and represents the input training sample pair, and Represents the low-dimensional features extracted by the sub-network for the input training sample; Represents the Euclidean distance of low-dimensional features; operator Indicates calculating the Euclidean distance.

[0148] The output of the twin neural network is the difference between the sample pairs, that is, the probability that the sample pairs come from the same fault category. With input When the input samples are from the same fault category, the output difference is a smaller value, indicating that the similarity between the input training samples is large and the probability of them being from the same fault category is high. With input When the output difference is from different fault categories, it is a larger value, indicating that the similarity of the training input sample pairs is small and the probability of being from the same fault category is small. Specifically, the output calculation formula is as follows:

[0149] (8)

[0150] Where: sigm represents the sigmoid function: Indicates the difference between training sample pairs.

[0151] Using cross entropy as the loss function of the twin neural network, the loss function of a single training is as follows:

[0152] (9)

[0153] Where: y = 1 means and belongs to For the same fault category, y=0 means and Do not belong to the same fault category.

[0154] After calculating the specific value of the cross entropy, compare the cross entropy value with the preset training threshold. When the cross entropy is greater than the preset training threshold, adjust the shared parameter weight w and bias b according to the preset gradient, and jump to execute a training sample pair of the preset feature training set ( , ) are respectively input into two identical sub-neural network modules of the initial twin neural network, and the training results are outputted until the cross entropy is less than or equal to the training threshold, and the initial twin neural network at the current moment can be determined as the target twin neural network. Specifically, the preset gradient is set according to the actual operation, and the embodiment of the present invention does not limit this.

[0155] The present invention detects fault diagnosis requests in real time to obtain fault diagnosis request information in the first time. When a fault diagnosis request is detected, the fault diagnosis request is parsed to determine the industrial robot corresponding to the fault diagnosis request. At this time, the operation features are extracted from the periodic operation data corresponding to the industrial robot, and the extracted operation features and fault feature samples are combined into feature sample pairs, and the feature sample pairs are input into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pairs, and the feature similarity is compared with the preset similarity threshold. According to the comparison result, the fault category corresponding to the industrial robot is determined, thereby solving the technical problem that the existing deep learning model fault diagnosis technology uses a large amount of data to train the model, and the industrial robot fault signal can be diagnosed after the training is completed, but the industrial robot fault data is difficult to obtain under natural conditions, resulting in low generalization ability of the trained model and difficulty in quickly adapting to different usage scenarios. The present invention uses a twin neural network structure combined with a new intelligent fault diagnosis model of a subnetwork to realize automatic feature extraction and fault identification. The twin neural network structure effectively expands the number of model training times, deeply mines the relationship between different samples, avoids problems such as network underfitting caused by insufficient sample number, and achieves better fault diagnosis effect in small sample conditions, and is better applied to actual production and manufacturing.

[0156] See also Figure 7 , Figure 7This is a structural block diagram of an industrial robot fault diagnosis system provided in Embodiment 3 of the present invention.

[0157] An embodiment of the present invention provides an industrial robot fault diagnosis system, comprising:

[0158] A fault diagnosis request module 301 is used to determine the industrial robot corresponding to the fault diagnosis request when a fault diagnosis request is detected;

[0159] An operation feature extraction module 302 is used to extract operation features from periodic operation data corresponding to the industrial robot;

[0160] A feature sample pair module 303 is used to combine the operation feature and the fault feature sample into a feature sample pair;

[0161] A feature similarity module 304 is used to input the feature sample pair into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair;

[0162] The feature similarity comparison module 305 is used to compare the feature similarity with a preset similarity threshold, and determine the fault category corresponding to the industrial robot according to the comparison result.

[0163] Optionally, the operation feature extraction module 302 includes:

[0164] The feedback speed submodule is used to obtain the feedback speed from the periodic operation data corresponding to the industrial robot;

[0165] The acceleration operation submodule is used to select the acceleration operation stage; wherein the feedback speed in the acceleration operation stage is greater than a predetermined threshold;

[0166] A submodule for obtaining feedback torque or feedback current is used to obtain feedback torque or feedback current corresponding to the acceleration operation stage;

[0167] The operation characteristic submodule is used to extract the time-frequency domain characteristics of the feedback torque or feedback current in the time-frequency domain as the operation characteristics.

[0168] Optionally, the feature similarity module 304 includes:

[0169] The input convolution layer submodule is used to input the operation feature and the fault feature sample in the feature sample pair into the convolution layer respectively, and convolve the operation feature and the fault feature sample respectively through the convolution layer to generate the first convolution features corresponding to the operation feature and the fault feature sample respectively and transmit them to the pooling layer;

[0170] The first pooling feature submodule is used to reduce the dimension of the first convolution feature through the pooling layer, generate the first pooling feature and transmit it to the multi-scale convolution layer.

[0171] The second convolution feature submodule is used to convolve the first pooling features respectively using multiple convolution kernels through a multi-scale convolution layer to generate multiple second convolution features and transmit them to the global maximum pooling layer.

[0172] The second pooling feature submodule is used to perform a global maximum pooling operation on all second convolutional features through a global maximum pooling layer, generate multiple second pooling features and transmit them to the fully connected layer.

[0173] The target feature submodule is used to classify all the second pooling features through the fully connected layer to obtain the target features.

[0174] The feature similarity submodule is used to calculate the Euclidean distance between target features and obtain the feature similarity corresponding to the feature sample pair.

[0175] The multi-head attention mechanism layer submodule is used to input the second convolutional features into the multi-head attention mechanism layer.

[0176] The input vector submodule is used to convert the second convolutional features into multiple input vectors through a multi-head attention mechanism layer.

[0177] The scaled dot product attention result submodule is used to map multiple input vectors to multiple subspaces for scaled dot product attention calculation and generate multiple scaled dot product attention results.

[0178] The new second convolutional feature submodule is used to concatenate all scaled dot product attention results to generate a new second convolutional feature.

[0179] Optionally, the feature similarity comparison module 305 may include:

[0180] A similarity comparison submodule is used to compare feature similarity with a preset similarity threshold;

[0181] If the feature similarity is greater than the similarity threshold, it is determined that the fault category corresponding to the industrial robot is the fault category associated with the fault feature sample;

[0182] If the feature similarity is less than or equal to the similarity threshold, then in response to the inputted new fault category, the fault category corresponding to the industrial robot is determined as the new fault category.

[0183] Optionally, the feature similarity comparison module 305 further includes:

[0184] The new fault feature sample submodule is used to obtain the remaining samples except the fault feature samples from the preset sample set as new fault feature samples.

[0185] The jump execution submodule is used to jump execution of the step of combining the operation feature and the fault feature sample into a feature sample pair until there are no remaining samples in the sample set.

[0186] The system also includes:

[0187] Build a twin neural network module to build an initial twin neural network using preset model parameters.

[0188] The training result output module is used to input the preset feature training set into the initial twin neural network in sequence and output the training results respectively.

[0189] The cross entropy calculation module is used to calculate the cross entropy corresponding to the initial twin neural network using a preset loss function combined with all training results.

[0190] A cross entropy comparison module is used to adjust the model parameters according to the preset gradient when the cross entropy is greater than the preset training threshold, jump to the step of inputting the preset feature training set into the initial twin neural network in sequence, and output the training results respectively;

[0191] When the cross entropy is less than or equal to the training threshold, the initial twin neural network at the current moment is determined as the target twin neural network.

[0192] The present invention detects fault diagnosis requests in real time to obtain fault diagnosis request information in the first time. When a fault diagnosis request is detected, the fault diagnosis request is parsed to determine the industrial robot corresponding to the fault diagnosis request. At this time, the operation features are extracted from the periodic operation data corresponding to the industrial robot, and the extracted operation features and fault feature samples are combined into feature sample pairs, and the feature sample pairs are input into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pairs, and the feature similarity is compared with the preset similarity threshold. According to the comparison result, the fault category corresponding to the industrial robot is determined, thereby solving the technical problem that the existing deep learning model fault diagnosis technology uses a large amount of data to train the model, and the industrial robot fault signal can be diagnosed after the training is completed, but the industrial robot fault data is difficult to obtain under natural conditions, resulting in low generalization ability of the trained model and difficulty in quickly adapting to different usage scenarios. The present invention uses a twin neural network structure combined with a new intelligent fault diagnosis model of a subnetwork to realize automatic feature extraction and fault identification. The twin neural network structure effectively expands the number of model training times, deeply mines the relationship between different samples, avoids problems such as network underfitting caused by insufficient sample number, and achieves better fault diagnosis effect in small sample conditions, and is better applied to actual production and manufacturing.

[0193] An embodiment of the present invention further provides an electronic device, comprising: a memory and a processor, wherein a computer program is stored in the memory; when the computer program is executed by the processor, the processor executes the industrial robot fault diagnosis method as described in any of the above embodiments.

[0194] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the industrial robot fault diagnosis method according to any embodiment of the present invention is implemented.

[0195] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and media can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0196] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0197] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0198] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0199] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0200] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for diagnosing faults of industrial robots, characterized in that: The method comprises: When a fault diagnosis request is detected, determining the industrial robot corresponding to the fault diagnosis request; Extracting operation features from periodic operation data corresponding to the industrial robot; The operation feature and the fault feature sample form a feature sample pair; Inputting the feature sample pair into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair; Comparing the feature similarity with a preset similarity threshold, and determining a fault category corresponding to the industrial robot according to the comparison result; The target twin neural network includes a convolution layer, a pooling layer, a multi-scale convolution layer, a multi-head attention mechanism layer, a global maximum pooling layer and a fully connected layer; Inputting the second convolution feature output by the multi-scale convolution layer into the multi-head attention mechanism layer, and converting the second convolution feature into multiple input vectors through the multi-head attention mechanism layer; Mapping the plurality of input vectors to a plurality of subspaces to perform scaled dot product attention calculations, and generating a plurality of scaled dot product attention results; All the scaled dot product attention results are concatenated to generate a new second convolutional feature.

2. The industrial robot fault diagnosis method according to claim 1, characterized in that: The step of extracting operation features from the periodic operation data corresponding to the industrial robot comprises: Obtaining feedback speed from periodic operation data corresponding to the industrial robot; Selecting an acceleration operation stage; wherein the feedback speed in the acceleration operation stage is greater than a predetermined threshold; Obtaining feedback torque or feedback current corresponding to the acceleration operation stage; The time-frequency domain features of the feedback torque or the feedback current in the time-frequency domain are extracted as operation features.

3. The industrial robot fault diagnosis method according to claim 1, characterized in that: The step of inputting the feature sample pair into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair includes: Inputting the operation feature and the fault feature sample in the feature sample pair into the convolution layer respectively, performing convolution on the operation feature and the fault feature sample respectively through the convolution layer, generating first convolution features corresponding to the operation feature and the fault feature sample respectively, and transmitting the first convolution features to the pooling layer; Performing dimension reduction on the first convolutional feature through the pooling layer to generate a first pooling feature and transmitting it to the multi-scale convolutional layer; Using the multi-scale convolution layer to convolve the first pooling features respectively with multiple convolution kernels, generate multiple second convolution features and transmit them to the multi-head attention mechanism layer, output new second convolution features, and transmit the new second convolution features to the global maximum pooling layer; Performing a global maximum pooling operation on all of the new second convolutional features through the global maximum pooling layer to generate a plurality of second pooling features and transmitting them to the fully connected layer; Classifying all the second pooling features through the fully connected layer to obtain target features; The Euclidean distance between the target features is calculated to obtain the feature similarity corresponding to the feature sample pair.

4. The industrial robot fault diagnosis method according to claim 1, characterized in that: The step of comparing the feature similarity with a preset similarity threshold and determining the fault category corresponding to the industrial robot according to the comparison result includes: Comparing the feature similarity with a preset similarity threshold; If the feature similarity is greater than the similarity threshold, determining that the fault category corresponding to the industrial robot is the fault category associated with the fault feature sample; If the feature similarity is less than or equal to the similarity threshold, then in response to the inputted new fault category, the fault category corresponding to the industrial robot is determined as the new fault category.

5. The industrial robot fault diagnosis method according to claim 4, characterized in that: Before the step of responding to the inputted newly added fault category and determining the fault category corresponding to the industrial robot as the newly added fault category, the method further includes: Acquire remaining samples except the fault feature samples from a preset sample set as new fault feature samples; Jump to the step of combining the operation feature and the fault feature sample into a feature sample pair until the remaining samples do not exist in the sample set.

6. The industrial robot fault diagnosis method according to claim 1, characterized in that: Also includes: Construct an initial twin neural network using preset model parameters; Inputting the preset feature training set into the initial twin neural network in sequence, and outputting the training results respectively; Using a preset loss function combined with all the training results, the cross entropy corresponding to the initial twin neural network is calculated; When the cross entropy is greater than a preset training threshold, the model parameters are adjusted according to a preset gradient, and the step of sequentially inputting the preset feature training set into the initial twin neural network and outputting the training results is skipped; When the cross entropy is less than or equal to the training threshold, the initial twin neural network at the current moment is determined as the target twin neural network.

7. An industrial robot fault diagnosis system, characterized in that: include: A fault diagnosis request module, configured to, when a fault diagnosis request is detected, determine the industrial robot corresponding to the fault diagnosis request; An operation feature extraction module, used to extract operation features from periodic operation data corresponding to the industrial robot; A feature sample pair module, used for combining the operation feature and the fault feature sample into a feature sample pair; A feature similarity module, used to input the feature sample pair into a preset target twin neural network to obtain the feature similarity corresponding to the feature sample pair; A feature similarity comparison module is used to compare the feature similarity with a preset similarity threshold, and determine the fault category corresponding to the industrial robot according to the comparison result; The target twin neural network includes a convolution layer, a pooling layer, a multi-scale convolution layer, a multi-head attention mechanism layer, a global maximum pooling layer and a fully connected layer; The feature similarity module is specifically used for: Inputting the second convolution feature output by the multi-scale convolution layer into the multi-head attention mechanism layer, and converting the second convolution feature into multiple input vectors through the multi-head attention mechanism layer; Mapping the plurality of input vectors to a plurality of subspaces to perform scaled dot product attention calculations, and generating a plurality of scaled dot product attention results; All the scaled dot product attention results are concatenated to generate a new second convolutional feature.

8. An electronic device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the industrial robot fault diagnosis method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the industrial robot fault diagnosis method according to any one of claims 1 to 6 is implemented.

Images