Industrial robot fault diagnosis model and fault diagnosis method

Abstract

The present invention discloses an industrial robot fault diagnosis model, comprising: a time series feature extraction module for extracting time series features and concatenating and fusing the memory information output from each time window. The resulting time series feature graph serves as the output of the entire time series feature extraction module; a multi-scale convolution module, comprising multiple feature extraction branches arranged in parallel, each of which extracts features from the time series feature graph, and merging the feature graphs extracted by all the feature extraction branches to obtain a fault category feature vector; and a multi-label classifier, comprising multiple fault label modules arranged in parallel, each of which includes a classifier corresponding to a component in a corresponding joint that may have a fault. Each classifier has an independent loss function and converts the fault category feature vector into a category probability vector. The present invention also discloses an industrial robot fault diagnosis method.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial equipment fault diagnosis, and specifically relates to an industrial robot fault diagnosis model and a fault diagnosis method. Background Art

[0002] As my country's automotive manufacturing industry continues to transform and upgrade towards automation and intelligent manufacturing, production scenarios are becoming increasingly complex and diverse. Automakers are placing increasing demands on all aspects of automated production equipment, particularly in terms of safety and reliability. Welding robots, core equipment in the body-in-white (BIW) assembly line, are vulnerable to sudden failures, disrupting production schedules, reducing BIW assembly quality, lowering production efficiency, and even posing threats to employee health and safety. Consequently, manufacturers place high demands on the safety, reliability, and operational stability of BIW welding robots. BIW welding robots are multifunctional, intelligent robotic arms specialized for BIW frame welding. They offer advanced intelligence, cost-effectiveness, and high safety and reliability, making them essential precision production equipment in the automotive production process. However, in complex and diverse production scenarios, welding robots often experience unforeseen failures, resulting in reduced positioning accuracy, suboptimal welding quality, unplanned downtime for the entire BIW assembly line, and even threatening worker safety. For manufacturers, sudden, unexpected failures and malfunctions in the welding robot system can bring the entire line to a standstill. If a faulty robot is not repaired promptly, it could lead to a major production accident. To prevent unplanned and unexpected anomalies and failures in welding robots, most current automobile manufacturers rely on the traditional method of regular manual inspection and maintenance. If an anomaly is detected in a welding robot, a comprehensive inspection of the abnormal robot is required to gradually locate the faulty component, which is extremely time-consuming and labor-intensive. Traditional methods of monitoring and diagnosing anomalies in welding robots are extremely wasteful of corporate resources and result in reduced production efficiency. Therefore, it is extremely necessary to conduct research on rapid fault diagnosis methods for welding robots to help engineers promptly detect early abnormalities in the robot and complete fault diagnosis and location based on the anomaly, thereby enabling timely and targeted component repairs and providing an effective reference for planned maintenance work on the production line.

[0003] Mechanical equipment fault diagnosis research is a comprehensive, multidisciplinary field. Fault diagnosis systems analyze abnormal signals using techniques such as signal analysis and feature extraction. Combined with the fault symptoms, they identify the fault type, locate the faulty component, and identify the cause, helping engineers determine the appropriate repair plan. Traditional fault diagnosis methods, including feature signal analysis, statistical analysis, and expert experience systems, rely on established mathematical, physical, and empirical models, impose high requirements on signal data quality, and are often limited in their applicable scenarios and applications. In actual production, the lack of deep-level signals representing the operating status of equipment often leads to the failure of traditional fault detection methods in specific scenarios. Furthermore, the variability of equipment operating conditions and the inevitable signal noise significantly impact the performance of traditional methods, making them inapplicable to complex and diverse production scenarios. In recent years, with the rapid development of AI, big data, and high-performance computing clusters, machine learning and data-driven fault diagnosis methods have become cutting-edge topics in the field of fault diagnosis. Machine learning algorithms such as neural networks are used to learn from massive amounts of equipment operating signal data and extract deep signal features, enabling intelligent fault diagnosis for automated equipment. Summary of the Invention

[0004] In view of this, an object of the present invention is to provide an industrial robot fault diagnosis model and a fault diagnosis method, which can realize rapid diagnosis of abnormal faults of industrial robots.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention first proposes an industrial robot fault diagnosis model, including:

[0007] The time series feature extraction module is used to extract the time series features of the current, speed, rotation angle and operation stage signals of each joint of the industrial robot, and to splice and fuse the memory information output in each time window. The spliced and fused time series feature graph is used as the output result of the entire time series feature extraction module;

[0008] The multi-scale convolution module includes multiple feature extraction branches arranged in parallel. Each feature extraction branch uses a different receptive field convolution kernel combination to extract features from the time series feature map. The fault category feature vector obtained by combining the feature maps extracted by all feature extraction branches is used as the output of the multi-scale convolution module.

[0009] A multi-label classifier includes multiple fault label modules arranged in parallel. The fault label modules are arranged in a one-to-one correspondence with the joints of the industrial robot. Each fault label module includes a classifier corresponding to the component that may fail in the corresponding joint. Each classifier has an independent loss function and converts the fault category feature vector into a category probability vector.

[0010] Furthermore, the time series feature extraction module adopts LSTM neural network.

[0011] Furthermore, the multi-scale convolution module is provided with five feature extraction branches in parallel, namely:

[0012] The MSCM_a feature extraction branch includes a 1×1 convolution kernel to preserve the feature information in the temporal feature map as much as possible;

[0013] MSCM_b feature extraction branch, used to perform Max-Pooling operations to reduce the input data dimension and extract the main feature information;

[0014] The MSCM_c feature extraction branch includes a 1×1 convolution kernel and a 3×3 convolution kernel connected in series;

[0015] The MSCM_d feature extraction branch includes a 1×1 convolution kernel and a 5×5 convolution kernel connected in series;

[0016] The MSCM_e feature extraction branch includes a 1×1 convolution kernel and a 7×7 convolution kernel connected in series.

[0017] Furthermore, in the MSCM_c feature extraction branch, the 3×3 convolution kernel is split into a 1×3 convolution kernel and a 3×1 convolution kernel connected in series.

[0018] Furthermore, in the MSCM_d feature extraction branch, the 5×5 convolution kernel is split into a 3×3 convolution kernel, a 1×3 convolution kernel, and a 3×1 convolution kernel; among them, the 3×3 convolution kernel is connected in series with the 1×1 convolution kernel in the MSCM_d feature extraction branch, and the 1×3 convolution kernel is connected in parallel with the 3×1 convolution kernel and then connected in series with the 3×3 convolution kernel.

[0019] Furthermore, in the MSCM_e feature extraction branch, the 7×7 convolution kernel is split into two 3×3 convolution kernels, one 1×3 convolution kernel, and one 3×1 convolution kernel; wherein, the two 3×3 convolution kernels are connected in series with each other, one 3×3 convolution kernel is connected in series with the 1×1 convolution kernel in the MSCM_e feature extraction branch, and the 1×3 convolution kernel is connected in parallel with the 3×1 convolution kernel and then connected in series with another 3×3 convolution kernel.

[0020] Furthermore, the multi-scale convolution module also includes a global average pooling module, which is used to merge the feature maps extracted by all feature extraction branches and obtain the fault category feature vector.

[0021] Furthermore, the sigmoid function is used as the activation function in the classifier.

[0022] The present invention also proposes an industrial robot fault diagnosis method, comprising the following steps:

[0023] Step 1: Data collection: The raw data is collected through the industrial robot control software and the production line equipment database. The collected raw data is preprocessed and then a data set is constructed.

[0024] Step 2: Build the industrial robot fault diagnosis model as described above;

[0025] Step 3: Use the dataset to train the industrial robot fault diagnosis model and evaluate the effectiveness of the industrial robot fault diagnosis model;

[0026] Step 4: Use the industrial robot control software to collect the current, speed, rotation angle and operation stage signals of each joint of the industrial robot in real time to achieve rapid diagnosis and positioning of industrial robot faults.

[0027] The beneficial effects of the present invention are:

[0028] The industrial robot fault diagnosis model of the present invention first uses a timing feature extraction module to extract the timing features of the current, speed, rotation angle and operation stage signals of each joint of the robot, integrates the timing memory feature quantities of each flexible time window to obtain a two-dimensional timing feature graph, and then further extracts the deep spatial features of the robot operation signal through the proposed multi-scale convolution module, and finally completes the fault diagnosis through a multi-label classifier; by setting multiple feature extraction branches in parallel in the multi-scale convolution module, each feature extraction branch uses a different receptive field convolution kernel combination to extract features from the timing feature graph, so as to obtain multiple features at the same position in the feature graph, which can effectively avoid the problem of special features. Using a single convolution kernel will cause the network to focus only on a specific signal feature and may lose some key signal features, making it difficult to distinguish the differences between similarities in signal data of different categories. By setting a fault label module in a multi-label classifier corresponding to the joints of the industrial robot, and setting a classifier in each fault label module corresponding to the parts that may fail in the corresponding joint, each classifier uses an independent loss function and converts the fault category feature vector into a category probability vector. There is no need to limit the sum of all category probabilities to 1. The strategy of splitting the large single-label classification into multiple binary classifications enables the classifier to have a very fine classification granularity and effectively improves the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to make the purpose, technical solutions and beneficial effects of the present invention more clear, the present invention provides the following drawings for illustration:

[0030] Figure 1 This is a flow chart of an embodiment of the industrial robot fault diagnosis method of the present invention;

[0031] Figure 2 This is a structural diagram of an embodiment of an industrial robot fault diagnosis model of the present invention;

[0032] Figure 3 This is the structural diagram of the temporal feature extraction module;

[0033] Figure 4 This is the structural diagram of the LSTM neural network;

[0034] Figure 5 This is the structural diagram of the multi-scale convolution module;

[0035] Figure 6 This is the structural diagram of the multi-label classifier. DETAILED DESCRIPTION

[0036] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0037] This embodiment uses the analysis of the welding robot in the body-in-white welding process as an example to illustrate the fault diagnosis method of industrial robots. Before performing fault diagnosis, it is necessary to first analyze the body-in-white welding process and the composition structure of the production line, and analyze the working characteristics, structure and performance of the welding robot; then, briefly analyze the forms and characteristics of abnormalities and faults of the welding robot, and propose an overall fault diagnosis scheme for the welding robot. Fault diagnosis is performed based on the overall fault diagnosis scheme, such as Figure 1 As shown, the industrial robot fault diagnosis method of this embodiment includes the following steps:

[0038] Step 1: Data collection: The raw data is collected through the industrial robot control software and the production line equipment database, and the collected raw data is preprocessed to construct a data set.

[0039] The welding robot's operational signal data is collected in real time by sensors within each joint, fed back to the controller software, and stored in the production line's network cabinet. This embodiment utilizes the welding robot's controller software, BOS6000, and the welding line's equipment database to complete data collection. Due to differences in robot and sensor models within the welding line, as well as fluctuations in signal acquisition frequency, the raw data collected from the production line can contain missing values, anomalies, and inconsistent data formats. This requires necessary preprocessing of the raw data, including grouping, gap filling, and data conversion.

[0040] Step 2: Build an industrial robot fault diagnosis model (LtcmNet).

[0041] like Figure 2 As shown, the industrial robot fault diagnosis model of this embodiment includes a time series feature extraction module, a multi-scale convolution module and a multi-label classifier.

[0042] (1) The time series feature extraction module is used to extract the time series features of the current, speed, rotation angle and operation stage signals of each joint of the industrial robot, and store the memory information h output in each time window. t The splicing and fusion are performed, and the time series feature graph obtained by splicing and fusion is used as the output result of the entire time series feature extraction module. Figure 3 As shown, in this embodiment, the time series feature extraction module adopts LSTM neural network.

[0043] Welding robots often follow preset procedures during welding operations, so the robot's operational signal data often exhibits time series characteristics. To mine the time series characteristics of the robot's joint signals, the LtcmNet head uses LSTM to extract time series features from the robot's joint current, speed, rotation angle, and operational phase signals.

[0044] Long Short-Term Memory (LSTM) is a time-recurrent neural network. Due to its unique design structure, LSTM is suitable for processing and predicting important events with very long intervals and delays in time series. Its structure is similar to that of an ordinary recurrent neural network. It achieves "relatively long" memory by introducing three gate units, and is mainly used to solve longer short-interval dependencies. The most important units in LSTM are the three control gates: input gate, forget gate, and output gate. The input gate controls the input of the network; the forget gate, as the core of LSTM, determines which knowledge or information needs to be retained and remembered, and which needs to be forgotten and removed, which is equivalent to the memory unit in the recurrent neural network; the output gate controls the output of the network. The structure of the LSTM neural network is as follows: Figure 4 shown.

[0045] The structure and principle of LSTM neural network:

[0046] ① Forget gate: decides which information to keep and discard from historical "memory". It uses the output h of the previous moment t-1 and the current input x t The combination is used as input, and the σ function outputs a value between [0,1] as the memory weight acting on the memory c of the previous moment. t-1 , which determines the degree of memory and forgetting. 0 means discarding the memory of the previous moment, and 1 means completely retaining the memory of the previous moment. The σ function often uses a function similar to Sigmoid, and its output range is [0,1], which represents the weight of memory information selection and rejection. The mathematical expression of the forget gate is:

[0047] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0048] ② Input gate: Determines memory updates and uses the σ function, whose input is the output of the previous moment and the input of the current moment. Similar to the forget gate, the σ function returns a value between [0, 1] as the memory update weight, which is used to determine which new memories need to be remembered and updated to the old knowledge base. Therefore, the input gate integrates the memories that need to be updated at the current time and the memories that need to be retained at the previous moment selected by the forget gate to obtain the new memory. Its mathematical expression is:

[0049]

[0050]

[0051] The result of formula 4.2 is i t and Doing element-wise multiplication gives the information to be remembered at the current moment t, and then using formula 4.3 to update it to the old memory, we get the latest memory c at the current moment t. t . c t As the latest memory information at the current moment and the last moment c t-1 The same will be passed on to the next moment.

[0052] ③ Output gate: Use the standard Sigmoid function to act on the current latest memory information to obtain the probability vector o t And use the tanh activation function to reshape the distribution of the current memory and output the probability vector o t After multiplication, we get the output vector h t The mathematical expression of the output gate is:

[0053]

[0054] The LSTM network can be composed of multiple cells, each of which contains the three control gate structures mentioned above. This unique functional structure enables LSTM to achieve good results in the task of extracting data time series features.

[0055] The temporal feature extraction module proposed in this embodiment is based on the LSTM network, which collects 20 channels of signals, including the motor current, speed, joint rotation angle and robot operation phase of the six joints of the industrial robot. r After normalization, it is used as the data input of the time series feature extraction module. The LSTM neural network is used to model the signal data and extract the time series features. The memory information h output by each time window is converted into t The splicing and fusion are performed as the output result of the entire temporal feature extraction module.

[0056] (2) A multi-scale convolution module, including multiple feature extraction branches arranged in parallel, each feature extraction branch uses a different receptive field convolution kernel combination to extract features from the time series feature map, and the fault category feature vector obtained by merging the feature maps extracted by all feature extraction branches is used as the output of the multi-scale convolution module. Specifically, the multi-scale convolution module of this embodiment also includes a global average pooling module, which is used to merge the feature maps extracted by all feature extraction branches and obtain the fault category feature vector. Figure 5 As shown, in this embodiment, five feature extraction branches are arranged in parallel in the multi-scale convolution module, namely:

[0057] The MSCM_a feature extraction branch includes a 1×1 convolution kernel to preserve the feature information in the temporal feature map as much as possible;

[0058] MSCM_b feature extraction branch, used to perform Max-Pooling operations to reduce the input data dimension and extract the main feature information;

[0059] The MSCM_c feature extraction branch includes a 1×1 convolution kernel and a 3×3 convolution kernel connected in series. In a preferred embodiment of the present invention, the 3×3 convolution kernel is split into a 1×3 convolution kernel and a 3×1 convolution kernel connected in series.

[0060] The MSCM_d feature extraction branch includes a 1×1 convolution kernel and a 5×5 convolution kernel connected in series. In a preferred embodiment of this embodiment, the 5×5 convolution kernel is split into a 3×3 convolution kernel, a 1×3 convolution kernel, and a 3×1 convolution kernel. The 3×3 convolution kernel is connected in series with the 1×1 convolution kernel in the MSCM_d feature extraction branch, and the 1×3 convolution kernel is connected in parallel with the 3×1 convolution kernel and then connected in series with the 3×3 convolution kernel.

[0061] The MSCM_e feature extraction branch includes a 1×1 convolution kernel and a 7×7 convolution kernel connected in series. In a preferred implementation of this embodiment, the 7×7 convolution kernel is split into two 3×3 convolution kernels, a 1×3 convolution kernel, and a 3×1 convolution kernel. The two 3×3 convolution kernels are connected in series, one of the 3×3 convolution kernels is connected in series with the 1×1 convolution kernel in the MSCM_e feature extraction branch, and the 1×3 convolution kernel is connected in parallel with the 3×1 convolution kernel and then connected in series with another 3×3 convolution kernel.

[0062] The LtcmNet head uses LSTM to complete the signal timing feature extraction task, and splices and fuses the memory information obtained in each time window to obtain a complete timing feature map of the signal data as the input of the subsequent network model. The strategy of splicing and fusing the memory output of each time step of LSTM is different from the strategy of only obtaining the output of the last cell. Although it can obtain shallow signal features and retain more comprehensive information, it will also accumulate useless information in the signal and cannot extract deep signal features. Therefore, this embodiment introduces a multi-scale convolution module to make up for the above shortcomings.

[0063] CNN achieves the purpose of feature extraction by sliding on the feature vector or matrix through a shared "convolution kernel" structure. During the convolution process, some old features will be discarded, and new features will be included at the same time, which reduces the weight parameter scheduling, promotes the network's processing ability for high-dimensional data, and greatly reduces the difficulty of feature engineering. CNN is often used in scenarios with huge feature input characteristics such as speech, video, image and natural language processing, and is also applicable to signal data processing scenarios. When applying CNN to extract features from signals, the traditional method is to convert the signal data into one dimension and then input it into the CNN network to complete the feature extraction task. However, the signal data of this embodiment is the periodic time series data of the current, speed and operation stage of the six joints of the welding robot. In addition to the time series characteristics hidden in the time domain of each joint current channel, speed channel, and operation stage channel, the three-channel signals of each joint also contain connections in the spatial domain. Considering that the current, speed, rotation angle, and operation phase signal data may be covered by strong background noise, resulting in the weakening of fault-related feature information, this embodiment adopts a two-dimensional multi-scale feature extraction strategy to perform deeper learning on the input signal data, fully extract the correlation characteristics between the various signals of the robot joints, and name the proposed multi-scale convolution module as the MSCM convolution module.

[0064] For the robot operation periodic signal data of this embodiment, the periodic signal data of different fault categories are very similar. If a specific single convolution kernel is used, the network will only focus on a specific signal feature, and some key signal features may be lost, which makes it difficult to distinguish the differences between similarities in signal data of different categories. In order to improve the generalization ability of the model, MSCM introduces convolution kernels of multiple sizes to perform parallel extraction operations on feature maps, and obtain multiple features at the same position in the feature map. The structure of the MSCM convolution module of this embodiment is shown in Figure 5, which consists of five branches, each branch extracts features from the input feature map, and finally merges the feature map outputs of the five branches to obtain a set of deep feature maps. Figure 3 Shown is the MSCM module structure diagram.

[0065] In the MSCM module, the first branch uses a 1×1 convolution kernel to preserve as much feature information as possible in the feature map. The second branch is a simple max-pooling operation, reducing the dimensionality of the input data and extracting key feature information. The third branch consists of a 1×1 convolution kernel and a 3×3 convolution kernel. The 3×3 convolution kernel is split into two asymmetric convolution kernels. The resulting 1×3 and 3×1 convolution kernels are equivalent to a single 3×3 convolution kernel. This convolution kernel splitting strategy significantly reduces network parameters while maintaining performance. The fourth branch consists of a 1×1 convolution kernel and a 5×5 convolution kernel. The 5×5 convolution kernel, with its larger receptive field, is used here to enhance the MSCM module's ability to extract spatial position relationship features. Considering the surge in network parameters caused by increasing the receptive field, a similar strategy as the third branch is adopted: the 5×5 convolution kernel is split into a parallel structure consisting of a 3×3 convolution kernel and 1×3 and 3×1 asymmetric convolution kernels. The fifth branch consists of a 1×1 convolution kernel and a 7×7 convolution kernel. Similar to the fourth branch, the 7×7 convolution kernel is decomposed accordingly.

[0066] The five branches of the MSCM module each use a combination of convolution kernels with different receptive fields to extract features from the feature map, and employ a structure that decomposes large convolution kernels into multiple asymmetric convolution kernels. The MSCM module rationally reduces dimensionality across its branches without compromising the network's feature extraction capabilities. Furthermore, by activating more output branches, it produces decoupled feature representations, generating high-order sparse features and accelerating convergence.

[0067] (3) A multi-label classifier, comprising a plurality of fault label modules arranged in parallel, wherein the fault label modules are arranged in one-to-one correspondence with the joints of the industrial robot, and each fault label module includes a classifier corresponding to the parts that may fail in the corresponding joint, and each classifier has an independent loss function and converts the fault category feature vector into a category probability vector.

[0068] Welding robots are widely used in automotive welding and assembly due to their high intelligence, high production efficiency, high safety, and significant economic benefits. Considering that welding robots are highly automated precision mechanical equipment composed of numerous complex components, during long, high-intensity, and variable-load production processes, they are highly likely to experience multiple component failures or combined failures of a single component. For example, a reduction gear and motor failure in a particular robot joint, or simultaneous failures in multiple joints, can occur. While combined failures can be assigned a single fault label, for the purposes of this example, each of the welding robot's six joints could experience a reduction gear or motor failure, and the fault labels for each joint are independent, resulting in a total of 2^12 possible classifications. However, a class space of 2^12 is excessively large. Using a simple single-label classifier to handle such a vast class space would significantly increase training pressure and the difficulty of model convergence, making this an unwise choice. To address these issues, this example employs a multi-label classifier as the tail of LtcmNet. Its function is to receive the feature vector output of the MSCM module and perform the multi-label classification task.

[0069] Traditional CNN networks often use Softmax as an activation function combined with a cross-entropy loss function when dealing with multi-classification problems. The Softmax function takes a k-dimensional row vector as input and outputs a k-dimensional row vector, with each dimension in the (0,1) interval and summing to 1. This function converts the category feature vectors of the convolutional network output layer into corresponding category feature probability vectors, so that the sum of the output values of each category in the output layer is 1. The model ultimately returns the category with the highest probability value as the overall model's discrimination result. The mathematical expression for the Softmax function is:

[0070]

[0071] The essence of the softmax operation is to convert the individual possibilities of the prediction results of each category into a probability value in the overall prediction. This method is only applicable to single-label multi-fault classification and not to complex fault diagnosis. To this end, multiple parallel sigmoid activation functions are introduced as the main structure of the multi-label classifier. Sigmoid is a nonlinear activation function, and its mathematical expression is:

[0072]

[0073] The sigmoid function graph presents an "S" shape. The sigmoid function can transform the continuous numerical value of the input into a value within the interval (0,1), which is equivalent to numerical compression. The multi-label classifier of this embodiment splits the complex combined fault categories of the welding robot into fault labels of the motors and reducers of 6 independent joints, and sets 6 fault labels respectively. Each fault label module includes a classifier corresponding to the parts that may fail in the corresponding joint. In this embodiment, the parts that may fail include reducers and motors, that is, each fault label module includes 2 classifiers, which perform fault diagnosis for the reducer and motor of each joint separately. In this way, Figure 6 As shown in the figure, 12 sigmoid functions are used in parallel to replace a single softmax function classifier. This essentially splits the original 2^12 single-label category space into 12 single-label category spaces. Each category has its own independent loss function and does not need to constrain the sum of all category probabilities to 1. This strategy of splitting a large single-label classification into multiple binary categories enables the classifier to have very fine classification granularity and effectively improves accuracy. Figure 4 Schematic diagram of the multi-label classifier structure.

[0074] Step 3: Use the dataset to train the industrial robot fault diagnosis model and evaluate the effectiveness of the industrial robot fault diagnosis model.

[0075] Step 4: Use the industrial robot control software to collect the current, speed, rotation angle and operation stage signals of each joint of the industrial robot in real time to achieve rapid diagnosis and positioning of industrial robot faults.

[0076] Specifically, in this embodiment, LtcmNet is divided into three components: head (time series feature extraction module), body (multi-scale convolution module) and tail (multi-label classifier):

[0077] (1)LtcmNet header

[0078] The LtcmNet head consists of an LSTM network, which extracts temporal features from robot signal data and concatenates and fuses the memory information acquired in each time window to output a temporal feature map. A single-layer LSTM network is used. The number of hidden layers is set to 128, as shown in the model comparison experimental results in the next section. The input dimension is 20. It should be noted that there is a lack of online robot fault data on the welding production line. Therefore, the dataset for this model includes fault signal data collected by engineers during offline debugging and maintenance of faulty robots. However, the debugging and maintenance process does not follow a fixed debugging procedure, resulting in the collected fault signals being non-periodic and of varying lengths. To address this issue, LtcmNet employs a flexible time window strategy. Specifically, the time window length T in the LSTM network in the model head can be flexibly adjusted based on the input data length, ensuring that the temporal feature map output by the model head has a uniform shape of 1×130×128.

[0079] (2)LtcmNet body

[0080] The LtcmNet body consists of a multi-scale deep convolutional layer consisting of multiple stacked MSCM multi-scale convolutional modules. This layer is used to extract deep features from time series feature maps. This layer contains five MSCM modules: MSCM_a, MSCM_b, MSCM_c, MSCM_d, and MSCM_e. The global average pooling 2D (GAP2D) module is used to output the fault category feature vector. Table 1 shows the structural parameters of the LtcmNet multi-scale convolutional layer.

[0081] Table 4.1LtcmNet multi-scale convolution layer structure parameters

[0082]

[0083] (3)LtcmNet tail

[0084] The model's tail consists of a multi-label classifier, which uses a structure with 12 parallel sigmoid activation functions. The multi-label classifier receives the fault category feature vector (1×12) output by the global average pooling layer (GAP2D), performs label fusion and fault discrimination, and ultimately outputs the fault diagnosis result.

[0085] This embodiment constructs the LtcmNet model based on LSTM and CNN to solve the problem of fault diagnosis of the body-in-white welding robot. First, a flexible time window LSTM is proposed to extract the time series feature diagram of the robot's fault operation signal. The flexible time window width can be flexibly adjusted according to the length of the input signal data, so that the LtcmNet model has the ability to process non-equal length signal data. Then, the MSCM multi-scale convolution module is proposed as the body of LtcmNet. The MSCM module consists of five parallel convolution branches with different receptive fields. It can fully extract the correlation characteristics between the various signals of the robot joints, and the features extracted by each convolution branch can achieve information complementarity, greatly retaining the overall and local spatial feature information of the signal, while weakening the interference of industrial noise. Finally, the tail of LtcmNet realizes accurate fault diagnosis of the welding robot through a multi-label classifier.

[0086] The above embodiments are merely preferred embodiments for the purpose of fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

Claims

1. An industrial robot fault diagnosis model, characterized by: include: The time series feature extraction module is used to extract the time series features of the current, speed, rotation angle and operation stage signals of each joint of the industrial robot, and to splice and fuse the memory information output in each time window. The spliced and fused time series feature graph is used as the output result of the entire time series feature extraction module; The multi-scale convolution module includes multiple feature extraction branches arranged in parallel. Each feature extraction branch uses a different receptive field convolution kernel combination to extract features from the time series feature map. The fault category feature vector obtained by combining the feature maps extracted by all feature extraction branches is used as the output of the multi-scale convolution module. A multi-label classifier includes multiple fault label modules arranged in parallel. The fault label modules are arranged in a one-to-one correspondence with the joints of the industrial robot. Each fault label module is equipped with a classifier in a one-to-one correspondence with a component in the corresponding joint that may fail. Each classifier has an independent loss function and converts the fault category feature vector into a category probability vector.

2. The industrial robot fault diagnosis model according to claim 1, characterized in that: The temporal feature extraction module adopts LSTM neural network.

3. The industrial robot fault diagnosis model according to claim 1, characterized in that: The multi-scale convolution module is provided with five feature extraction branches in parallel, namely: The MSCM_a feature extraction branch includes a 1×1 convolution kernel to preserve the feature information in the temporal feature map as much as possible; MSCM_b feature extraction branch, used to perform Max-Pooling operations to reduce the input data dimension and extract the main feature information; The MSCM_c feature extraction branch includes a 1×1 convolution kernel and a 3×3 convolution kernel connected in series; The MSCM_d feature extraction branch includes a 1×1 convolution kernel and a 5×5 convolution kernel connected in series; The MSCM_e feature extraction branch includes a 1×1 convolution kernel and a 7×7 convolution kernel connected in series.

4. The industrial robot fault diagnosis model according to claim 3, characterized in that: In the MSCM_c feature extraction branch, the 3×3 convolution kernel is split into a 1×3 convolution kernel and a 3×1 convolution kernel connected in series.

5. The industrial robot fault diagnosis model according to claim 3, characterized in that: In the MSCM_d feature extraction branch, the 5×5 convolution kernel is split into a 3×3 convolution kernel, a 1×3 convolution kernel, and a 3×1 convolution kernel; the 3×3 convolution kernel is connected in series with the 1×1 convolution kernel in the MSCM_d feature extraction branch, and the 1×3 convolution kernel is connected in parallel with the 3×1 convolution kernel and then connected in series with the 3×3 convolution kernel.

6. The industrial robot fault diagnosis model according to claim 3, characterized in that: In the MSCM_e feature extraction branch, the 7×7 convolution kernel is split into two 3×3 convolution kernels, one 1×3 convolution kernel and one 3×1 convolution kernel; among them, the two 3×3 convolution kernels are connected in series with each other, one 3×3 convolution kernel is connected in series with the 1×1 convolution kernel in the MSCM_e feature extraction branch, and the 1×3 convolution kernel is connected in parallel with the 3×1 convolution kernel and then connected in series with another 3×3 convolution kernel.

7. The industrial robot fault diagnosis model according to claim 1, characterized in that: The multi-scale convolution module further includes a global average pooling module, which is used to merge the feature maps extracted by all feature extraction branches to obtain a fault category feature vector.

8. The industrial robot fault diagnosis model according to claim 1, characterized in that: The sigmoid function is used as the activation function in the classifier.

9. A method for diagnosing faults in an industrial robot, characterized by: The steps include: Step 1: Data collection: The raw data is collected through the industrial robot control software and the production line equipment database. The collected raw data is preprocessed and then a data set is constructed. Step 2: Constructing an industrial robot fault diagnosis model as described in any one of claims 1 to 8; Step 3: Use the dataset to train the industrial robot fault diagnosis model and evaluate the effectiveness of the industrial robot fault diagnosis model; Step 4: Use the industrial robot control software to collect the current, speed, rotation angle and operation stage signals of each joint of the industrial robot in real time to achieve rapid diagnosis and positioning of industrial robot faults.

Images