Unsupervised industrial robot anomaly detection method based on IMU sensor error calibration

Abstract

The invention discloses an unsupervised industrial robot anomaly detection method based on IMU sensor error calibration. The unsupervised industrial robot anomaly detection method comprises the steps that attitude signals in collected original data are calibrated; calculating a three-dimensional free acceleration signal according to the calibrated attitude signal and a gravitational acceleration signal in the original data; dividing the three-dimensional free acceleration signal into a training set and a test set; training the convolutional auto-encoder model based on the training set; each signal in the training set and the test set is used as the input of a trained convolutional auto-encoder model, and then a corresponding output signal is obtained through model reconstruction; subtracting the input from the output to obtain a three-dimensional reconstruction error signal; obtaining an error sphere radius based on the three-dimensional reconstruction error signal; obtaining an abnormal score according to the error sphere radius; according to the abnormal score of the training set signal, determining an abnormal boundary value as an abnormal detection threshold value; and comparing the signal abnormal score of the test set with an abnormal detection threshold value to realize detection. According to the invention, the abnormity of the industrial robot can be effectively detected.

Description

Technical Field

[0001] The present invention relates to an industrial robot anomaly detection method based on a convolutional autoencoder model and a posture sensor, and belongs to the field of anomaly detection and deep learning. Background Art

[0002] As the level of factory intelligence increases, expectations for industrial robots have risen to the point where they can operate stably for a long time in an unsupervised environment and achieve the goal of long-term autonomy. The core of long-term autonomy is that the robot has the ability to detect anomalies. The occurrence of anomalies will not only affect the service life of the robot, but also the quality of the product. Therefore, enhancing the anomaly detection capability of industrial robots is of vital importance to extending their service life and ensuring product quality.

[0003] In existing technologies, abnormality detection signals use external sensor signals or internal sensor signals. However, among these signals, vibration acceleration signals have low sensitivity to low frequencies and are insensitive to low-frequency anomalies; sound signals are easily disturbed by the environment; and the sampling rate of controller internal signals is low and insensitive to anomalies. In addition, existing technologies cannot meet the requirements of long-term autonomy when detecting signals with high feature dimensions, noise interference, and periodic components. Summary of the invention

[0004] The present invention provides an unsupervised industrial robot anomaly detection method based on IMU sensor error calibration, which realizes robot anomaly detection by collecting terminal signals through posture sensors and reconstructing the signals using convolutional autoencoders.

[0005] The technical solution of the present invention is:

[0006] According to a first aspect of the present invention, there is provided an unsupervised industrial robot anomaly detection method based on IMU sensor error calibration, comprising the following steps:

[0007] S1. Install an inertial measurement unit at the end of the industrial robot;

[0008] S2. Collecting raw data of the robot in normal and abnormal states according to the inertial measurement unit; calibrating the posture signal in the collected raw data; and then calculating the three-dimensional free acceleration signal according to the calibrated posture signal and the gravity acceleration signal in the raw data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal;

[0009] S3, using a part of the normal signals in the 3D free acceleration signal as a training set; and using another part of the normal signals and all the abnormal signals in the 3D free acceleration signal as a test set;

[0010] S4. Based on the training set, the convolutional autoencoder model is trained to determine the hyperparameters of the convolutional autoencoder model, and a trained convolutional autoencoder model is obtained; each signal in the training set and the test set is used as the input of the trained convolutional autoencoder model, and then the corresponding output signal is obtained by model reconstruction; the input and the output are subtracted to obtain a three-dimensional reconstruction error signal of the training set and the test set signal;

[0011] S5. Based on the three-dimensional reconstruction error signal, obtain the error sphere radius; according to the error sphere radius of each signal in the training set and the test set, obtain the anomaly score of each signal in the training set and the test set; according to the anomaly score of the training set signal, determine the anomaly boundary value, and use the anomaly boundary value as the anomaly detection threshold;

[0012] S6. Compare the test set signal anomaly score with the anomaly detection threshold, and detect the state of each signal in the test set; the state is classified as normal or abnormal.

[0013] The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration is used to detect the industrial robot based on the data obtained in real time, specifically: the industrial robot obtains real-time data based on the inertial measurement unit installed at the end of the robot; then the posture signal in the real-time data is calibrated; the three-dimensional free acceleration signal is calculated according to the calibrated posture signal and the gravity acceleration signal in the real-time data; the three-dimensional free acceleration signal is input into the trained convolutional autoencoder model to obtain the output signal; the input is subtracted from the output to obtain a three-dimensional reconstruction error signal; based on the three-dimensional reconstruction error signal, the error sphere radius is obtained, and the error sphere radius is used as the anomaly score of the current real-time data; the anomaly score of the current real-time data is compared with the anomaly detection threshold: if the anomaly score of the real-time data is greater than the anomaly detection threshold, it indicates that the robot state corresponding to the current real-time data is abnormal; otherwise it is normal.

[0014] Furthermore, the Kalman filtering algorithm is used to calibrate the attitude signal in the collected raw data.

[0015] Furthermore, the anomaly score of each signal in the training set and the test set is obtained based on the error sphere radius of each signal in the training set and the test set, specifically: the maximum value of the error sphere radius of the reconstructed error signal of each signal in the training set and the test set is used as the anomaly score of each signal.

[0016] Furthermore, the abnormal boundary value is determined based on the abnormal score of the signal in the training set, specifically: the maximum value of the abnormal score of the signal in the training set is used as the abnormality detection threshold.

[0017] According to a second aspect of the present invention, there is provided an unsupervised industrial robot anomaly detection system based on IMU sensor error calibration, comprising a module of any one of the unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration described above.

[0018] According to a third aspect of the present invention, a processor is provided, wherein the processor is used to run a program, wherein when the program is run, any one of the unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration described above is executed.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration.

[0020] The beneficial effects of the present invention are as follows: taking into account the problem of insensitivity to anomalies by controlling internal signals and vibration acceleration signals; at the same time, the existing technology cannot meet the requirements of long-term autonomy when detecting signals with high feature dimensions, noise interference and dominant periodic components; the present invention uses a posture sensor to collect the free acceleration signal of the robot end and further calibrates it, and uses normal data to train the convolution autoencoder model on the basis of calibration, and then obtains the error sphere radius as the signal anomaly score based on the reconstruction error of the signal, and then determines the threshold through the anomaly score of the training set data, and then performs anomaly detection on the test set signal, and according to the anomaly detection model, the robot abnormal situation can be efficiently detected, and the anomaly detection required for the long-term autonomy of the robot can be realized. After verification by examples, the present invention can effectively detect the anomalies of industrial robots, has wide practicability, and the detection results are reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of the present invention;

[0022] Figure 2 The results before and after the free acceleration calibration of the present invention;

[0023] Figure 3 The convolutional autoencoder model structure of the present invention;

[0024] Figure 4 It is the free acceleration signal time domain waveform obtained by collecting raw data and calibrating the posture sensor of the Qianjiang QJR6-1 robot in the normal motion state in the actual application case of the present invention;

[0025] Figure 5It is the free acceleration signal time domain waveform obtained by collecting raw data and calibrating the posture sensor of the Qianjiang QJR6-1 robot in an abnormal motion state in the actual application case of the present invention;

[0026] Figure 6 This is an example of the error sphere radius distribution of a random abnormal signal of the present invention;

[0027] Figure 7 It is the result of abnormality detection by the method of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other arbitrarily without conflict.

[0029] Example 1: Figure 1-7 As shown, according to a first aspect of an embodiment of the present invention, a method for detecting anomalies of an industrial robot based on a convolutional autoencoder model and a posture sensor is provided, comprising the following steps:

[0030] S1. Install an inertial measurement unit (IMU) at the end of the industrial robot.

[0031] S2. Collecting the original data of the robot in normal and abnormal states according to the inertial measurement unit; calibrating the attitude signal in the collected original data by using the Kalman filter algorithm, that is, removing the drift of the attitude signal; then calculating the three-dimensional free acceleration signal according to the calibrated attitude signal and the gravity acceleration signal in the original data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal;

[0032] S3, using a part of the normal signals in the 3D free acceleration signal as a training set; and using another part of the normal signals and all the abnormal signals in the 3D free acceleration signal as a test set;

[0033] S4. Based on the training set, the convolutional autoencoder model is trained to determine the hyperparameters of the convolutional autoencoder model, and a trained convolutional autoencoder model is obtained; each signal in the training set and the test set is used as the input of the trained convolutional autoencoder model, and then the corresponding output signal is obtained by model reconstruction; the input and the output are subtracted to obtain a three-dimensional reconstruction error signal of the training set and the test set signal;

[0034] S5. Based on the three-dimensional reconstruction error signal, obtain the error sphere radius; according to the error sphere radius of each signal in the training set and the test set, obtain the anomaly score of each signal in the training set and the test set; according to the anomaly score of the training set signal, determine the anomaly boundary value, and use the anomaly boundary value as the anomaly detection threshold;

[0035] S6. Compare the test set signal anomaly score with the anomaly detection threshold, and detect the state of each signal in the test set; the state is classified as normal or abnormal.

[0036] Furthermore, in addition to the test set, the present invention can also perform detection based on data obtained in real time, and the specific process is: the industry obtains real-time data based on the inertial measurement unit installed at the end of the robot; then the posture signal in the real-time data is calibrated using the Kalman filtering algorithm; the three-dimensional free acceleration signal is calculated based on the calibrated posture signal and the gravity acceleration signal in the real-time data; the three-dimensional free acceleration signal is input into the trained convolutional autoencoder model to obtain the output signal; the input is subtracted from the output to obtain a three-dimensional reconstruction error signal; based on the three-dimensional reconstruction error signal, the error sphere radius is obtained, and the error sphere radius is used as the anomaly score of the current real-time data; the anomaly score of the current real-time data is compared with the anomaly detection threshold: if the anomaly score of the real-time data is greater than the anomaly detection threshold, it indicates that the robot state corresponding to the current real-time data is abnormal; otherwise it is normal.

[0037] Furthermore, the present invention provides the following implementation process:

[0038] The robot data comes from the Qianjiang QJR6-1 robot, which has a total of six joint-connected arms. The sampling rate of all data is 400Hz, and the number of sampling points for a sample signal is 5120.

[0039] S1. Due to the serial or parallel structural characteristics of industrial robots, the free acceleration at the end position is sensitive to abnormal behavior; and IMU provides an economical and effective means of data collection. Therefore, the IMU is installed at the end of the industrial robot to collect the original data of the robot in normal and abnormal states.

[0040] Specifically: The robot's work tasks are usually highly repetitive. Let the robot in normal motion simulate the welding task and move along a rectangular trajectory at the end; stop for 1s after each task and continue to perform the next task; collect the original data of the end through IMU, and the original data at this time is a normal signal. After the normal signal is collected, obstacles are set during the robot's movement to collide with the robot's arm, and the original data of the end during the movement is collected through IMU, and the original data at this time is an abnormal signal. Through experiments, 880 groups of normal signals were collected and 220 groups of abnormal signals were collected.

[0041] Step 2: There is a drift phenomenon in the attitude signal in the raw data collected by the IMU, and this drift further leads to differences in the free acceleration signals between different cycles. The Kalman filtering algorithm is used to calibrate the attitude signal collected by the inertial measurement unit (IMU), that is, to remove the drift of the attitude signal. Then, the three-dimensional free acceleration signal is calculated based on the calibrated attitude and the gravity acceleration signal in the raw data, and the three-dimensional free acceleration signal is used for anomaly detection. For example, 1000 seconds (83 normal signal samples) are selected from the entire mission cycle, and the free acceleration of the 83 samples collected before and after calibration is as follows: Figure 2 shown.

[0042] Step 3: 80% of the normal signals are used as the training set; the remaining 20% ​​of the normal signals and all the abnormal signals are used as the test set. Figure 4 As shown; an abnormal signal is Figure 5 shown.

[0043] Robot anomalies are usually random and uncertain, which leads to the scarcity of abnormal data. Therefore, only normal signals are used in the training set, so that the convolutional autoencoder model can fully learn the data distribution of normal samples.

[0044] The convolutional autoencoder model of this embodiment can be represented as two parts: encoder f: R N →R D and decoder g:R D →R N The convolutional autoencoder structure is as follows Figure 3 As shown; R N , R D Represents the dimension of the sample, N>D, in the embodiment of the present invention, N is set to 5120 (the same as the number of data points of a single sample), and D is set to 1280. The encoder maps the sample to the feature space as: z∈R D The decoder reconstructs the sample through the sample space features For an N-dimensional sample x∈R 1×N , the encoding and decoding process is expressed as:

[0045] z=f θ (x) = s(W x +b) (1)

[0046]

[0047] Among them, θ, θ′ represent model hyperparameters, θ={W, b}, θ′={W′, b′}, W′=WT, W, W′ are weight matrices, b, b′ are bias vectors; the i-th sample x i are mapped to the corresponding z i and Optimize the model's hyperparameters to minimize the reconstruction error:

[0048]

[0049] in, is the mean square error loss function; θ * ,θ′ * They represent the final model hyperparameters; n represents the number of samples; after the iteration is stable, the model hyperparameters are determined as: θ * ,θ′ * .

[0050] Step 4: Train the convolutional autoencoder model based on the training set to determine the hyperparameters of the convolutional autoencoder model and obtain the trained convolutional autoencoder model; use each signal in the training set and the test set as the input of the trained convolutional autoencoder model, and then reconstruct the corresponding output signal through the model; subtract the input and output to obtain the three-dimensional reconstruction error signal of the training set and the test set signal, which is expressed as follows:

[0051]

[0052] Among them, εi is the i-th sample reconstruction error signal, xi is the i-th sample original input signal, is the i-th sample output signal.

[0053] Step 5: Based on the 3D reconstruction error signal, obtain the error sphere radius; according to the error sphere radius of each signal in the training set and the test set, obtain the anomaly score of each signal in the training set and the test set; according to the anomaly score of the training set signal, determine the anomaly boundary value, and use the anomaly boundary value as the anomaly detection threshold;

[0054] Specifically: the maximum value of the error sphere radius of the reconstructed error signal of each signal in the training set and the test set is taken as the abnormal score of each signal (for a signal sample, there are multiple data points, and an error sphere radius can be obtained for each data point, such as Figure 6 The figure shows the distribution of the error sphere radius of an abnormal signal. Figure 6It can be seen that the error sphere radius shows that the abnormal points in the signal are distributed on the surface of the sphere, while the non-abnormal points are concentrated at the center of the sphere. Therefore, choosing the maximum value of the error sphere radius of all data points under the signal as the abnormal score of the signal sample is in line with the distribution law of industrial robot signals); the maximum value of the abnormal score of each signal in the training set is used as the abnormal detection threshold; the abnormal scores of each signal in the test set are compared according to the abnormal detection threshold: if the abnormal score of the corresponding signal in the test set is less than or equal to the abnormal detection threshold, the corresponding signal is considered to be a normal signal; otherwise, it is an abnormal signal. Exemplarily, the abnormal scores corresponding to the normal signal and abnormal signal in the test set are represented by dots and circles, respectively, and the results are plotted in a polar coordinate diagram. The results are shown in the figure. Figure 7 As shown, from Figure 7 It can be seen that the abnormal signals and normal signals in the test set are distinguished by the anomaly detection threshold, and the anomaly recognition rate reaches 100%.

[0055] The expressions of the error sphere radius and the anomaly score are as follows:

[0056]

[0057] in, is the radius of the jth sphere of the ith sample, and are the average values ​​of the X, Y and Z axis reconstruction errors of the i-th sample, and is the jth reconstruction error value of the i-th sample in the X, Y and Z axes, S i is the anomaly score of the ith sample.

[0058] In summary, the method proposed in the present invention can be successfully applied to anomaly detection of industrial robots, which proves the feasibility and effectiveness of the method.

[0059] According to the second aspect of an embodiment of the present invention, there is provided an unsupervised industrial robot anomaly detection system based on IMU sensor error calibration, including a module of any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration. Specifically including: a first module for executing S1: installing an inertial measurement unit at the end of the industrial robot; a second module for executing S2: collecting raw data of the robot in normal and abnormal states according to the inertial measurement unit; calibrating the posture signal in the collected raw data; and then calculating the three-dimensional free acceleration signal according to the calibrated posture signal and the gravity acceleration signal in the raw data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal; a third module for executing S3: using a part of the normal signals in the three-dimensional free acceleration signal as a training set; another part of the normal signals and all the abnormal signals in the three-dimensional free acceleration signal as a test set; a fourth module for executing S4: training the convolutional autoencoder model based on the training set to determine the convolutional autoencoder model The hyperparameters are used to obtain the trained convolutional autoencoder model; each signal in the training set and the test set is used as the input of the trained convolutional autoencoder model, and then the corresponding output signal is obtained through model reconstruction; the input and output are subtracted to obtain the three-dimensional reconstruction error signal of the training set and the test set signal; the fifth module is used to execute S5: based on the three-dimensional reconstruction error signal, the error sphere radius is obtained; according to the error sphere radius of each signal in the training set and the test set, the abnormal score of each signal in the training set and the test set is obtained; according to the abnormal score of the training set signal, the abnormal boundary value is determined, and the abnormal boundary value is used as the abnormal detection threshold; the sixth module is used to execute S6: the abnormal score of the test set signal is compared with the abnormal detection threshold, and the state of each signal in the test set is detected; the state is divided into normal or abnormal. For the parts not described in detail in the above for each module, please refer to the relevant description of the embodiment.

[0060] According to a third aspect of an embodiment of the present invention, a processor is provided, wherein the processor is used to run a program, wherein when the program is running, any one of the unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration described above is executed.

[0061] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration.

[0062] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0063] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. An unsupervised industrial robot anomaly detection method based on IMU sensor error calibration, characterized in that: The following steps are involved: S1. Install an inertial measurement unit at the end of the industrial robot; S2. Collecting raw data of the robot in normal and abnormal states according to the inertial measurement unit; calibrating the posture signal in the collected raw data; and then calculating the three-dimensional free acceleration signal according to the calibrated posture signal and the gravity acceleration signal in the raw data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal; S3, using a part of the normal signals in the 3D free acceleration signal as a training set; and using another part of the normal signals and all the abnormal signals in the 3D free acceleration signal as a test set; S4. Based on the training set, the convolutional autoencoder model is trained to determine the hyperparameters of the convolutional autoencoder model, and a trained convolutional autoencoder model is obtained; each signal in the training set and the test set is used as the input of the trained convolutional autoencoder model, and then the corresponding output signal is obtained by model reconstruction; the input and the output are subtracted to obtain a three-dimensional reconstruction error signal of the training set and the test set signal; S5. Based on the three-dimensional reconstruction error signal, obtain the error sphere radius; according to the error sphere radius of each signal in the training set and the test set, obtain the anomaly score of each signal in the training set and the test set; according to the anomaly score of the training set signal, determine the anomaly boundary value, and use the anomaly boundary value as the anomaly detection threshold; S6. Compare the test set signal anomaly score with the anomaly detection threshold, and detect the state of each signal in the test set; the state is classified as normal or abnormal.

2. The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to claim 1, characterized in that: The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration is used to detect the industrial robot based on the data obtained in real time, specifically: the industrial robot obtains real-time data based on the inertial measurement unit installed at the end of the robot; then the posture signal in the real-time data is calibrated; the three-dimensional free acceleration signal is calculated according to the calibrated posture signal and the gravity acceleration signal in the real-time data; the three-dimensional free acceleration signal is input into the trained convolutional autoencoder model to obtain the output signal; the input is subtracted from the output to obtain a three-dimensional reconstruction error signal; based on the three-dimensional reconstruction error signal, the error sphere radius is obtained, and the error sphere radius is used as the anomaly score of the current real-time data; the anomaly score of the current real-time data is compared with the anomaly detection threshold: if the anomaly score of the real-time data is greater than the anomaly detection threshold, it indicates that the robot state corresponding to the current real-time data is abnormal; otherwise it is normal.

3. The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to claim 1, characterized in that: The Kalman filtering algorithm is used to calibrate the attitude signal in the collected raw data.

4. The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to claim 1, characterized in that: The method obtains the anomaly score of each signal in the training set and the test set based on the error sphere radius of each signal in the training set and the test set, specifically: taking the maximum value of the error sphere radius of the reconstructed error signal of each signal in the training set and the test set as the anomaly score of each signal.

5. The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to claim 1, characterized in that: The abnormal boundary value is determined based on the abnormal score of the training set signal, specifically: the maximum value of the abnormal score of the signal in the training set is used as the abnormality detection threshold.

6. An unsupervised industrial robot anomaly detection system based on IMU sensor error calibration, characterized in that: A module comprising the unsupervised industrial robot anomaly detection method based on IMU sensor error calibration as described in any one of claims 1-5.

7. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the unsupervised industrial robot anomaly detection method based on IMU sensor error calibration as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the unsupervised industrial robot anomaly detection method based on IMU sensor error calibration as described in any one of claims 1 to 5.

Images