Unsupervised industrial robot anomaly detection method based on IMU sensor error calibration

Abstract

The present invention discloses an unsupervised industrial robot anomaly detection method based on IMU sensor error calibration, comprising the following steps: calibrating the posture signal in the collected raw data; then calculating a three-dimensional free acceleration signal based on the calibrated posture signal and the gravity acceleration signal in the raw data; dividing the three-dimensional free acceleration signal into a training set and a test set; training a convolutional autoencoder model based on the training set; using each signal in the training set and the test set as the input of the trained convolutional autoencoder model, and then reconstructing the model to obtain the corresponding output signal; subtracting the input and output to obtain a three-dimensional reconstruction error signal; obtaining an error sphere radius based on the three-dimensional reconstruction error signal; obtaining an anomaly score based on the error sphere radius; determining an anomaly boundary value as an anomaly detection threshold based on the anomaly score of the training set signal; and comparing the anomaly score of the test set signal with the anomaly detection threshold to implement detection. The present invention can effectively detect anomalies in industrial robots.

Description

Technical Field

[0001] The present invention relates to an industrial robot anomaly detection method based on a convolutional autoencoder model and a posture sensor, and belongs to the field of anomaly detection and deep learning. Background Art

[0002] As factories become more intelligent, expectations for industrial robots are rising to include the ability to operate stably and for extended periods in unsupervised environments, achieving long-term autonomy. The core of long-term autonomy lies in the robot's ability to detect anomalies. Anomalies not only impact the robot's service life but also product quality. Therefore, enhancing industrial robots' anomaly detection capabilities is crucial for extending their service life and ensuring product quality.

[0003] Existing technologies use signals from external or internal sensors for anomaly detection. However, vibration acceleration signals are insensitive to low frequencies and are insensitive to low-frequency anomalies; sound signals are susceptible to environmental interference; and internal controller signals have a low sampling rate and are insensitive to anomalies. Furthermore, existing technologies cannot achieve long-term autonomy when detecting signals with high feature dimensions, predominantly noise interference, and periodic components. Summary of the Invention

[0004] The present invention provides an unsupervised industrial robot anomaly detection method based on IMU sensor error calibration, which realizes robot anomaly detection by collecting terminal signals through posture sensors and reconstructing the signals using convolutional autoencoders.

[0005] The technical solution of the present invention is:

[0006] According to a first aspect of the present invention, there is provided an unsupervised industrial robot anomaly detection method based on IMU sensor error calibration, comprising the following steps:

[0007] S1. Install an inertial measurement unit at the end of the industrial robot;

[0008] S2. Collecting raw data of the robot in normal and abnormal states using an inertial measurement unit; calibrating the posture signal in the collected raw data; and then calculating a three-dimensional free acceleration signal based on the calibrated posture signal and the gravity acceleration signal in the raw data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal;

[0009] S3, using a portion of normal signals in the 3D free acceleration signal as a training set; and using another portion of normal signals and all abnormal signals in the 3D free acceleration signal as a test set;

[0010] S4. Training the convolutional autoencoder model based on the training set to determine the hyperparameters of the convolutional autoencoder model and obtain a trained convolutional autoencoder model; using each signal in the training set and the test set as the input of the trained convolutional autoencoder model, and then reconstructing the model to obtain the corresponding output signal; subtracting the input and output to obtain a three-dimensional reconstruction error signal of the training set and test set signals;

[0011] S5. Obtain an error sphere radius based on the 3D reconstruction error signal; obtain an anomaly score for each signal in the training set and the test set based on the error sphere radius of each signal in the training set and the test set; determine an anomaly boundary value based on the anomaly score of the training set signal, and use the anomaly boundary value as an anomaly detection threshold;

[0012] S6. Compare the test set signal anomaly score with the anomaly detection threshold, and detect the status of each signal in the test set; the status is classified as normal or abnormal.

[0013] The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration is used to detect industrial robots based on real-time data, specifically: the industrial robot obtains real-time data based on the inertial measurement unit installed at the end of the robot; then the posture signal in the real-time data is calibrated; the three-dimensional free acceleration signal is calculated based on the calibrated posture signal and the gravity acceleration signal in the real-time data; the three-dimensional free acceleration signal is input into a trained convolutional autoencoder model to obtain an output signal; the input and output are subtracted to obtain a three-dimensional reconstruction error signal; based on the three-dimensional reconstruction error signal, the error sphere radius is obtained, and the error sphere radius is used as the anomaly score of the current real-time data; the anomaly score of the current real-time data is compared with the anomaly detection threshold: if the anomaly score of the real-time data is greater than the anomaly detection threshold, it indicates that the robot state corresponding to the current real-time data is abnormal; otherwise, it is normal.

[0014] Furthermore, the Kalman filtering algorithm is used to calibrate the posture signal in the collected raw data.

[0015] Furthermore, the anomaly score of each signal in the training set and the test set is obtained based on the error sphere radius of each signal in the training set and the test set. Specifically, the maximum value of the error sphere radius of the reconstructed error signal of each signal in the training set and the test set is used as the anomaly score of each signal.

[0016] Furthermore, the abnormal boundary value is determined based on the abnormal score of the training set signal, specifically: the maximum value of the abnormal score of the signal in the training set is used as the abnormality detection threshold.

[0017] According to a second aspect of the present invention, there is provided an unsupervised industrial robot anomaly detection system based on IMU sensor error calibration, comprising a module of any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration.

[0018] According to a third aspect of the present invention, a processor is provided, which is used to run a program, wherein when the program is run, any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration is executed.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration.

[0020] The beneficial effects of the present invention are: taking into account the problem of insensitivity to anomalies by controlling internal signals and vibration acceleration signals; at the same time, the existing technology cannot meet the requirements of long-term autonomy when detecting signals with high feature dimensions, noise interference and periodic components; the present invention uses a posture sensor to collect the free acceleration signal of the robot end and further calibrates it. On the basis of calibration, normal data is used to train the convolutional autoencoder model, and then the error sphere radius is obtained as the signal anomaly score based on the reconstruction error of the signal, and then the threshold is determined by the anomaly score of the training set data, and then the test set signal is detected for anomalies. According to the anomaly detection model, the robot anomaly can be efficiently detected, and the anomaly detection required for the long-term autonomy of the robot can be realized. After verification by examples, the present invention can effectively detect anomalies of industrial robots, has wide practicality, and the detection results are reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of the present invention;

[0022] Figure 2 The results before and after the free acceleration calibration of the present invention;

[0023] Figure 3 The convolutional autoencoder model structure of the present invention;

[0024] Figure 4 This is the time domain waveform of the free acceleration signal obtained by collecting raw data and calibrating the posture sensor of the Qianjiang QJR6-1 robot in a normal motion state in the actual application case of the present invention;

[0025] Figure 5This is the time domain waveform of the free acceleration signal obtained by collecting raw data and calibrating the posture sensor of the Qianjiang QJR6-1 robot in an abnormal motion state in the actual application case of the present invention;

[0026] Figure 6 This is an example of the error sphere radius distribution of a random abnormal signal in the present invention;

[0027] Figure 7 This is the result of abnormality detection by the method of the present invention. DETAILED DESCRIPTION

[0028] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other in any way.

[0029] Example 1: Figure 1-7 As shown, according to a first aspect of an embodiment of the present invention, a method for detecting anomalies in an industrial robot based on a convolutional autoencoder model and a posture sensor is provided, comprising the following steps:

[0030] S1. Install an inertial measurement unit (IMU) at the end of the industrial robot.

[0031] S2. Collecting raw data of the robot in normal and abnormal states using an inertial measurement unit; calibrating the attitude signal in the collected raw data using a Kalman filter algorithm, i.e., removing drift from the attitude signal; and then calculating a three-dimensional free acceleration signal based on the calibrated attitude signal and the gravity acceleration signal in the raw data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal;

[0032] S3, using a portion of normal signals in the 3D free acceleration signal as a training set; and using another portion of normal signals and all abnormal signals in the 3D free acceleration signal as a test set;

[0033] S4. Training the convolutional autoencoder model based on the training set to determine the hyperparameters of the convolutional autoencoder model and obtain a trained convolutional autoencoder model; using each signal in the training set and the test set as the input of the trained convolutional autoencoder model, and then reconstructing the model to obtain the corresponding output signal; subtracting the input and output to obtain a three-dimensional reconstruction error signal of the training set and test set signals;

[0034] S5. Obtain an error sphere radius based on the 3D reconstruction error signal; obtain an anomaly score for each signal in the training set and the test set based on the error sphere radius of each signal in the training set and the test set; determine an anomaly boundary value based on the anomaly score of the training set signal, and use the anomaly boundary value as an anomaly detection threshold;

[0035] S6. Compare the test set signal anomaly score with the anomaly detection threshold, and detect the status of each signal in the test set; the status is classified as normal or abnormal.

[0036] Furthermore, in addition to the test set, the present invention can also perform detection based on data obtained in real time. The specific process is: the industry obtains real-time data based on the inertial measurement unit installed at the end of the robot; then the posture signal in the real-time data is calibrated using the Kalman filtering algorithm; the three-dimensional free acceleration signal is calculated based on the calibrated posture signal and the gravity acceleration signal in the real-time data; the three-dimensional free acceleration signal is input into the trained convolutional autoencoder model to obtain the output signal; the input and the output are subtracted to obtain a three-dimensional reconstruction error signal; based on the three-dimensional reconstruction error signal, the error sphere radius is obtained, and the error sphere radius is used as the anomaly score of the current real-time data; the anomaly score of the current real-time data is compared with the anomaly detection threshold: if the anomaly score of the real-time data is greater than the anomaly detection threshold, it indicates that the robot state corresponding to the current real-time data is abnormal; otherwise, it is normal.

[0037] Furthermore, the present invention provides the following implementation process:

[0038] The robot data comes from the Qianjiang QJR6-1 robot, which has a total of six joints. The sampling rate of all data is 400 Hz, and the number of sampling points per sample signal is 5120.

[0039] S1. Due to the serial or parallel structure of industrial robots, the free acceleration at the end position is sensitive to abnormal behavior. In addition, IMUs provide an economical and effective means of data acquisition. Therefore, IMUs are installed at the end of industrial robots to collect raw data in both normal and abnormal states.

[0040] Specifically, robot tasks are typically highly repetitive. A robot in normal motion simulates a welding task, following a rectangular trajectory. After each task, the robot pauses for one second before continuing to the next. The IMU collects raw data from the end-point, representing normal signals. After collecting normal signals, obstacles are placed during the robot's motion to cause collisions with the robot's arm. The IMU collects raw data from the end-point during this motion, representing abnormal signals. The experiment collected 880 sets of normal signals and 220 sets of abnormal signals.

[0041] Step 2: There is a drift phenomenon in the attitude signal in the raw data collected by the IMU, and this drift further leads to differences in the free acceleration signals between different cycles. The attitude signal collected by the inertial measurement unit (IMU) is calibrated using the Kalman filter algorithm, that is, the attitude signal is de-drifted. Then, the three-dimensional free acceleration signal is calculated based on the calibrated attitude and the gravity acceleration signal in the raw data, and the three-dimensional free acceleration signal is used for anomaly detection. For example, 1000 seconds (83 normal signal samples) are selected from the entire mission cycle, and the free acceleration of the 83 samples collected before and after calibration is as follows: Figure 2 shown.

[0042] Step 3: 80% of the normal signals are used as the training set; the remaining 20% ​​of the normal signals and all the abnormal signals are used as the test set. Figure 4 As shown; an abnormal signal is Figure 5 shown.

[0043] Robot anomalies are often random and uncertain, resulting in a scarcity of abnormal data. Therefore, only normal signals are used in the training set, allowing the convolutional autoencoder model to fully learn the data distribution of normal samples.

[0044] The convolutional autoencoder model of this embodiment can be represented as two parts: encoder f: R N →R D and decoder g:R D →R N The convolutional autoencoder structure is as follows: Figure 3 As shown; R N 、R D Represents the dimension of the sample, N>D, in the embodiment of the present invention, N is set to 5120 (the same as the number of data points of a single sample), and D is set to 1280. The encoder maps the sample to the feature space as: z∈R D The decoder reconstructs the sample through the sample space features For an N-dimensional sample x∈R 1×N , the encoding and decoding process is expressed as:

[0045] z=f θ (x) = s(W x +b) (1)

[0046]

[0047] Among them, θ and θ′ represent model hyperparameters, θ={W, b}, θ′={W′, b′}, W′=WT, W and W′ are weight matrices, b and b′ are bias vectors; the i-th sample x i are mapped to the corresponding z i and Optimize the model's hyperparameters to minimize the reconstruction error:

[0048]

[0049] in, is the mean square error loss function; θ * ,θ′ * They represent the final model hyperparameters; n represents the number of samples; after the iteration is stable, the model hyperparameters are determined as: θ * ,θ′ * .

[0050] Step 4: Train the convolutional autoencoder model based on the training set to determine the hyperparameters of the convolutional autoencoder model and obtain the trained convolutional autoencoder model; use each signal in the training set and the test set as the input of the trained convolutional autoencoder model, and then reconstruct the corresponding output signal through the model; subtract the input and output to obtain the three-dimensional reconstruction error signal of the training set and test set signals, which is expressed as follows:

[0051]

[0052] Among them, εi is the reconstructed error signal of the i-th sample, xi is the original input signal of the i-th sample, is the i-th sample output signal.

[0053] Step 5: Based on the 3D reconstruction error signal, obtain the error sphere radius; obtain the anomaly score of each signal in the training set and test set based on the error sphere radius of each signal in the training set and test set; determine the anomaly boundary value based on the anomaly score of the training set signal, and use the anomaly boundary value as the anomaly detection threshold;

[0054] Specifically: the maximum value of the error sphere radius of the reconstructed error signal of each signal in the training set and the test set is taken as the abnormal score of each signal (for a signal sample, there are multiple data points, and an error sphere radius can be obtained for each data point, such as Figure 6 The figure shows the distribution of the error sphere radius of an abnormal signal. Figure 6It can be seen that the error sphere radius shows that the abnormal points in the signal are distributed on the surface of the sphere, while the non-abnormal points are concentrated at the center of the sphere. Therefore, choosing the maximum value of the error sphere radius of all data points under the signal as the abnormal score of the signal sample is in line with the distribution law of industrial robot signals); the maximum value of the abnormal score of each signal in the training set is used as the abnormality detection threshold; the abnormal scores of each signal in the test set are compared according to the abnormality detection threshold: if the abnormal score of the corresponding signal in the test set is less than or equal to the abnormality detection threshold, the corresponding signal is considered to be a normal signal; otherwise, it is an abnormal signal. For example, the abnormal score values ​​corresponding to the normal signal and abnormal signal in the test set are represented by dots and circles respectively, and the results are plotted in a polar coordinate diagram, as shown in the following figure. Figure 7 As shown, from Figure 7 It can be seen that the abnormal signals and normal signals in the test set are distinguished by the abnormal detection threshold, and the abnormal recognition rate reaches 100%.

[0055] The expressions of the error sphere radius and anomaly score are as follows:

[0056]

[0057] in, is the radius of the jth sphere of the ith sample, and are the average values ​​of the X, Y and Z axis reconstruction errors of the i-th sample, and is the jth reconstruction error value of the i-th sample in the X, Y and Z axes, S i is the anomaly score of the i-th sample.

[0058] In summary, the method proposed in this invention can be successfully applied to anomaly detection of industrial robots, which proves the feasibility and effectiveness of the method.

[0059] According to the second aspect of an embodiment of the present invention, there is provided an unsupervised industrial robot anomaly detection system based on IMU sensor error calibration, including a module of any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration. Specifically comprising: a first module for executing S1: installing an inertial measurement unit at the end of the industrial robot; a second module for executing S2: collecting raw data of the robot in normal and abnormal states based on the inertial measurement unit; calibrating the posture signal in the collected raw data; and then calculating the three-dimensional free acceleration signal based on the calibrated posture signal and the gravity acceleration signal in the raw data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal; a third module for executing S3: using a part of the normal signal in the three-dimensional free acceleration signal as a training set; using another part of the normal signal and all the abnormal signals in the three-dimensional free acceleration signal as a test set; and a fourth module for executing S4: training the convolutional autoencoder model based on the training set to determine the convolutional autoencoder model The hyperparameters are used to obtain a trained convolutional autoencoder model; each signal in the training set and the test set is used as the input of the trained convolutional autoencoder model, and then the corresponding output signal is obtained through model reconstruction; the input and output are subtracted to obtain the three-dimensional reconstruction error signal of the training set and the test set signal; the fifth module is used to execute S5: based on the three-dimensional reconstruction error signal, the error sphere radius is obtained; according to the error sphere radius of each signal in the training set and the test set, the anomaly score of each signal in the training set and the test set is obtained; according to the anomaly score of the training set signal, the anomaly boundary value is determined, and the anomaly boundary value is used as the anomaly detection threshold; the sixth module is used to execute S6: the anomaly score of the test set signal is compared with the anomaly detection threshold, and the state of each signal in the test set is detected; the state is divided into normal or abnormal. For the parts not described in detail in the above for each module, please refer to the relevant description of the embodiment.

[0060] According to a third aspect of an embodiment of the present invention, a processor is provided, which is used to run a program, wherein when the program is run, any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration is executed.

[0061] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned unsupervised industrial robot anomaly detection methods based on IMU sensor error calibration.

[0062] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0063] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. An unsupervised industrial robot anomaly detection method based on IMU sensor error calibration, characterized in that: The following steps are involved: S1. Install an inertial measurement unit at the end of the industrial robot; S2. Collecting raw data of the robot in normal and abnormal states using an inertial measurement unit; calibrating the posture signal in the collected raw data; and then calculating a three-dimensional free acceleration signal based on the calibrated posture signal and the gravity acceleration signal in the raw data; wherein the three-dimensional free acceleration signal includes a normal signal and an abnormal signal; S3, using a portion of normal signals in the 3D free acceleration signal as a training set; and using another portion of normal signals and all abnormal signals in the 3D free acceleration signal as a test set; S4. Training the convolutional autoencoder model based on the training set to determine the hyperparameters of the convolutional autoencoder model and obtain a trained convolutional autoencoder model; using each signal in the training set and the test set as the input of the trained convolutional autoencoder model, and then reconstructing the model to obtain the corresponding output signal; subtracting the input and output to obtain a three-dimensional reconstruction error signal of the training set and test set signals; S5. Obtain an error sphere radius based on the 3D reconstruction error signal; obtain an anomaly score for each signal in the training set and the test set based on the error sphere radius of each signal in the training set and the test set; determine an anomaly boundary value based on the anomaly score of the training set signal, and use the anomaly boundary value as an anomaly detection threshold; S6. Compare the test set signal anomaly score with the anomaly detection threshold and detect the status of each signal in the test set; the status is classified as normal or abnormal; The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration is used to detect industrial robots based on real-time data, specifically: real-time data is obtained based on the inertial measurement unit installed at the end of the industrial robot; then the posture signal in the real-time data is calibrated; a three-dimensional free acceleration signal is calculated based on the calibrated posture signal and the gravity acceleration signal in the real-time data; the three-dimensional free acceleration signal is input into a trained convolutional autoencoder model to obtain an output signal; the input and output are subtracted to obtain a three-dimensional reconstruction error signal; based on the three-dimensional reconstruction error signal, an error sphere radius is obtained, and the error sphere radius is used as the anomaly score of the current real-time data; the anomaly score of the current real-time data is compared with the anomaly detection threshold: if the anomaly score of the real-time data is greater than the anomaly detection threshold, it indicates that the robot state corresponding to the current real-time data is abnormal; otherwise, it is normal.

2. The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to claim 1 is characterized in that: The Kalman filtering algorithm is used to calibrate the posture signal in the collected raw data.

3. The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to claim 1 is characterized in that: The anomaly score of each signal in the training set and the test set is obtained based on the error sphere radius of each signal in the training set and the test set. Specifically, the maximum value of the error sphere radius of the reconstructed error signal of each signal in the training set and the test set is used as the anomaly score of each signal.

4. The unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to claim 1, characterized in that: The abnormal boundary value is determined based on the abnormal score of the training set signal, specifically: the maximum value of the abnormal score of the signal in the training set is used as the abnormality detection threshold.

5. An unsupervised industrial robot anomaly detection system based on IMU sensor error calibration, characterized in that: A module comprising the unsupervised industrial robot anomaly detection method based on IMU sensor error calibration as described in any one of claims 1 to 4.

6. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the unsupervised industrial robot anomaly detection method based on IMU sensor error calibration according to any one of claims 1 to 4.

Images