A robot residual compensation method based on parameter reconfiguration neural network

Abstract

The present invention discloses a robot residual compensation method based on parameter reorganization neural network, comprising the following steps: obtaining the original position data of the measured object; obtaining the initial position in the world coordinate system through the calibrated system kinematic model; reorganizing the parameters related to the system kinematic model and the original data, inputting the reorganized parameters into DNN to obtain the residual error; performing error compensation based on the residual error and the initial position, obtaining the corrected position, and completing the residual compensation. The present invention improves positioning accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of robot residual compensation, and in particular relates to a robot residual compensation method based on parameter reorganization neural network. Background Art

[0002] The combination of industrial robots and 3D vision sensors has been widely used in high-precision positioning fields, such as assembly, machine tool processing, welding, sorting, etc. However, there are some factors that introduce nonlinear position errors, such as geometric-kinematic calibration errors, joint clearances, non-uniform encoder feedback, and mechanical deformation. Therefore, robot kinematic calibration and error compensation are important ways to improve global absolute positioning accuracy.

[0003] In recent decades, many studies have focused on methods to improve the positioning accuracy of robot vision systems, which can be divided into model-based methods and non-model methods. Combined with binocular stereo vision, Luo X. proposed an error model based on joint clearance to predict and compensate for the positioning accuracy of arbitrary positions. This model-based method requires an accurate kinematic model of the robot vision system, however, the error introduced by modeling is inevitable. In addition, non-model methods directly predict the position accuracy based on nonlinear regression. Wang Z. proposed a calibration method using artificial neural network (ANN). First, the joint angle workspace is divided into several local regions according to the mass distribution of the robot. Then, the system geometric parameters of each region are identified using the Denavit-Hartenberg (DH) model. The model-based EKF robot geometric error identification combined with the neural network compensation method can effectively solve the correction problem of all robot error sources. Zhang Q proposed a global compensation method based on GA-DNN nonlinear regression and flexible error to achieve high-precision positioning of the robot under any external load.

[0004] Similarly, in the study by Aoyagi, a neural network was used to reduce the residual error while obtaining training data through the nominal inverse kinematics of the robot

[24] . These methods simply ignore the analytical model and thus will increase the computational workload, waste more time, and reduce accuracy. To this end, Nguyen et al. proposed a new artificial neural network (ANN) method to compensate for these unmodeled errors by using partial robot drive joint coordinates, which helps to reduce nonlinear errors in different robot configurations.

[0005] In this paper, the parameters are reorganized based on the error propagation model, and a new DNN structure is proposed to improve the positioning accuracy. The system uses a 3D sensor integrated in the robot hand to scan the target. The data set is established by calibrating the object posture, establishing a strong mapping correlation between input and output, and a new deep neural network structure is proposed for training and cyclic verification. Summary of the invention

[0006] The purpose of the present invention is to propose a robot residual compensation method based on parameter reorganization neural network to improve positioning accuracy.

[0007] To achieve the above object, the present invention provides a robot residual compensation method based on parameter reorganization neural network, comprising the following steps:

[0008] Obtain the original position data of the object being measured;

[0009] Obtain the initial position in the world coordinate system through the calibrated system kinematic model;

[0010] Reorganizing the parameters related to the system kinematic model and the original data, and inputting the reorganized parameters into the DNN to obtain residual errors;

[0011] Error compensation is performed based on the residual error and the initial position to obtain a corrected position and complete residual compensation.

[0012] Optionally, obtaining the original position data of the object to be measured specifically includes: driving the scanner to the measuring point by a robot, and then collecting the position in the scanner coordinate system as the original data of the object to be measured.

[0013] Optionally, the system kinematic model is calibrated according to a classical model-based approach and several standard spheres.

[0014] Optionally, obtaining the initial position in the world coordinate system specifically includes: collecting calibration spherical coordinates in the local and world coordinate systems, calibrating the calibrated system kinematic model, and obtaining the initial position in the world coordinate system.

[0015] Optionally, reorganizing the parameters related to the system kinematic model and the original position data specifically includes:

[0016] According to the rigid transformation of each joint, the parameters in the robot base coordinate system are obtained;

[0017] Based on the parameters, a reorganization model is constructed using the DH method to obtain a nonlinear function summarizing all variables affecting the k-th joint error;

[0018] A plurality of variables are obtained according to the nonlinear function.

[0019] Optionally, the kth joint error includes: a conversion from the joint to the robot base, kinematic parameters of the joint itself, and a conversion from the joint to the laser scanner.

[0020] Optionally, the reorganized parameters are input into the DNN to obtain the residual error. The DNN includes: an input layer, a fusion layer, and a data regression layer;

[0021] The reorganized parameters are input into the input layer, which has several independent multi-layer perceptrons, each of which corresponds to several joints of the robot:

[0022] According to the output results of the plurality of independent multi-layer perceptrons of the input layer, the output results are fused into a one-dimensional vector by means of data concatenation;

[0023] The one-dimensional vector of the fusion layer is passed through a plurality of independent multi-layer perceptrons in a regression manner to predict the residual of the robot at the current position.

[0024] Optionally, error compensation is performed based on the residual error and the initial position to obtain the corrected position, specifically including: calculating the theoretical coordinates of the robot position in the world coordinate system based on the system kinematic model, inputting the kinematic model reconstruction parameters of the position into the trained DNN to obtain the residual error, and obtaining the corrected position by adding the theoretical coordinates and the residual error.

[0025] Technical effect of the invention: The invention discloses a robot residual compensation method based on parameter reorganization neural network, and proposes a new DNN structure to improve positioning accuracy. The system uses a 3D sensor integrated in the robot hand to scan the target; establishes a data set by calibrating the object posture, establishes a strong mapping correlation between input and output, and proposes a new deep neural network structure for training and cyclic verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0027] Figure 1 This is a schematic diagram of a robot vision online detection system according to an embodiment of the present invention;

[0028] Figure 2 This is a flow chart of a method for compensating residual errors of a robot system based on a parameter reorganization neural network according to an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of a deep neural network architecture for robot residual regression according to an embodiment of the present invention;

[0030] Figure 4 Schematic diagram of system construction and initial calibration of an embodiment of the present invention, wherein (a) is a schematic diagram of the swing-type laser three-dimensional sensor, and (b) is a physical diagram of the experimental system;

[0031] Figure 5 This is a schematic diagram of model parameters of the initial robot exterior according to an embodiment of the present invention;

[0032] Figure 6 Schematic diagram of automobile sheet metal parts detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] like Figure 1-6 As shown, this embodiment provides a robot residual compensation method based on parameter reorganization neural network, including:

[0036] Obtain the original position data of the object being measured;

[0037] Obtain the initial position in the world coordinate system through the calibrated system kinematic model;

[0038] Reorganizing the parameters related to the system kinematic model and the original data, and inputting the reorganized parameters into the DNN to obtain residual errors;

[0039] Error compensation is performed based on the residual error and the initial position to obtain a corrected position and complete residual compensation.

[0040] Further optimizing the scheme to obtain the original position data of the object to be measured specifically includes: driving the scanner to the measuring point by the robot, and then collecting the position in the scanner coordinate system as the original data of the object to be measured.

[0041] To further optimize the scheme, the system kinematic model is calibrated according to a classical model-based approach and several standard balls.

[0042] Further optimizing the scheme, obtaining the initial position in the world coordinate system specifically includes: collecting the coordinates of the calibration sphere in the local and world coordinate systems, calibrating the calibrated system kinematic model, and obtaining the initial position in the world coordinate system.

[0043] Further optimizing the scheme, reorganizing the parameters related to the system kinematic model and the original position data specifically includes:

[0044] According to the rigid transformation of each joint, the parameters in the robot base coordinate system are obtained;

[0045] Based on the parameters, a reorganization model is constructed using the DH method to obtain a nonlinear function summarizing all variables affecting the k-th joint error;

[0046] A plurality of variables are obtained according to the nonlinear function.

[0047] Further optimizing the scheme, the kth joint error includes: the conversion from the joint to the robot base, the kinematic parameters of the joint itself, and the conversion from the joint to the laser scanner.

[0048] To further optimize the scheme, the reorganized parameters are input into the DNN to obtain the residual error. The DNN includes: input layer, fusion layer and data regression layer;

[0049] The reorganized parameters are input into the input layer, which has several independent multi-layer perceptrons, each of which corresponds to several joints of the robot:

[0050] According to the output results of the plurality of independent multi-layer perceptrons of the input layer, the output results are fused into a one-dimensional vector by means of data concatenation;

[0051] The one-dimensional vector of the fusion layer is passed through a plurality of independent multi-layer perceptrons in a regression manner to predict the residual of the robot at the current position.

[0052] A further optimization scheme performs error compensation based on the residual error and the initial position to obtain the corrected position, specifically including: calculating the theoretical coordinates of the robot position in the world coordinate system based on the system kinematic model, inputting the kinematic model reconstruction parameters of the position into the trained DNN to obtain the residual error, and obtaining the corrected position by adding the theoretical coordinates to the residual error.

[0053] Nonlinear residual compensation system and process

[0054] An oscillating laser 3D scanner is integrated into the hand of the industrial robot. Figure 1As shown. A calibrator consisting of four standard balls is used for model calibration and nonlinear error regression. By scanning the sphere from different poses, a set of data sets can be collected, including the positions in the scanner coordinates and world coordinates. Then, the kinematic model is calibrated according to the classic model-based method. Then, a new DNN structure is proposed to fit the nonlinear residual. The training and prediction processes of DNN are almost the same. The flowchart of nonlinear residual error compensation based on DNN is shown in Figure 2 shown.

[0055] First, the robot drives the scanner to the measurement point and then collects the position in the scanner coordinate system as raw data. Next, the initial position in the world coordinate system is calculated using the calibrated kinematic model. At the same time, the parameters related to the model and the raw data are transformed and reorganized as the input of the DNN. Finally, the residual is calculated by the DNN to compensate for the position error.

[0056] First, during the DNN training phase, the robot drives the scanner around four standard balls, and manually randomly selects 50 different angles around each ball for scanning, obtaining a total of 200 measurement data, which correspond one to one with the standard coordinate values ​​of the four balls in the world coordinate system. Next, the robot is kinematically modeled and the model is calibrated based on the above measurement data; during the calibration, the residual error of each measurement data can be obtained as the output of the DNN. Based on the calibrated kinematic model and measurement data, the error transfer parameters in the model are calculated and reorganized as the input of the DNN. Then, the DNN training is completed.

[0057] Secondly, the trained DNN is used to compensate for the robot position error. The robot drives the scanner to the working position, scans the object to be measured, and calculates the theoretical initial position in the world coordinate system based on the calibrated kinematic model. At the same time, the error transfer parameters in the model are calculated based on the measurement data and reorganized as the input of the DNN. Finally, the residual error is calculated by the DNN and compensated to the theoretical initial position in the world coordinate system to achieve the position correction of the object to be measured.

[0058] Parameter Reorganization Based on Error Propagation Model

[0059] Using basic NN / DNN models to perform nonlinear regression on robot position errors is a common solution. However, existing methods simplify the input and output of the regression model, which makes it difficult for the model to learn the correlation between the residual and the scan position.

[0060] In order to make the DNN model learn the strong correlation between the scan points and the residuals, the input parameters are reorganized based on the error transfer model. Assume that the kinematic model with parameter vector is given by:

[0061]

[0062] Where C is the scan point from the 3D laser scanner.

[0063] By the rigid transformation F of each joint i (q i ), and obtain P in the robot base coordinate system. In fact, the position error can be described as follows:

[0064] P+δ P =F(q+δ q )·C (2)

[0065] By subtracting equation (2) from equation (1), the result is as follows:

[0066]

[0067] By expanding the above formula and ignoring higher-order terms, the following position error form can be obtained:

[0068]

[0069] Each robot joint is modeled using the DH method, which means that the four kinematic parameters (including the joint angle θ k , connecting rod offset d k , connecting rod distance a k and the torsion angle α k ) can describe the static state of the manipulator. Therefore, the error F of the rigid body transformation k (δq k ) can be rewritten as a nonlinear function g. In addition, and They are the forward and reverse transformation matrices of the kth joint, respectively, which can be abbreviated in the form of translation distance and Euler angle.

[0070] δF k (q k )=g(θ k , d k , a k , α k )

[0071]

[0072]

[0073] Then, all variables affecting the k-th joint error are summarized as a nonlinear function h k , as shown in Equation (6). This equation means that the error of the kth joint is packaged into three parts: the transformation from the joint to the robot base, the kinematic parameters of the joint itself, and the transformation from the joint to the laser scanner.

[0074]

[0075] Among them, h k is a nonlinear function with 16 variables:

[0076]

[0077] In summary, the contribution of the kth joint to the error is given by h k It is obtained by nonlinear combination of various variables in. Therefore, h k These 16 variables in are used as the input vectors of the neural network regression model, and the error vector δ P is defined as output.

[0078] Residual Regression Neural Network with Independent Joint Parameter Restructuring

[0079] The influence of each robot joint on the error can be summarized into 16 variables. And different robot joints are independent of each other. Therefore, the standard DNN structure is not applicable, and a new architecture is proposed.

[0080] The DNN architecture proposed for residual regression is as follows Figure 3 As shown, the entire structure can be divided into three layers. The parameter variables representing each joint in the current position state of the robot in equation (7) are converted into hidden features by inputting 6 MLPs. Next, these hidden features are concatenated into a long vector in the second layer. Finally, MLP is used to implement the regression of the residual.

[0081] We introduce multi-layer perceptron (MLP) as the basic element. LeakyRelu is used as the activation function, and each neuron function can be described as:

[0082] y=σ(w T x+b) (8)

[0083] Among them, (w, b) is the weight coefficient and bias coefficient of the neuron function, and σ(·) is the activation function.

[0084] The first layer is the input layer. In order to separate the influence of each robot joint, the input layer is designed as 6 independent multi-layer perceptrons (MLPs), corresponding to the six joints of the robot:

[0085]

[0086] Among them, J k is the output of the kth joint, set to a vector of length 64. k are the 16 variables of the kth joint in equation (7), which are used as the input of the kth MLP in the input layer. The activation function is LeakyRelu(·).

[0087] The second layer is the fusion layer. According to the output results of the 6 independent MLPs in the first layer, k , fused into a one-dimensional vector y with a length of 384 by data concatenation cnt :

[0088] y cnt =concat(J1,J2,J3,J4,J5,J6) (9)

[0089] Among them, concat(·) is the vector concatenation function.

[0090] The third layer is the data regression layer. The fusion vector y of the second layer is cnt Predict the residual δ of the robot at the current position through MLP regression P =(δ x ,δ y ,δ z ).

[0091] Usually, there are few options for loss functions to solve regression problems. We use MSE loss as the optimization objective function:

[0092]

[0093] According to our system design, the robot vision measurement system consists of three parts: an industrial robot (FANUC LRMate 200iD), a laser scanner, and a calibration object consisting of four standard ceramic balls.

[0094] First, we designed a oscillating laser 3D sensor based on a single-mirror galvanometer. The schematic diagram is shown in Figure 4 (a). The line laser hits the mirror and the object surface in sequence and is finally captured by a fixed camera. By rotating the mirror, multiple laser stripes can be displayed in the image. Then, the point cloud and shape center can be further calculated by reconstruction and regression algorithms, and the root mean square error (RMSE) of the measured sphere is 0.0107mm. Then, the laser scanner is integrated into the end of the robot to build a measurement system, as shown in Figure 4 In addition, four ceramic balls distributed around the robot are used to calibrate the model and compensate for system residuals.

[0095] The experimental data comes from scanning four standard spheres, and 50 different postures are collected for each sphere. Then, the initial kinematic model is calibrated according to the DH model, and the parameters are shown in Tables 1 and 2. At the same time, the transformation from the 3D scanner to the robot hand is defined as The transformation from the robot base to the world coordinate system is defined as As shown in Table 3 in the form of Euler angles.

[0096] Table 1

[0097]

[0098] Table 2

[0099]

[0100] Table 3

[0101]

[0102] Neural network training

[0103] 200 data were obtained from scanning each ball at 50 different viewing angles. According to formula (5), 96 parameter groups (16 parameters for each of the 6 joints) corresponding to each measurement point data were calculated as the data set of the neural network. 160 data were randomly selected for training and the remaining 40 data were used for testing. Training on RTX2060 Figure 3 The proposed deep neural network results are as follows Figure 5 As shown. After training, the parameter model of the deep neural network for robot residual regression is obtained.

[0104] Residual Error Compensation of Robotic Inspection System

[0105] The application logic sequence in the actual scenario is as follows Figure 2 As shown in the figure, take the automobile sheet metal parts detection scenario as an example. Figure 6 As shown. The robot carries the swinging three-dimensional sensor and moves to the set detection position. By reading the current robot's six joint angle data and combining the initial calibration data in Table 1, Table 2, and Table 3, the 16 physical parameters corresponding to each joint at the current position can be calculated according to formula (5), and a total of 96 parameters for the six joints. These six groups of 16 parameter sets are input into the trained residual regression deep neural network, and the residual δ of the swinging three-dimensional sensor position at the current position can be obtained. res According to formula (10), the precise position P of the current sensor can be calculated:

[0106]

[0107] Among them, Δ i is the rotation angle of the i-th joint relative to the zero position, and DH(·) is the transformation matrix based on the DH model of the robot joint.

[0108] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A robot residual compensation method based on parameter reorganization neural network, characterized in that: The following steps are involved: Obtain the original position data of the object being measured; Obtain the initial position in the world coordinate system through the calibrated system kinematic model; Reorganizing the parameters related to the system kinematic model and the original position data, and inputting the reorganized parameters into the DNN to obtain residual errors; The parameter reorganization of the parameters related to the system kinematic model and the original position data specifically includes: According to the rigid transformation of each joint, the parameters in the robot base coordinate system are obtained; Based on the parameters, a reorganization model is constructed using the DH method to obtain a nonlinear function summarizing all variables affecting the k-th joint error; Obtaining a number of variables according to the nonlinear function; The kth joint error includes: the transformation from the joint to the robot base, the kinematic parameters of the joint itself, and the transformation from the joint to the laser scanner; The reorganized parameters are input into the DNN to obtain the residual error. The DNN includes: input layer, fusion layer and data regression layer; The reorganized parameters are input into the input layer, which has several independent multi-layer perceptrons, each of which corresponds to several joints of the robot: According to the output results of the plurality of independent multi-layer perceptrons of the input layer, the output results are fused into a one-dimensional vector by means of data concatenation; The one-dimensional vector of the fusion layer is passed through a plurality of independent multi-layer perceptrons using a regression method to predict the residual of the robot at the current position; error compensation is performed based on the residual error and the initial position, the corrected position is obtained, and residual compensation is completed.

2. The robot residual compensation method based on parameter reorganization neural network according to claim 1, characterized in that: Acquiring the original position data of the object to be measured specifically includes: driving the scanner to the measuring point by the robot, and then collecting the position in the scanner coordinate system as the original data of the object to be measured.

3. The robot residual compensation method based on parameter reorganization neural network according to claim 1, characterized in that: The system kinematic model is calibrated according to a classical model-based approach and several calibration spheres.

4. The robot residual compensation method based on parameter reorganization neural network according to claim 3, characterized in that: Acquiring the initial position in the world coordinate system specifically includes: collecting calibration spherical coordinates in the local and world coordinate systems, calibrating the calibrated system kinematic model, and acquiring the initial position in the world coordinate system.

5. The robot residual compensation method based on parameter reorganization neural network according to claim 1, characterized in that: Based on the residual error and the initial position, error compensation is performed to obtain the corrected position, specifically including: calculating the theoretical coordinates of the robot position in the world coordinate system based on the system kinematic model, inputting the kinematic model reconstruction parameters of the position into the trained DNN to obtain the residual error, and obtaining the corrected position by adding the theoretical coordinates to the residual error.

Images