An Adaptive Fault-Tolerant Control Method for Manipulator Based on Uncalibrated Vision Model

Abstract

The present invention discloses a method for adaptive fault-tolerant control of a robotic arm based on an uncalibrated vision model. By online estimating the unknown faults suffered by the robotic arm and compensating them into the control torque, the influence of partial actuator failures is suppressed. And an online gradient descent minimization method is designed to handle the uncertainty of uncalibrated camera parameters, realizing the fixed-point tracking control of the robotic arm in the camera image plane. To improve the environmental adaptability of the robotic arm, unknown actuator faults and unknown internal and external parameter matrices caused by uncalibrated cameras are added to the robotic arm dynamics equation. According to the designed adaptive algorithm, the unknown information is estimated online, and the stable tracking control performance of the visual servo robotic arm system in case of faults is guaranteed. This method fully considers the fault problems faced in the production application process of the robotic arm and has the advantages of wide application range and good control performance.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arm control. Specifically, it is an adaptive fault-tolerant control method for robotic arms based on an uncalibrated vision model. Background Art

[0002] The realization of the autonomous ability of robots mainly depends on the environmental perception of sensors. Visual sensors can collect and analyze images, obtaining more information about targets and the environment. Moreover, their installation methods are more flexible and the working ranges are larger, so they are widely used. In a robotic system with a traditional vision model, a calibrated monocular camera is generally adopted. However, the internal and external parameters of the camera will change in different working environments. To simplify the operation process and improve the system control and tracking accuracy, the research on robotic arm control based on an uncalibrated vision model has started to develop. At the same time, the system components of robotic arms often fail due to various unexpected situations. In minor cases, it will reduce the system performance and damage the entire working system. In severe cases, it will endanger personal and property safety, leading to catastrophic accidents. Based on this, the research on fault estimation and fault-tolerant control methods has received the attention of many scholars at home and abroad. Most traditional fault diagnosis and fault-tolerant control methods are for simple systems, and for modern complex industrial systems, traditional methods have great limitations. At the same time, facing the increasing requirements for reliability and safety, the existing fault diagnosis and fault-tolerant control methods cannot fully meet the industrial needs. Therefore, researching the fault-tolerant control algorithm for robotic arms under an uncalibrated vision model has important theoretical research significance and application value. Summary of the Invention

[0003] Aiming at the problems existing in the prior art, the purpose of the present invention is to provide an adaptive fault-tolerant control method for robotic arms based on an uncalibrated vision model. This technical solution suppresses the influence of partial actuator failures by online estimating the unknown faults suffered by the robotic arm and compensating them into the control torque, and designs an online gradient descent minimization method to handle the uncertainty of uncalibrated camera parameters, realizing the fixed-point tracking control of the robotic arm in the camera image plane.

[0004] To solve the above problems, the present invention adopts the following technical solutions.

[0005] An adaptive fault-tolerant control method for robotic arms based on an uncalibrated vision model, the control input and the fault estimation adaptive law are defined as follows:

[0006]

[0007]

[0008] where g(q) represents the gravity term, K 1is the velocity gain matrix, sign() represents the sign function, ξ represents a constant positive number, and J T represents the transpose of the Jacobian matrix, represents the transpose of the estimated value of the depth-independent interaction matrix, represents the estimated value of the third row vector of the camera internal and external parameter matrix M, B represents a positive definite matrix; where, x represents the position coordinates of the feature point at the end of the robotic arm in three-dimensional space, and y d represents the desired position of the feature point in the image plane, y represents the actual position of the feature point in the image plane, Δy represents the position error of the feature point, q represents the angle of each joint, represents the angular velocity of each joint, represents the control input, represents the unknown actuator fault function, τ d represents the unknown disturbance, represents the estimated value of the camera parameters, represents the fault estimated value.

[0009] An adaptive fault-tolerant control method for a robotic arm based on an uncalibrated vision model, characterized in that:

[0010] The uncalibrated visual servo control strategy uses an adaptive method to estimate the unknown internal and external camera parameters, and the expression of the adaptation law is:

[0011]

[0012] Where, represents the estimated value of the camera parameters, Θ -1 is the inverse of the positive definite diagonal gain matrix Θ, Ω T is the transpose of the linearized parameter matrix Ω, L T represents the transpose of the camera parameter linearization matrix L, t p (p = 1, 2, 3…, 5) represents five moments on the motion trajectory of the feature point, e(t p , t) represents the position error between the moment of t p and the current moment.

[0013] An adaptive fault-tolerant control method for a robotic arm based on an uncalibrated vision model, characterized in that:

[0014] The uncalibrated visual control algorithm estimates the internal and external camera parameters by selecting the position information at five different moments on the operation trajectory of the robotic arm, thus avoiding the complex calibration process of the camera.

[0015] Advantages of the present invention

[0016] Compared with the prior art, the advantages of the present invention are:

[0017] The robotic arm may suddenly malfunction during actual applications. The fault-tolerant control algorithm proposed in the present invention can effectively suppress the negative impacts brought by actuator failures. In addition, the design of the calibration-free visual servo control method successfully avoids the complex calibration process of the camera. Description of the Drawings

[0018] Figure 1 is the block diagram of the adaptive fault-tolerant control algorithm for the robotic arm based on the calibration-free visual model;

[0019] Figure 2 is the comparison diagram of the motion trajectories of the feature points in the horizontal axis direction of the image plane;

[0020] Figure 3 is the comparison diagram of the motion trajectories of the feature points in the vertical axis direction of the image plane;

[0021] Figure 4 is the comparison diagram of the position error ME index in the image plane;

[0022] Figure 5 is the diagram of the process of selecting points in the operating trajectory of the robotic arm. Detailed Implementation Manner

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Please refer to Figures 1 to 5 , and its overall control block diagram is as shown in Figure 1 . Among them, x represents the position coordinates of the feature point at the end of the robotic arm in three-dimensional space, y d represents the desired position of the feature point in the image plane, y represents the actual position of the feature point in the image plane, Δy represents the feature point position error, q represents the angles of each joint, represents the angular velocity of each joint, represents the control input, represents the unknown actuator fault function, τ d represents the unknown disturbance, represents the estimated value of the camera parameters, represents the fault estimated value.

[0025] In the above solution, the described adaptive fault-tolerant control method for the robotic arm based on the calibration-free visual model first designs an adaptive fault-tolerant control algorithm that can estimate the fault threshold online. The control input and the fault estimation adaptive law definitions are as follows:

[0026]

[0027]

[0028] Among them, g(q) represents the gravity term, K 1 is the velocity gain matrix, sign() represents the sign function, ξ represents a constant positive number, J T represents the transpose of the Jacobian matrix, represents the transpose of the estimated value of the interaction matrix independent of depth, represents the estimated value of the third row vector of the camera internal and external parameter matrix M, and B represents a positive definite matrix.

[0029] In the above scheme, the adaptive fault-tolerant algorithm enables the end effector of the manipulator to asymptotically approach the desired position in the image plane, and the unknown actuator fault information can be estimated and compensated online, ensuring the control performance of visual servo.

[0030] In the above scheme, the calibration-free visual servo control strategy uses the adaptive method to estimate the unknown internal and external camera parameters, and the expression of this adaptive law is:

[0031]

[0032] Among them, represents the estimated value of the camera parameters, Θ -1 is the inverse of the positive definite diagonal gain matrix Θ, Ω T is the transpose of the linearized parameter matrix Ω, L T represents the transpose of the camera parameter linearization matrix L, t p (p = 1, 2, 3…, 5) represents five moments on the motion trajectory of the feature point, e(t p , t) represents the position error between the moment of t p and the current moment.

[0033] In the above scheme, the calibration-free visual servo method improves the environmental adaptability of the manipulator by estimating the unknown camera parameters.

[0034] In the above scheme, the control method adjusts the control input in real time according to the actual position information and fault disturbance information during the motion of the manipulator to achieve the purpose of stable tracking.

[0035] In this example, first, the coordinate position of the manipulator in the three-dimensional space is mapped to the image plane of the camera through the visual model, and the process is as follows:

[0036] First, the three-dimensional coordinates (X, Y, Z) are converted into the internal coordinates (x, y, z) of the camera through the pinhole projection model:

[0037]

[0038] where \(f\) represents the camera focal length, \(z\) represents the depth distance between the feature point of the robotic arm and the camera, and then the camera coordinates \((x, y, z)\) are transformed into the pixel coordinates \((u, v)\) of the camera's image plane:

[0039]

[0040] where \(dx\) and \(dy\) represent the pixel length, and \(c\) x and \(c\) y represent the pixel offset, and then the mapping of the feature point at the end of the robotic arm from the camera coordinates to the image coordinates is obtained. The intermediate transformation matrix is the internal and external parameter matrix of the camera, that is, the perspective projection matrix \(M\):

[0041]

[0042] The rank of the perspective projection matrix \(M\) is 3. When the parameters are not calibrated, the perspective projection matrix is unknown and needs to be estimated from the feature points and their projection coordinates on the image plane. Note that there are 12 unknown components in the matrix. Since two equations correspond to one feature point, 6 feature points are required to estimate the perspective projection matrix. Let the unknown parameters in \(M\) be \(\theta\), then an adaptive law can be designed to estimate the parameters in the perspective projection matrix, as shown in Equation (3). Select the positions at five different moments on the operating trajectory of the robotic arm, then the internal and external parameters of the camera can be estimated according to the selected position information. The process of selecting points is as Figure 5 shown. Then, the fixed-point tracking control of the robotic arm is realized through the control torque proposed in Equation (1). The adaptive fault-tolerant control law in this process is as shown in Equation (2).

[0043] Next, simulation experiments are used to verify the performance of the control method proposed in the present invention. The main simulation platform uses Matlab 2010b version. To verify the effectiveness of this control method, the whole experiment includes three simulation experiments: the first simulation experiment uses the fault-tolerant control algorithm designed in the present invention under unknown faults, the second simulation experiment uses the control algorithm of traditional visual servo under unknown faults, and the third experiment uses the fault-tolerant control algorithm of the present invention under normal (no fault) conditions. To compare the performance, the control method proposed in the present invention is compared with the control method of traditional visual servo. Their image plane tracking trajectories are respectively as Figure 2 , 3 shown, and their position error \(ME\) and average position error \(AME\) are shown in Table 1 and Figure 4 shown, where the solid line represents the method proposed in the present invention, the double dashed line represents the traditional visual servo control method, and the dotted line represents the control method of the present invention under normal conditions.

[0044] As can be seen from the simulation results, by using the method proposed in the present invention, the AME index is reduced by nearly 48.98%, and the controller also has better tracking performance and avoids the chattering phenomenon.

[0045] Table 1

[0046]

[0047]

[0048] The above are only the preferred specific embodiments of the present invention; however, the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and its improved concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. An adaptive fault-tolerant control method for a robotic arm based on an uncalibrated vision model, characterized in that, it includes the following steps: First, the coordinate position of the robotic arm in the three-dimensional space is mapped to the image plane of the camera through a visual model. The three-dimensional coordinates (X, Y, Z) are transformed into the internal coordinates (x, y, z) of the camera through the pinhole projection model, and then the camera coordinates (x, y, z) are converted into the pixel coordinates (u, v) of the camera's image plane, thereby obtaining the mapping of the feature point at the end of the robotic arm from the camera coordinates to the image coordinates. The intermediate transformation matrix is the internal and external parameter matrix of the camera, that is, the perspective projection matrix M. Let the unknown parameters in M be θ, design an adaptive law to estimate the parameters in the perspective projection matrix, select the positions at multiple different moments on the operating trajectory of the robotic arm, then the internal and external parameters of the camera can be estimated according to the selected position information. After that, the fixed-point tracking control of the robotic arm is realized through the control torque proposed in Equation (1), and the control input and the definitions of the fault estimation adaptive law are as follows respectively: where, g(q) represents the gravity term, K 1 is the velocity gain matrix, sign() represents the sign function, ξ represents a constant positive number, J T represents the transpose of the Jacobian matrix, represents the transpose of the estimated value of the interaction matrix independent of depth, represents the estimated value of the third row vector of the camera intrinsic and extrinsic parameter matrix M, B represents a positive definite matrix; where, x represents the position coordinates of the feature point at the end of the robotic arm in three-dimensional space, y d represents the desired position of the feature point in the image plane, y represents the actual position of the feature point in the image plane, Δy represents the feature point position error, q represents the joint angles, represents the angular velocity of each joint, represents the control input, represents the unknown actuator fault function, τ d represents the unknown disturbance, represents the estimated value of the camera parameters, represents the fault estimate value.

2. The adaptive fault-tolerant control method for a robotic arm based on an uncalibrated vision model according to claim 1, characterized in that: The uncalibrated visual servo control strategy uses an adaptive method to estimate the unknown internal and external parameters of the camera, and the expression of the adaptation law is: Among them, represents the estimated value of the camera parameters, Θ -1 is the inverse of the positive definite diagonal gain matrix Θ, Ω T is the transpose of the linearization parameter matrix Ω, L T represents the transpose of the camera parameter linearization matrix L, t p (p = 1, 2, 3..., 5) represents five moments on the motion trajectory of the feature point, e(t p , t) represents the position error between the moment of t p and the current moment.

3. The adaptive fault-tolerant control method for a robotic arm based on an uncalibrated vision model according to claim 2, characterized in that: The uncalibrated visual control algorithm estimates the internal and external parameters of the camera by selecting the position information at five different moments on the operating trajectory of the robotic arm, thus avoiding the complex calibration process of the camera.

Images