Robot visual servo control method based on exponential finite time neural network

Abstract

A robot visual servo control method based on an exponential finite time neural network solves the visual servo control problem of a robot and comprises the steps that firstly, a visual servo kinematical equation is written based on the geometric principle of visual servo, then the target point position or the target track of the visual servo is set, and the joint expectation angle of the robot is given; secondly, describing a robot visual servo control problem as a quadratic programming problem, and constructing a novel exponentiation finite time neural network solver for solving the quadratic programming visual servo control problem based on a KKT condition met by a quadratic programming optimal solution; the quadratic programming type visual servo control problem is solved under the conditions of no noise interference and no noise interference in sequence, and it is verified that the method has the advantages of being higher in convergence speed and better in visual servo control robustness.

Description

Technical Field

[0001] The present invention studies an exponential finite-time neural network method for the robot visual servo control problem. Specifically, a new piecewise exponential activation function is designed, and an exponential finite-time neural network is constructed based on the function to solve the robot's visual servo control problem, achieving ideal control accuracy. Background Art

[0002] Introducing visual servo control in robot motion planning can increase the dexterity of robot motion. Compared with image visual servo, position visual servo tracking control is more sensitive to calibration errors, and it directly obtains the position and pose information of the target point from image features. The motion planning strategy that only considers the robot kinematic equation has no safety constraints for the robot end effector relative to the specific task, and is not convenient for application in scenes with motion space constraints. However, robot visual servo control can make up for this shortcoming.

[0003] Since the time-varying quadratic programming strategy can equivalently describe the motion planning problems under different optimization indicators (DE Whitney. Resolved motion rate control of manipulators and human prostheses, IEEE Trans. Man-Machine Syst., 1969, 10 (2): 47-53, i.e., motion rate control of manipulators and artificial prostheses), it has been widely used in the motion planning and trajectory tracking of various robots in recent years (Y. Zhang, J. Wang, Y. Xia. A dual neural network for redundancy resolution of kinematically redundant manipulators subject to joint limits and joint velocity limits. IEEE Trans Neural Netw., 2003, 14 (3): 658-667, i.e., a dual neural network for redundancy resolution of kinematically redundant manipulators subject to joint limits and joint velocity limits).

[0004] Usually, the quadratic programming type motion planning strategy considering only the robot kinematic equations is described as,

[0005]

[0006] Where P = η θ (θ(t)-θ d (0)), η θ >0,ηr > 0, r(t) represents the Cartesian position vector, θ(t) is the joint vector, is the Jacobian matrix, and the robot's forward kinematics position and velocity equations are r = f(θ) and

[0007] Due to the defects of autonomous endoscopes such as difficulty in maneuverability and poor anti-interference performance, visual servo control technology has attracted widespread attention from researchers in fields such as robotic endoscopes (W.Li, PWYChiu, Z.Li. An accelerated finite-time convergent neural network for visual servoing of a flexible surgical endoscope with physical and RCM constraints, IEEE Trans.NeuralNetw.Learn.Syst., 2020, 31(12): 5272-5284). With the help of geometric analysis, the kinematic equations of visual servoing are described as,

[0008]

[0009] in, ω l ∈R 3 ,ω v ∈R 3 Respectively represent the linear velocity and angular velocity of the camera frame, J img ∈R 2 ×n represents the image Jacobian matrix;

[0010] Research has shown that the motion planning strategy based on quadratic programming is extremely effective in dealing with robot trajectory tracking control problems (ZJ Zhang, ZX Li, S. Yang. A barrier varying-parameter dynamic learning network for solving time-varying quadratic programming problems with multiple constraints, IEEE Trans. Neural Netw. Learn. Syst., 2022, 52 (9): 8781-8792, i.e., a barrier varying-parameter dynamic learning network for solving time-varying quadratic programming problems with multiple constraints). Combining the visual servo kinematic equations described above with other quadratic programming motion planning strategies can form a new quadratic programming visual servo motion control scheme. The quadratic programming motion planning scheme itself only contains the constraints and optimal indicators that need to be satisfied by the target task. Its optimal solution is usually based on the KKT condition and requires various zeroing neural network methods to solve. Therefore, it is necessary to study and explore more effective zeroing neural network methods.

[0011] The power-type neural network is the most typical finite-time neural network, which is divided into single-power type and double-power type. The convergence speed of the single-power type neural network is poor, so researchers have developed a better double-power type neural network, which not only retains the terms with power less than 1 of the single-power type neural network, but also adds terms with power greater than 1, and has fast finite-time convergence under any initial error conditions. However, most of the above-mentioned power-type neural networks do not consider the noise suppression performance of the network. The new power-type finite-time neural network proposed in this paper has a faster convergence speed and can maintain a certain degree of noise resistance, which is more suitable for solving various time-varying neural calculations and robot trajectory tracking problems.

[0012] Compared with the power-type neural network, the new exponential finite-time neural network can maintain bounded excitation and finite-time convergence when the initial error is small; it also has a faster convergence speed and higher calculation accuracy when the initial error is large. In addition, in the presence of noise, the new exponential neural network shows stronger robustness and still has fast convergence. Summary of the invention

[0013] In order to overcome the shortcomings of the existing technology, the present invention provides a robot visual servo control method based on a new exponential finite-time neural network. The present invention describes the robot visual servo control problem as a quadratic programming problem by means of the visual servo geometric principle and the quadratic programming type repeated motion planning strategy, and further constructs a new exponential finite-time neural network solver according to the KKT condition satisfied by the quadratic programming optimal solution, and solves the quadratic programming type visual servo control problem in the absence and presence of noise interference, respectively, to achieve higher visual servo control accuracy.

[0014] The technical solution adopted by the present invention to solve its technical problem is:

[0015] A robot visual servo control method based on an exponential finite-time neural network, the method comprising the following steps:

[0016] (1) Define the position point in the camera frame as p = [x, y, z] T , and its corresponding position point in the image frame is represented as p I =[p u ,p v ] T , assuming that the pixel coordinates of the object in the image are s = [u, v] T , According to the geometric principle of camera visual servoing, the expression of the visual servoing kinematic equation is obtained as follows:

[0017]

[0018] in, ω l ∈R 3 ,ω v ∈R 3 Respectively represent the linear velocity and angular velocity of the camera frame, J img ∈R 2×n Represents the image Jacobian matrix, which is in the following form:

[0019]

[0020] Where λ is the focal length of the camera,

[0021] (2) Define r(t)∈R m is the pose vector of the robot end effector, θ∈R n is the joint angle vector, and the forward kinematics equation of the robot is described as,

[0022] r = f(θ) (2);

[0023] The kinematic equation of the velocity layer is:

[0024]

[0025] The pose velocity vector Substituting into equation (1), the visual servo kinematic equation can be rewritten as:

[0026]

[0027] (3) Assume that the desired joint vector of the robot is θ d (0), the initial joint vector is θ(0) = θ0, and the initial joint vector θ0 is not at the expected joint vector position, that is, θ0≠θ d (0);

[0028] (4) Combining the descriptions of (1)-(4), the robot visual servo control problem is described as the following quadratic programming:

[0029]

[0030] in, M=I; Then the quadratic programming strategy (5) ensures that the position error converges asymptotically to 0;

[0031] (5) Construct a power-type finite-time neural network, whose dynamic subsystem is expressed as:

[0032]

[0033] in,

[0034]

[0035]

[0036] Moreover, a,q,b>1,δ>0, the larger a,q,b are, the better the convergence performance of the power-type finite-time neural network is;

[0037] The exponential finite-time neural network represented by formula (6) has a finite-time convergence characteristic, and its convergence time analysis is as follows:

[0038] Define the Lyapunov function as,

[0039]

[0040] Since the activation function is divided into two stages, the following proof is divided into two steps:

[0041] 1) When |e ij (0)|<δ, the derivative of the Lyapunov function can be calculated as follows:

[0042]

[0043] because make Calculation can be obtained Solve the differential equation and let v ij (t) = 0, the expression of convergence time is:

[0044]

[0045] 2) When |e ij (0)|≥δ, the convergence time needs to be calculated in the following two steps;

[0046] a) when e ij (t) from e ij (0) converges to |e ij (t1)|=δ, the derivative of the Lyapunov function is calculated as,

[0047]

[0048] The convergence time is calculated as,

[0049]

[0050] b) When e ij (t) from |e ij (t1)|=δconverges to |e ij (t2)|=0, refer to 1) to get the convergence time of this stage:

[0051]

[0052] The total convergence time is t s (v ij (0)) = t s1 +t s2 ,Right now,

[0053]

[0054] Define the Lagrangian function,

[0055]

[0056] in, is the Lagrange multiplier vector; Respectively and Taking the partial derivative and setting it to zero gives,

[0057] KY = Z (13);

[0058] in, J=J img J m(θ), I is the identity matrix;

[0059] To solve the quadratic programming visual servo control problem in step (4), the error function is defined by equation (13):

[0060]

[0061] The solution variable Y(t) is calculated by the above neural network, and its integration can obtain the angles θ of each joint of the robot.

[0062] Furthermore, the method further comprises the following steps:

[0063] In order to analyze the robustness of the proposed exponential finite-time neural network to noise interference, the following exponential neural network model subject to noise interference is given:

[0064]

[0065] Where n(t) represents noise interference;

[0066] When there is bounded noise and the noise satisfies the condition hour, Indicates | ij |The upper bound of the power-type neural network subjected to noise asymptotically converges to 0, which is proved as follows:

[0067] Define the Lyapunov function

[0068] a) when |e ij (0)|<δ, the derivative of the Lyapunov function is calculated as,

[0069]

[0070] b) When |e ij (0)|≥δ, the derivative of the Lyapunov function is calculated as,

[0071]

[0072] Observe a) and b) It is easy to find that when the upper bound of the absolute value of the noise Less than hour, It always holds true, which means that the power-type neural network converges asymptotically to 0.

[0073] The technical concept of the present invention is: construct the following quadratic programming visual servo control scheme (5) for the robot, and further construct the following exponential finite time neural network (6) for the quadratic programming scheme in formula (5), a, q, b>1, δ>0, the larger a, q, b is, the better the convergence performance of the exponential finite time neural network is; the exponential neural network described in formula (6) has finite time convergence and has strong noise suppression. When the error function E(t)={e ij When} converges to 0, the visual servo trajectory tracking control error will also converge to 0, indicating that the tracking control of the target trajectory is achieved.

[0074] The beneficial effects of the present invention are mainly manifested in: fast convergence speed, stronger noise resistance, when the initial error is small, the new exponential finite-time neural network can simultaneously maintain bounded excitation and finite-time convergence; when the initial error is large, it can still guarantee faster convergence speed and higher calculation accuracy. In addition, when there is noise interference, the new exponential neural network has stronger noise resistance and can still converge quickly. The new exponential neural network adopts a two-stage form, cleverly applies a bounded excitation function, and also makes improvements for the case of large initial errors. It is more suitable for practical applications and has obvious effectiveness and superiority. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 The flowchart of the robot visual servo control method based on the novel neural network studied in the present invention is given, wherein PRAF represents a single power activation function, DPRAF represents a double power activation function, and MPERAF represents a power activation function φ studied in the present invention. MF (·).

[0076] Figure 2 Plots of the new exponential activation function and the power-order activation function are presented.

[0077] Figure 3 It is the characteristic error curve of the bi-power neural network for solving the visual servo tracking control problem.

[0078] Figure 4 The graphs of all process variables of the new neural network solving the visual servo tracking control problem are given.

[0079] Figure 5 The actual running trajectory and characteristic variable curve of the bi-power neural network for solving the visual servo tracking control problem in the presence of noise are given.

[0080] Figure 6 The actual running trajectory and characteristic variable curve of the new neural network for solving the visual servo tracking control problem in the presence of noise are given. DETAILED DESCRIPTION

[0081] The present invention will be further described below in conjunction with the accompanying drawings.

[0082] Reference Figure 1 to Figure 6 The present invention conducts computer simulation of visual servo control for UR5 robot, builds simulink simulation platform on MATLAB software, and tests the proposed exponential finite time neural network method. In addition, for comparative analysis, the present invention also gives the simulation results of solving visual servo control with a dual exponential neural network in the implementation and analysis verification section.

[0083] A robot visual servo control method based on exponential finite-time neural network. Figure 1 This is a flowchart of the robot visual servo control solution based on the power-type neural network, which consists of the following five steps: 1. According to the geometric principle of visual servo, write the visual servo kinematic equation; 2. Combine the visual servo kinematic equation written in the previous step with the robot speed level kinematic equation to obtain the complete visual servo kinematic equation; 3. Set the target point position or target trajectory of the visual servo, and give the expected joint angle of the robot; 4. Convert the robot visual servo control problem into a quadratic programming solution; 5. Construct a new power-type finite-time neural network solver for the quadratic programming solution completed in the previous step, and calculate the real-time solution of each joint;

[0084] The robot visual servo control method based on the exponential finite-time neural network of this embodiment comprises the following steps:

[0085] 1) The initial parameters of the robot are set as follows:

[0086] The initial joint settings of the UR5 robot are The desired joint is set to

[0087] 2) Write the visual servo kinematic equation as follows:

[0088]

[0089] 3) The robot’s target trajectory is set as follows:

[0090] Given a visual servo image object, the target pixel coordinates are s d =[u d ,v d ] T =[-5,5] T ;

[0091] 4) Construct the following quadratic programming visual servo control scheme:

[0092]

[0093] in, M=I;

[0094] 5) Construct the following exponential finite-time neural network:

[0095]

[0096] in,

[0097]

[0098] Among them, a, q, b>1, δ>0;

[0099] Define the following Lagrangian function,

[0100]

[0101] in, is the Lagrange multiplier vector; Respectively and Find the partial derivative and set it to zero, and we get,

[0102] KY = Z (13);

[0103] in, J=J img J m (θ), I is the identity matrix;

[0104] The error function is defined as E(t)=KY-Z by equation (13). Substituting the error function into equation (6) yields the following exponential finite-time neural network model:

[0105]

[0106] In order to facilitate the analysis and reading of the following text, the bi-power neural network model for reference and comparison in the invention is given as follows:

[0107]

[0108] The simulation parameters of this embodiment are set as follows: the parameters of the bi-power neural network are set to κ1=κ2=1, The parameters of the new exponential finite-time neural network are set as

[0109] ρ=1,b=10,a=20,q=2,δ=2, and the noise interference term is n(t)=5cos(3t).

[0110] Figure 2A comparison of the curves of the new exponential activation function and the power activation function is given. It should be noted that the larger the exponential activation function is, the better the convergence performance of the power-type finite-time neural network is, and it performs better than the power-type neural network. In addition, the parameters can be freely adjusted according to actual problems.

[0111] Figure 3 The characteristic error curve of bi-power neural network for solving visual servo tracking control problem is given.

[0112] Figure 4 (a) shows the actual running trajectory of the new neural network to solve the visual servo tracking control problem, as shown in the dotted curve in the figure. The solid line in the figure represents the position of the visual servo target point, and it can be found that the actual trajectory quickly tracks the position of the target point. Figure 4 (b) The characteristic variable curve of the new neural network for solving the visual servo tracking control problem is given, as shown in the dotted curve in the figure. The solid line in the figure represents the coordinates of the target point. Figure 4 (c) The characteristic error curve of the new neural network for solving the visual servo tracking control problem is given. It can be observed that the error curve converges to 0 quickly. Figure 4 (d) and Figure 4 (e) The curves of joint angle and joint angular velocity of the new neural network for solving the visual servo tracking control problem are given respectively. The joint velocity eventually returns to 0, and the joint approximately reaches the desired joint position.

[0113] observe Figure 3 It can be found that in the absence of noise, the error curve of the bi-power neural network for solving the visual servo tracking control problem converges to 0 in about 1.75 seconds. Figure 4 In the paper, the error curve of the new neural network solving the visual servo tracking control problem converges to 0 in about 0.75 seconds.

[0114] contrast Figure 5 and Figure 6 The results show that the bi-power neural network has obvious tracking errors when solving the visual servo tracking control problem in the presence of noise, and its trajectory to the target point is more tortuous and unstable. Figure 6 It can be seen that the actual running trajectory of the new exponential neural network for solving the visual servo tracking control problem in the presence of noise can still track the position of the target point quickly and stably. It can also be seen from the characteristic variable curve that the characteristic variable still reaches the target point position quickly, that is, the trajectory tracking error converges quickly to 0.

[0115] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A robot visual servo control method based on exponential finite-time neural network, characterized in that: The method comprises the following steps: (1) Define the position point in the camera frame as p = [x, y, z] T , and its corresponding position point in the image frame is represented as p I =[p u ,p v ] T , assuming that the pixel coordinates of the object in the image are s = [u, v] T , According to the geometric principle of camera visual servoing, the expression of the visual servoing kinematic equation is obtained as follows: in, ω l ∈R 3 ,ω v ∈R 3 Respectively represent the linear velocity and angular velocity of the camera frame, J img ∈R 2×n Represents the image Jacobian matrix, which is in the following form: Where λ is the focal length of the camera, (2) Define r(t)∈R m is the pose vector of the robot end effector, θ∈R n is the joint angle vector, and the forward kinematics equation of the robot is described as, r = f(θ) (2); The kinematic equation of the velocity layer is: The pose velocity vector Substituting into equation (1), the visual servo kinematic equation can be rewritten as: (3) Assume that the desired joint vector of the robot is θ d (0), the initial joint vector is θ(0) = θ0, and the initial joint vector θ0 is not at the expected joint vector position, that is, θ0≠θ d (0); (4) Combining the descriptions of (1)-(4), the robot visual servo control problem is described as the following quadratic programming: in, M=I; Then the quadratic programming strategy (5) ensures that the position error converges asymptotically to 0; (5) Construct a power-type finite-time neural network, whose dynamic subsystem is expressed as: in, Moreover, a,q,b>1,δ>0, the larger a,q,b are, the better the convergence performance of the power-type finite-time neural network is; The exponential finite-time neural network represented by formula (6) has a finite-time convergence characteristic, and its convergence time analysis is as follows: Define the Lyapunov function as, Since the activation function is divided into two stages, the following proof is divided into two steps: 1) When |e ij (0)|<δ, the derivative of the Lyapunov function can be calculated as follows: because make Calculation can be obtained Solve the differential equation and let v ij (t) = 0, the expression of convergence time is: 2) When |e ij (0)|≥δ, the convergence time needs to be calculated in the following two steps; a) when e ij (t) from e ij (0) converges to |e ij (t1)|=δ, the derivative of the Lyapunov function is calculated as, The convergence time is calculated as, b) When e ij (t) from |e ij (t1)|=δconverges to |e ij (t2)|=0, refer to 1) to get the convergence time of this stage: The total convergence time is t s (v ij (0)) = t s1 +t s2 ,Right now, Define the Lagrangian function, Among them, ζ is the Lagrange multiplier vector; Respectively And taking the partial derivative and setting it to zero, we get, KY = Z (13); in, J=J img J m (θ), I is the identity matrix; To solve the quadratic programming visual servo control problem in step (4), the error function is defined by equation (13): The solution variable Y(t) is calculated by the above neural network, and its integration can obtain the angles θ of each joint of the robot.

2. The robot visual servo control method based on exponential finite-time neural network as claimed in claim 1, characterized in that: The method further comprises the following steps: In order to analyze the robustness of the proposed exponential finite-time neural network to noise interference, the following exponential neural network model subject to noise interference is given: Where n(t) represents noise interference; When there is bounded noise and the noise satisfies the condition hour, Indicates | ij |The upper bound of the power-type neural network subjected to noise converges asymptotically to 0, which is proved as follows: Define the Lyapunov function a) when |e ij (0)|<δ, the derivative of the Lyapunov function is calculated as, b) When |e ij (0)|≥δ, the derivative of the Lyapunov function is calculated as, Observe a) and b) It is easy to find that when the upper bound of the absolute value of the noise Less than hour, It always holds true, which means that the power-type neural network converges asymptotically to 0.

Images