A finite-time control method for visual servoing based on velocity observer

Through the limited time control method of visual servo based on speed observer, the problem of difficult trajectory tracking stability and rapidity of the robotic arm visual servo system is solved. The adaptive law and image speed observer are designed to realize the estimation of unknown parameters and Finite time tracking control improves the system's convergence speed and robustness to image noise.

CN118322205BActive Publication Date: 2024-11-15GUANGXI STAGE ROBOT CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410555818.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-07
Publication Date
2024-11-15
Estimated Expiration
2044-05-07

AI Technical Summary

Technical Problem

The existing robotic arm vision servo control system has problems of difficulty in achieving stability and rapidity during tracking, and the adaptive solution is not robust to image noise, resulting in a long convergence time and a degradation of control performance.

Method used

Using the finite time control method of visual servo based on speed observer, the visual servo model is constructed, parameter separation, and the adaptive estimation law and image speed observer are designed to realize the estimation of unknown parameters and finite time tracking control, thereby improving the convergence of the system. Speed ​​and robustness to image noise.

Benefits of technology

An adaptive law is designed to quickly estimate unknown parameters of the visual servo system, realize finite time trajectory tracking control, improve the convergence speed of the system, and improve the robustness of the servo system to image noise.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118322205B_ABST
    Figure CN118322205B_ABST
Patent Text Reader

Abstract

The present invention discloses a visual servo finite time control method based on a velocity observer, which belongs to the field of visual servo control of a manipulator. The method comprises the following steps: constructing a visual servo model based on image information and joint information; performing parameter separation based on the visual servo model to obtain unknown parameters; constructing an image velocity observer, and designing an adaptive estimation law based on the image velocity observer; and estimating the unknown parameters based on the adaptive estimation law to realize visual servo finite time trajectory tracking control. The present invention realizes finite time control of a manipulator visual servo system by utilizing feature point image information and manipulator joint information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of visual servo control of robotic arms, and in particular relates to a visual servo finite time control method based on a speed observer. Background Art

[0002] The robot visual servo controls the servo system with the image information of the feature points and the joint information of the robot. The visual system is usually a camera and a robot system. The traditional solution is to use the image information and joint information to construct the image Jacobian matrix to design the control rate to complete the servo task. The structure of this solution is simple and the controller design is relatively concise. However, its parameters are strongly coupled, which makes it difficult to identify the parameters and design the control scheme. Dynamic visual servo usually uses various adaptive algorithms to identify the system parameters online. However, the visual system is a complex nonlinear system with multiple unknown parameters. When dealing with tracking problems, the existing control schemes do not guarantee the convergence time, which means that the system cannot converge within a limited time, and the convergence of these control schemes may be very time-consuming. By setting a larger control gain for the controller, the convergence time can be significantly shortened, but as the image noise increases, the control performance may decrease.

[0003] In addition to the above issues, the adaptive scheme control input does not require image space velocity measurements, but the parameter adaptation law uses image space velocity signals. Due to the relatively long processing time or delay of image information, such velocity signals are often very noisy, so velocity measurements should be avoided in controller design.

[0004] In summary, the realization of stability and rapidity of trajectory tracking of existing robot visual servo control systems is a challenge to the overall performance of the system. Therefore, it is of great significance to design a suitable visual servo control solution to solve the above problems. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a visual servo finite time control method based on a speed observer to solve the above problems existing in the prior art.

[0006] To achieve the above object, the present invention provides a visual servo finite time control method based on a speed observer, comprising:

[0007] Construct a visual servo model based on image information and joint information;

[0008] Perform parameter separation based on the visual servo model to obtain unknown parameters;

[0009] constructing an image velocity observer, and designing an adaptive estimation law based on the image velocity observer;

[0010] The unknown parameters are estimated based on the adaptive estimation law to realize visual servoing finite time trajectory tracking control.

[0011] Optionally, the process of constructing the visual servoing model includes constructing feature points, and the coordinate formula of the feature points in pixel coordinates is:

[0012]

[0013] Where y = (u, v) T , y is the coordinate of the feature point in the pixel plane, z represents the depth of the feature point in the camera coordinates, is the product of the internal parameter matrix and the external parameter matrix, is the i-th row of M, is the homogeneous transformation matrix from the robot base coordinate system to the camera coordinate system, and x is the three-dimensional coordinate of the feature point in the base coordinate system.

[0014] Optionally, the depth of the feature point in the camera coordinates and the coordinates of the feature point in the pixel coordinates are temporally differentiated to obtain:

[0015]

[0016]

[0017] In the formula, It represents the time differentiation of the depth of the feature point in the camera coordinate. Indicates the time differentiation of the coordinates of the feature point in pixel coordinates, R and P represent The rotation and translation components of Representative Matrix The i-th row of represents the joint velocity, Depth-independent image Jacobian matrix representing the unified kinematic frame.

[0018] Optionally, the unknown parameters include: depth parameters, kinematic parameters and dynamic parameters; and the process of obtaining the unknown parameters by performing parameter separation based on the visual servo model includes:

[0019] The known parameters are obtained by performing parameter separation based on the differential formula of the coordinates of the feature points in the pixel coordinates;

[0020] Parameter separation is performed based on the differential formula of the depth of the feature point in the camera coordinate, and the depth of the feature point in the camera coordinate is linearly parameterized to obtain the depth parameter and kinematic parameter;

[0021] A mechanical arm dynamics equation is constructed, and the mechanical arm dynamics equation is linearized based on a regression matrix to obtain dynamic parameters.

[0022] Optionally, the expression for linearizing the mechanical arm dynamics equation based on the regression matrix is:

[0023]

[0024] In the formula, is the regression matrix, n is the number of joint angles, p is the number of unknown parameters, and are the joint velocity and acceleration, is the inertia matrix, is the Coriolis matrix, is the gravity moment, and q is the joint angle.

[0025] Optionally, the image velocity observer is:

[0026]

[0027] In the formula, represents the image velocity observer, y ob is the observed value of the image space position, α is the first positive design constant, β is the second positive design constant, and They are z and The estimate of y d is the expected trajectory, Estimation of the Jacobian matrix of the compensated depth-independent image.

[0028] Optionally, the adaptive estimation law is:

[0029]

[0030]

[0031]

[0032] In the formula, Ψ d ,Ψ k and z denote the positive definite symmetric matrices with appropriate dimensions in the three parameter estimates, sign(·) is the standard sign function, represents the reference joint velocity, represents the reference joint acceleration, e represents the image position tracking error, K1 is the gain matrix to be determined, and α1 is the constant to be determined.

[0033] Compared with the prior art, the present invention has the following advantages and technical effects:

[0034] The present invention constructs a visual servo model based on image information and joint information; performs parameter separation based on the visual servo model to obtain unknown parameters; constructs an image velocity observer, and designs an adaptive estimation law based on the image velocity observer; estimates the unknown parameters based on the adaptive estimation law to realize visual servo finite-time trajectory tracking control. The present invention provides a visual servo control method based on a velocity observer and finite-time control, which realizes finite-time control of a robotic arm visual servo system by utilizing feature point image information and robotic arm joint information. Its beneficial effects include: 1) An adaptive law is designed to estimate the unknown parameters of visual servo. 2) A visual servo control scheme with finite-time control is designed to improve the convergence speed of the system. 3) In the system, an image velocity observer is designed to improve the robustness of the servo system to image noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 is a structural block diagram of a visual servoing system according to an embodiment of the present invention;

[0037] Figure 2 is a control system block diagram of an embodiment of the present invention;

[0038] Figure 3 A feature point trajectory diagram according to an embodiment of the present invention;

[0039] Figure 4 is the image position tracking error norm of the embodiment of the present invention;

[0040] Figure 5 is the image position tracking error of the embodiment of the present invention. DETAILED DESCRIPTION

[0041] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] Embodiment 1

[0044] In view of the problems existing in the existing visual servo control methods of robotic arms, the present invention provides a visual servo control method based on a velocity observer and finite time control. The present invention adopts a depth-independent matrix to establish a controlled model, which has a relatively simple parameter identification scheme. Based on this model, homogeneous system theory, quasi-differential technology and non-smooth continuous functions are used to realize finite time trajectory tracking control. In order to introduce visual errors into the design, the present invention proposes a controller with a visual position term, which is a non-smooth continuous function and can be regarded as a proportional term. In addition, in order to avoid the measurement of image velocity, the present invention proposes an image velocity observer to improve the robustness of the servo system to image noise. The main advantages of this method are its simplicity and the separation of multiple uncertainties in the visual servo robot system, thereby avoiding the problem of over-parameterization of existing work.

[0045] The present invention adopts the following technical solution: a visual servo control method based on speed observer and finite time control, including the construction of visual servo model, separation of parameters, and design of controller. The specific steps are as follows:

[0046] 1. Based on the image information and the joint information of the robot arm, a depth-independent model is constructed as the controlled object;

[0047] 2. Use homogeneous system theory, quasi-differential technology and non-smooth continuous functions to design adaptive estimation laws to achieve finite-time trajectory tracking control.

[0048] 3. In order to avoid the measurement of image velocity, an image velocity observer is designed to improve the robustness of the servo system to image noise.

[0049] like Figure 1 As shown, in order to better illustrate the technology and implementation of this solution, the following uses two-degree-of-freedom visual servoing for illustration. The system is an eye-in-hand type, that is, the camera is installed at the end of the robotic arm. The specific implementation steps of the present invention are as follows:

[0050] Construct a visual servo model based on image information and joint information; the specific implementation process includes:

[0051] Step 1: Build a visual servo model using images and joint information

[0052] Figure 1It is a structure diagram of the visual servo system, which reflects the physical structure of the system. It is mainly divided into three parts. The first part is the robot arm part, that is, the main execution part and the controlled part. After the controller generates an input command, the robot arm specifically performs motion operations to complete the servo task. The second part is the perception part. The present invention uses a camera. The camera collects images of objects in space and transmits them to the computing platform for processing. It is mainly necessary to extract and locate the feature points of the collected image. After positioning, it is necessary to match the feature points of the expected image with the current image to construct the task function and control rate. The third part is the target object, a three-dimensional object in space, and the object information needs to be collected by the camera to construct the feature points.

[0053] Let y=(u,v) T is the coordinate of the feature point in the pixel plane, which can be expressed as:

[0054]

[0055] Among them, z represents the depth of the feature point in the camera coordinates, is the product of the internal parameter matrix and the external parameter matrix, is the i-th row of M, is the homogeneous transformation matrix from the robot base coordinate system to the camera coordinate system, and x is the three-dimensional coordinate of the feature point in the base coordinate system.

[0056] Depth can be expressed as:

[0057]

[0058] Let R and P represent The rotation and translation components of (2) are differentiated with respect to time:

[0059]

[0060] Among them, d T for:

[0061]

[0062] In the formula, Representative Matrix The i-th row of (consisting of the first three columns of M). Taking the time derivative of (1) we get:

[0063]

[0064] in, is the depth-independent image Jacobian matrix of the unified kinematics framework and can be expressed as:

[0065]

[0066] The joint angles of the two degrees of freedom are q1 and q2, and the connecting rod lengths are l1 and l2, then It is expressed as follows:

[0067]

[0068] Parameter separation is performed based on the visual servo model to obtain unknown parameters; the specific implementation process includes:

[0069] Step 2: Parameter separation

[0070] In the above-mentioned visual servo system, the extrinsic parameters and intrinsic parameters of the camera, the uncertain dynamic parameters and the time-varying depth parameters of the feature points are all unknown. It is very desirable to separate the multiple uncertainties of the processing system, which has the advantages of avoiding over-parameterization and simplifying the control scheme, thereby making the performance of the closed-loop system better. The Jacobian matrix is ​​nonlinear, and the depth parameter in the Jacobian matrix is ​​also nonlinear. The present invention linearizes the components of the Jacobian matrix and the depth using a regression matrix and an unknown parameter vector, respectively.

[0071] The process of obtaining unknown parameters by parameter separation based on the visual servo model includes:

[0072] The known parameters are obtained by performing parameter separation based on the differential formula of the coordinates of the feature points in the pixel coordinates;

[0073] Parameter separation is performed based on the differential formula of the depth of the feature point in the camera coordinate, and the depth of the feature point in the camera coordinate is linearly parameterized to obtain the depth parameter and kinematic parameter;

[0074] The dynamic equation of the robotic arm is constructed, and the dynamic parameters are obtained by linearizing the dynamic equation of the robotic arm based on the regression matrix.

[0075] According to the above formula (5), the known parameters and unknown parameters are separated to obtain:

[0076]

[0077]

[0078] in, is generated by the joint angles q1 and q2 The set of elements in is the known parameters.

[0079] According to the above formula (3),

[0080] d T η=Y k,2 (q,η)θ k (10)

[0081]

[0082] The depth z has the following linear parameterization form:

[0083] z=Y z (q)θ z (12)

[0084] Among them, Y z (q)=(r 11 ,r 12 ,r 13 ,r 21 ,r 22 ,r 23 ,r 31 ,r 32 ,r 33 ,ρ1,ρ2,ρ3,1) are generated by the joint angles q1 and q2 The set of elements in .

[0085] The above θ k and θ z It is a parameter composed of a mixture of elements of M and x, that is, an unknown parameter.

[0086] The dynamic equation of the robotic arm is:

[0087]

[0088] In the formula and are the joint velocity and acceleration, is the inertia matrix, is the Coriolis matrix, is the gravitational moment, is the control torque vector to be designed, equation (13) can be transformed into a linear form:

[0089]

[0090] in, is the regression matrix, n is the number of joint angles, and p is the number of unknown parameters.

[0091] Construct an image velocity observer and design an adaptive estimation law based on the image velocity observer; estimate the unknown parameters based on the adaptive estimation law to realize visual servo finite time trajectory tracking control; the specific implementation process includes:

[0092] Step 3: Design adaptive finite-time control

[0093] Define the image tracking error e:

[0094] e=yyd (15)

[0095] Define the compensated depth-independent Jacobian matrix:

[0096]

[0097] Among them, Q is called the compensated depth-independent image Jacobian matrix. The compensated depth-independent image Jacobian matrix is ​​introduced to take into account the lack of time-varying depth in controller design.

[0098] Therefore, the estimated value of Q is:

[0099]

[0100] It can be inferred that:

[0101]

[0102] in,

[0103] The image velocity observer is:

[0104]

[0105] Among them, y ob is the observed value of the image space position, α is the first positive design constant, β is the second positive design constant, and They are z and Estimates.

[0106] The compensation-based depth-independent image Jacobian matrix is ​​defined as follows:

[0107]

[0108] Among them, λ is a constant to be determined, is the reference speed, is the desired trajectory velocity. Using the estimated depth and the estimated compensated depth-independent image Jacobian matrix, the reference joint velocity is defined:

[0109]

[0110] Differentiating the reference joint velocity with time, we get the following reference joint acceleration:

[0111]

[0112] The sliding vector in the joint space is defined as follows:

[0113]

[0114] By using the time-varying estimated dynamic parameters, sliding vector and image position tracking error, the following adaptive finite-time dynamic visual tracking controller is proposed:

[0115]

[0116] The first term in the above formula is the feedforward control term, the second term corresponds to the feedback control term in the image space, and the second term corresponds to the feedback control term in the joint space. K1 and K2 are the gain matrices to be determined, α1 and α2 are the constants to be determined, sign(*) α is a nonlinear function, defined as:

[0117] sign(ξ) α =[|ξ1| α sign(ξ1),...,|ξ n | α sign(ξ n )] T (25)

[0118] in ξ is a random parameter with no specific meaning, and sign(·) is a standard sign function:

[0119]

[0120] To obtain the above estimates, the adaptation law is as follows:

[0121]

[0122] Among them Ψ d ,Ψ k and z is a positive definite symmetric matrix of appropriate dimension.

[0123] Figure 2 It is the control system block diagram. The first term of the controller is the compensation of the forward dynamic compensation, which is estimated online by the adaptive law of the unknown dynamic parameters. Dynamic compensation is conducive to eliminating the negative effects of nonlinearity and load changes of the robot arm during high-speed movement. The second term is the proportional term of the fractional image error. This term provides a time-varying control input to force the system state to reach equilibrium. The last term includes the velocity state in the joint space, which is intended to improve the rapid response of the system.

[0124] Figure 3 is the feature point trajectory diagram, from Figure 3 It can be seen that both finite-time control and asymptotically stable control can successfully track dynamic targets and complete servo tasks. Figure 4 is the pixel position error norm of the visual tracking task, Figure 5is the pixel position error of the visual tracking task. The simulation results show that the convergence time of the finite-time control scheme is significantly shorter than that of the asymptotically stable controller.

[0125] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A visual servo finite time control method based on velocity observer, characterized in that: include: Construct a visual servo model based on image information and joint information; The process of constructing the visual servo model includes constructing feature points, and the coordinate formula of the feature points in pixel coordinates is: In the formula, , y is the coordinate of the feature point in the pixel plane, , Indicates the depth of the feature point in camera coordinates, is the product of the internal parameter matrix and the external parameter matrix, for No. OK, is the homogeneous transformation matrix from the robot base coordinate system to the camera coordinate system, is the three-dimensional coordinate of the feature point in the base coordinate system; The depth of the feature point in the camera coordinates and the coordinates of the feature point in the pixel coordinates are differentiated in time to obtain: In the formula, It represents the time differentiation of the depth of the feature point in the camera coordinate. Indicates the time differentiation of the coordinates of the feature point in pixel coordinates, , R and P represent The rotation and translation components of Representative Matrix No. OK, represents the joint velocity, Depth-independent image Jacobian matrix representing the unified kinematic frame; Perform parameter separation based on the visual servo model to obtain unknown parameters; The unknown parameters include: depth parameters, kinematic parameters and dynamic parameters; the process of obtaining the unknown parameters by performing parameter separation based on the visual servo model includes: The known parameters are obtained by performing parameter separation based on the differential formula of the coordinates of the feature points in the pixel coordinates; Parameter separation is performed based on the differential formula of the depth of the feature point in the camera coordinate, and the depth of the feature point in the camera coordinate is linearly parameterized to obtain the depth parameter and kinematic parameter; Constructing a dynamic equation of the robot arm, and linearizing the dynamic equation of the robot arm based on a regression matrix to obtain dynamic parameters; The linear expression of the mechanical arm dynamics equation based on the regression matrix is: In the formula, is the regression matrix, is the number of joint angles, is the number of unknown parameters, and are the joint velocity and acceleration, is the inertia matrix, is the Coriolis matrix, is the gravitational moment, is the joint angle; An image velocity observer is constructed, and an adaptive estimation law is designed based on the image velocity observer using the Lyapunov method; Wherein, the image velocity observer is: In the formula, represents the image velocity observer, is the observed value of the image space position, is the first positive design constant, is the second positive design constant, and They are and The estimate, is the expected trajectory, To estimate the Jacobian matrix of compensated depth-independent images; The adaptive estimation law is: In the formula, , and denote the positive definite symmetric matrices with appropriate dimensions in the three parameter estimates, is the standard symbolic function, represents the reference joint velocity, represents the reference joint acceleration, represents the image position tracking error, is the gain matrix to be determined, is a constant to be determined; Estimating the unknown parameters based on the adaptive estimation law to achieve visual servo finite time trajectory tracking control; The process of realizing visual servo finite time trajectory tracking control includes constructing an adaptive finite time dynamic visual tracking controller; the expression of the adaptive finite time dynamic visual tracking controller is: In the formula, and is the gain matrix to be determined, and is a constant to be determined, is a nonlinear function, e represents the image position tracking error, and s represents the sliding vector in the joint space.

Citation Information

Patent Citations

  • Uncalibrated mechanical-arm visual servo control device

    CN106737774A

  • Visual servo trajectory tracking control method and system oriented to mechanical arm

    CN112894812A

  • Visual servo control method based on event triggering model prediction

    CN117506913A

  • Uncalibrated visual servoing using real-time velocity optimization

    US20120307027A1

  • An adaptive vision-based controller

    WO1989001850A1