A robot uncalibrated visual servoing control method with fast convergence

Abstract

The application discloses a robot non-calibration visual servo control method with fast convergence, which comprises the following steps: constructing a perspective projection model containing unknown camera internal and external parameters; linear parameterization; designing an auxiliary matrix, an auxiliary vector and a sliding variable; designing an adaptive law for online updating unknown parameter information; designing a torque controller of a robot visual servo system; and feeding back the calculated torque value to the mathematical model of the degree of freedom robot system. The application realizes the visual servo control of the robot under the condition that the camera internal and external parameters are unknown, controls the robot to move to the desired position quickly, and guarantees the safety and reliability of the robot visual servo system in the complex and changeable working environment.

Description

Technical Field

[0001] This invention belongs to the field of stability and control technology of robot vision servo systems, and specifically relates to a calibration-free vision servo control method for achieving rapid convergence. Background Technology

[0002] In contemporary society, robots are widely used in various industries, such as industrial manufacturing, military, education, healthcare, and entertainment, and they are playing an increasingly important role. However, as the complexity of robot application environments and the difficulty of performing tasks increase, robots need to become more flexible and intelligent to better cope with these new challenges. Integrating sensors into robots is a feasible solution, endowing them with a certain degree of perception. Equipping robots with cameras and using the visual information acquired by the cameras to guide the robot's movement achieves a non-contact measurement motion control, namely, visual servo control. In traditional robot visual servo control methods, it is necessary to calibrate the intrinsic and extrinsic parameters of the camera offline, and then construct a mapping relationship between visual information and robot joint information based on the intrinsic and extrinsic parameters. This mapping relationship is crucial for the stability and accuracy of robot visual servo control. However, offline camera calibration methods are often limited in unstructured environments and high-temperature and high-risk environments; furthermore, as the robot's operating environment changes, such as small shifts in camera position, offline camera calibration methods may struggle to handle such changes.

[0003] Currently, existing visual servo control frameworks are divided into three types: position-based visual servo control, image-based visual servo control, and hybrid visual servo control. Position-based and hybrid visual servo control involve the robot's 3D position information to varying degrees when designing the objective function. Since constructing the connection between 3D position information and 2D image information relies on offline calibrated camera intrinsic and extrinsic parameters, these two frameworks are not suitable for integration with uncalibrated visual servo control. Image-based visual servo control relies solely on 2D image information to construct the objective function, making this framework highly suitable for integration with uncalibrated visual servo control.

[0004] In image-based visual servo control frameworks, researchers have developed uncalibrated visual servo control methods that update the image Jacobian matrix or interaction matrix used to construct the relationship between 2D image information and robot joints by estimating the intrinsic and extrinsic parameters of the camera online. However, existing uncalibrated visual servo control methods often neglect the convergence of parameter estimation, assuming that parameter convergence does not affect control performance and that the method is effective as long as the control function is achieved. However, it is a well-known fact that if parameters converge to their true values, it will improve the overall performance of the control system. Therefore, some researchers have proposed a composite learning method that uses the prediction error to design an adaptive law to update unknown camera parameters online, making the estimated camera parameters converge to their true values, thereby improving the overall performance of uncalibrated visual servo control. However, this method often faces difficulties in analyzing the convergence of the estimated parameters due to the coupling between the estimated parameters and the predictor error.

[0005] Robot visual servoing systems often rely heavily on the accuracy of offline calibrated camera parameters. During robot operation, minor shifts in camera position inevitably occur, rendering previously calibrated camera parameters invalid and affecting the control performance of the robot's visual servoing system. Existing methods have limitations in online camera parameter estimation. Therefore, addressing the issue of uncalibrated visual servoing control for robots remains a worthy research topic for achieving better visual servoing control. Summary of the Invention

[0006] To address the technical problem of low performance in robot visual servo control under unknown camera intrinsic and extrinsic parameters in existing technologies, the primary objective of this invention is to provide a calibration-free visual servo control method for robots that achieves rapid convergence. This method enables online estimation of unknown camera parameters to their true values, thereby improving the tracking performance of the visual servo control system and ensuring the safety and reliability of the robot's visual servo control system in complex and ever-changing working environments.

[0007] The second objective of this invention is to provide a calibration-free vision servo control system for robots that achieves rapid convergence.

[0008] A third objective of this invention is to provide a processor.

[0009] The first objective of this invention is achieved by the following steps:

[0010] S1. Construct a perspective projection model containing unknown camera intrinsic and extrinsic parameters. Based on the perspective projection model, incorporate an invertible scaling matrix to obtain a new perspective projection model. Linearly parameterize the new perspective projection model.

[0011] S2. Based on the linearly parameterized perspective projection model, design auxiliary matrices and auxiliary vectors; simultaneously, design sliding variables based on pixel error information.

[0012] S3. Based on the auxiliary matrix, auxiliary vector, and sliding variable, design an adaptive law for online updating of unknown parameter information;

[0013] S4. Based on the parameter information and sliding variables estimated by the adaptive law, design the torque controller for the robot vision servo system.

[0014] S5. Based on the designed torque controller, the calculated torque value is fed back to... In the mathematical model of a degree-of-freedom robot system.

[0015] Preferably, the specific process of constructing the perspective projection model containing unknown camera intrinsic and extrinsic parameters in step S1 is as follows:

[0016] S101. In robot visual servoing technology with an eye-to-hand configuration, the perspective projection model used to represent the relationship between the position of a feature point fixed at the robot's end effector in the world coordinate system and its position in the image coordinate system is:

[0017] ;

[0018] in, This indicates the position of the feature point in the image coordinate system. , Representing the coordinates in the image coordinate system shaft and Components of the axis, This represents the depth of the feature point in the camera coordinate system. This represents the camera intrinsic parameter matrix. This represents the extrinsic parameter matrix of the camera. This indicates the position of the feature point in the world coordinate system. They represent in the world coordinate system axis, shaft and Components of the axis, Indicates time, Indicates transpose. express 3D real vector Represents positive real numbers. express A real matrix, express A real matrix, express A dimensional real vector;

[0019] The specific form of the intrinsic parameter matrix is ​​as follows:

[0020] ;

[0021] in, Indicates the camera's focal length. and Representing the coordinates in the image coordinate system Axial direction and The number of pixels per unit distance along the axial direction. express shaft and Deflection between axes Indicates the coordinates of the principal point. , ;

[0022] The specific form of the camera extrinsic matrix is ​​as follows:

[0023] ;

[0024] in, Represents the rotation matrix. Represents the translation vector. , and They represent axis, shaft and Translation components of the axis;

[0025] S102, Define the scaling intrinsic parameter matrix ,in This represents an invertible scaling matrix. The definition of is:

[0026] ;

[0027] in, ;

[0028] Define a new matrix ,matrix The specific form is as follows:

[0029] ;

[0030] in, express The first line, express The second line, express The 3rd line, Set it to a small constant value;

[0031] Depth information is represented as:

[0032] ;

[0033] in, Representation matrix The third line;

[0034] S103, will Substituting the perspective projection model, we get:

[0035] .

[0036] Preferably, step S1, which linearly parameterizes the new perspective projection model, specifically involves: letting A new camera model is obtained by linearly parameterizing the new perspective projection model:

[0037] ;

[0038] in, , This is a regression matrix, and its specific form is as follows:

[0039] ;

[0040] in, express The first element, express The second element, express The third element, Indicates inclusion The subvectors of the first 3 elements; in the new perspective projection model , express The 4th element, Representation matrix The element in the 3rd row and 4th column.

[0041] Preferably, the specific process of designing the auxiliary matrix and auxiliary vector in step S2 is as follows:

[0042] S201, Designing an auxiliary regression matrix and auxiliary vector The specific expression is as follows:

[0043] ;

[0044] in, Forgetting factor, Used to adjust incentive levels;

[0045] S202, Define another auxiliary variable The expression for this variable is:

[0046] ;

[0047] in, express The estimated value, Indicates the estimation error;

[0048] S203, Define another auxiliary matrix As shown below:

[0049] ;

[0050] Initial conditions ,in , It is the identity matrix;

[0051] S204. Based on the expressions in S201 and S203, the following relationship is obtained:

[0052] ;

[0053] in, This represents the residual term that converges to 0, i.e. ,and ;

[0054] S205. Based on the expressions in S202 and S204, define another auxiliary vector. The specific expression is as follows:

[0055] .

[0056] Preferably, the specific process of designing the sliding variable in step S2 is as follows:

[0057] S206, Define pixel error The specific expression is as follows:

[0058] ;

[0059] in, Indicates the current pixel position. Indicates the desired pixel position;

[0060] S207. Based on the definition of pixel error, define the sliding variable. The expression is as follows:

[0061] ;

[0062] in, Indicates pixel error rate. , , , ;parameter The magnitude of the hyperbolic tangent function determines The degree of approximation, The smaller the value, the better for the sign function. The closer the approximation, the better; Let be a diagonal matrix, where They represent The first, second, and third elements; in the sign function In the middle, when hour, ;when , ;when , .

[0063] Preferably, the adaptive law of step S3 The expression is:

[0064] ;

[0065] in, For auxiliary vectors and For sliding variables, Represents a positive constant learning gain; , , and Both represent design parameters; the adaptive law updates unknown parameter information online. .

[0066] Preferably, the specific process of designing the torque controller for the robot vision servo system in step S4 is as follows:

[0067] S401. First, establish the relationship between the image velocity of the feature points and the velocity of the feature points in the world coordinate system. The expression is:

[0068] ;

[0069] in, Indicates image speed. Represents a depth-independent interaction matrix. This represents the velocity of the feature point in the world coordinate system;

[0070] S402. Establish the relationship between the velocity of feature points in the world coordinate system and the robot joint velocity using robot forward kinematics, as shown in the following expression:

[0071] ;

[0072] in, The robot Jacobian matrix representing feature points; Indicates the speed of the robot's joints;

[0073] S403. Substituting the expression in S402 into the expression in S401, we get:

[0074] ;

[0075] in, ;

[0076] S404, Introducing a The mathematical model of a degree-of-freedom robot system is expressed as follows:

[0077] ;

[0078] in, These represent the robot's joint position, joint angular velocity, and joint angular acceleration, respectively. The inertia matrix of the robot system is represented by... This represents the centrifugal torque of the robot system. This represents the gravitational torque of the robot system; This represents the control torque of the robot system; express A real matrix, express 3D real vector;

[0079] S405. Design a torque controller based on the estimated parameters and the sliding variable containing image information, as shown in the following expression:

[0080] ;

[0081] in, express The false rebellion, express The estimated value; , Represents a diagonal matrix. They represent The first element, the second element, and the third element, express The derivative; , Indicates pixel error rate. Represents a diagonal matrix. They represent The first element, the second element, and the third element, Represents a diagonal matrix. They represent The first, second, and third elements.

[0082] Preferably, step S5 specifically includes:

[0083] S501. Based on the torque controller of the designed robot system, calculate the control torque required for the robot joint movement;

[0084] S502. The calculated control torque command is sent to the integrated controller of the robot system via the CAN bus.

[0085] S503, the integrated controller converts the received control signals into drive signals for the robot joint motors, controlling the speed and direction of the robot joint motors, and moving the robot joints. In other words, the robot is guided by visual information to move to the desired pixel position.

[0086] The second objective of this invention is achieved by including:

[0087] The perspective projection model construction and linear parameterization module is used to construct a perspective projection model containing unknown camera intrinsic and extrinsic parameters, obtain a new perspective projection model by incorporating an invertible scaling matrix into the perspective projection model, and linearly parameterize the new perspective projection model.

[0088] The auxiliary design module is used to design auxiliary matrices and auxiliary vectors based on the linearly parameterized perspective projection model; at the same time, it designs sliding variables based on pixel error information.

[0089] The adaptive law design module is used to design adaptive laws for online updating of unknown parameter information based on auxiliary matrices, auxiliary vectors, and sliding variables.

[0090] The torque controller design module is used to design the torque controller for a robot vision servo system based on the parameter information and sliding variables estimated by the adaptive law.

[0091] The torque feedback module is used to feed back the calculated torque value based on the designed torque controller. In the mathematical model of a degree-of-freedom robot system.

[0092] The third objective of the present invention is achieved by the processor running a program that, when running, executes any of the described methods for achieving fast convergence of the robot's uncalibrated visual servo control.

[0093] The beneficial effects of this invention are:

[0094] 1. This invention features fast convergence speed and strong robustness, enabling tracking control of the robot vision servo system and ensuring its safety and reliability in complex and ever-changing working environments.

[0095] 2. This invention, based on a linearly parameterized perspective projection model, designs auxiliary matrices and auxiliary vectors to extract parameter error information instead of using traditional methods where parameter estimation and prediction errors are coupled. It also utilizes designed sliding variables, which together are used to design an adaptive law to estimate unknown camera parameters online, bringing the estimated camera parameters to their true values ​​and improving control performance. A torque controller is designed using the estimated parameters and sliding variables incorporating a hyperbolic tangent function to avoid singularity problems. The torque value calculated by the torque controller is fed back into the mathematical model of the robot system, forming a closed-loop control that enables the robot to perform the desired motion based on visual information. Attached Figure Description

[0096] Figure 1 This is a flowchart of the method of the present invention;

[0097] Figure 2 This is a schematic diagram of the 7-DOF robot vision servo system in Example 7;

[0098] Figure 3 This is the parameter estimation response provided in Example 7;

[0099] Figure 4 It is the parameter estimation error in norm form provided in Example 7;

[0100] Figure 5 This is the pixel tracking error response provided in Example 7. Detailed Implementation

[0101] The present invention will be further described below with reference to the embodiments and accompanying drawings, but this does not limit the present invention in any way. Any changes or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0102] Example 1

[0103] As attached Figure 1 As shown, the robot calibration-free vision servo control method for achieving fast convergence in this embodiment includes the following steps:

[0104] S1. Construct a perspective projection model containing unknown camera intrinsic and extrinsic parameters. Based on the perspective projection model, incorporate an invertible scaling matrix to obtain a new perspective projection model. Linearly parameterize the new perspective projection model.

[0105] S2. Based on the linearly parameterized perspective projection model, design auxiliary matrices and auxiliary vectors; simultaneously, design sliding variables based on pixel error information.

[0106] S3. Based on the auxiliary matrix, auxiliary vector, and sliding variable, design an adaptive law for online updating of unknown parameter information;

[0107] S4. Based on the parameter information and sliding variables estimated by the adaptive law, design the torque controller for the robot vision servo system.

[0108] S5. Based on the designed torque controller, the calculated torque value is fed back to... In the mathematical model of a degree-of-freedom robot system.

[0109] Example 2

[0110] The method for fast-converging, calibration-free visual servo control of a robot in this embodiment is based on Embodiment 1. Specifically, step S1, which constructs a perspective projection model containing unknown camera intrinsic and extrinsic parameters, involves the following steps:

[0111] S101. In robot visual servoing technology with an eye-to-hand configuration, the perspective projection model used to represent the relationship between the position of a feature point fixed at the robot's end effector in the world coordinate system and its position in the image coordinate system is:

[0112] (1);

[0113] in, This indicates the position of the feature point in the image coordinate system. , Representing the coordinates in the image coordinate system shaft and Components of the axis, This represents the depth of the feature point in the camera coordinate system. This represents the camera intrinsic parameter matrix. This represents the extrinsic parameter matrix of the camera. This indicates the position of the feature point in the world coordinate system. They represent in the world coordinate system axis, shaft and Components of the axis, Indicates time, Indicates transpose. express 3D real vector Represents positive real numbers. express A real matrix, express A real matrix, express A dimensional real vector;

[0114] The specific form of the intrinsic parameter matrix is ​​as follows:

[0115] (2);

[0116] in, Indicates the camera's focal length. and Representing the coordinates in the image coordinate system Axial direction and The number of pixels per unit distance along the axial direction. express shaft and Deflection between axes Indicates the coordinates of the principal point. , ;

[0117] The specific form of the camera extrinsic matrix is ​​as follows:

[0118] (3);

[0119] in, Represents the rotation matrix. Represents the translation vector. , and They represent axis, shaft and Translation components of the axis;

[0120] S102, Considering the camera intrinsic parameter matrix Some elements in the matrix are usually larger than those in the extrinsic matrix. The elements in the matrix are much larger. To balance the intrinsic and extrinsic parameters, a scaling intrinsic parameter matrix is ​​defined. ,in This represents an invertible scaling matrix. The definition of is:

[0121] (4);

[0122] in, ;

[0123] Define a new matrix ,matrix The specific form is as follows:

[0124] (5);

[0125] in, express The first line, express The second line, express The 3rd line, Set it to a small constant value;

[0126] Depth information is represented as:

[0127] (6);

[0128] in, Representation matrix The third line;

[0129] S103, will Substituting the perspective projection model, we get:

[0130] (7);

[0131] S104, Order A new camera model is obtained by linearly parameterizing the new perspective projection model:

[0132] (8);

[0133] in, , This is a regression matrix, and its specific form is as follows:

[0134] (9);

[0135] in, express The first element, express The second element, express The third element, Indicates inclusion The subvectors of the first 3 elements; in the new perspective projection model , express The 4th element, Representation matrix The element in the 3rd row and 4th column;

[0136] In step S1, because the intrinsic and extrinsic parameters of the monocular camera in the perspective projection model (1) are unknown, the linear parameterized camera model established in (8) suffers from... To achieve uncalibrated visual servoing, an adaptive law is designed to estimate the unknown intrinsic and extrinsic parameters of the camera, thereby establishing a connection between the image coordinate system and the world coordinate system.

[0137] The specific process of designing the auxiliary matrix and auxiliary vector in step S2 is as follows:

[0138] S201. Based on the linearly parameterized camera model shown in (8), design an auxiliary regression matrix. and auxiliary vector The specific expression is as follows:

[0139] (10);

[0140] in, Forgetting factor, Used to adjust incentive levels;

[0141] S202, Define another auxiliary variable The expression for this variable is:

[0142] (11);

[0143] in, express The estimated value, Indicates the estimation error;

[0144] S203, due to Includes Information, and because The changes will affect This has an impact, which in turn affects the performance of parameter estimation. To eliminate this impact, another auxiliary matrix is ​​defined. As shown below:

[0145] (12);

[0146] Initial conditions ,in , It is the identity matrix;

[0147] S204. According to equations (10) and (12), the following relationship is obtained:

[0148] (13);

[0149] in, This represents the residual term that converges to 0, i.e. ,and ;

[0150] S205. Based on (11) and (13), define another auxiliary vector. The specific expression is as follows:

[0151] (14);

[0152] S206. Before defining the sliding variable, define the pixel error first. The specific expression is as follows:

[0153] (15);

[0154] in, Indicates the current pixel position. Indicates the desired pixel position;

[0155] S207. Based on the definition of pixel error, define the sliding variable. The expression is as follows:

[0156] (16);

[0157] in, Indicates pixel error rate. , , , ;parameter The magnitude of the hyperbolic tangent function determines The degree of approximation, The smaller the value, the better for the sign function. The closer the approximation, the better; Let be a diagonal matrix, where They represent The first, second, and third elements; in the sign function In the middle, when hour, ;when , ;when , It should be noted that other symbolic functions in this invention... Similarly;

[0158] The auxiliary vector of (14) will be used The sliding variables in (16) Design Adaptive Law Online Update S3 Step Adaptive Law The expression is:

[0159] (17);

[0160] in, For auxiliary vectors and For sliding variables, Represents a positive constant learning gain; , , and Both represent design parameters; the adaptive law updates unknown parameter information online. ;

[0161] The specific process of designing the torque controller for a robot vision servoing system using steps S4 is as follows:

[0162] S401. Before designing the torque controller, the relationship between the image velocity of the feature points and the velocity of the feature points in the world coordinate system will be established. To achieve this, the derivative of the perspective projection model (1) in step S101 can be obtained as follows:

[0163] (18);

[0164] in, Indicates image speed. Represents a depth-independent interaction matrix. This represents the velocity of the feature point in the world coordinate system;

[0165] S402. Establish the relationship between the velocity of feature points in the world coordinate system and the robot joint velocity using robot forward kinematics, as shown in the following expression:

[0166] (19);

[0167] in, The robot Jacobian matrix representing feature points; Indicates the speed of the robot's joints;

[0168] S403. Substituting (19) into (18), we get:

[0169] (20);

[0170] in, ;

[0171] S404. Before designing the torque controller, introduce a... The mathematical model of a degree-of-freedom robot system is expressed as follows:

[0172] (twenty one);

[0173] in, These represent the robot's joint position, joint angular velocity, and joint angular acceleration, respectively. The inertia matrix of the robot system is represented by... This represents the centrifugal torque of the robot system. This represents the gravitational torque of the robot system; This represents the control torque of the robot system; express A real matrix, express 3D real vector;

[0174] S405. Design a torque controller based on the estimated parameters and the sliding variable containing image information, as shown in the following expression:

[0175] (twenty two);

[0176] in, express The false rebellion, express The estimated value; , Represents a diagonal matrix. They represent The first element, the second element, and the third element, express The derivative; , Indicates pixel error rate. Represents a diagonal matrix. They represent The first element, the second element, and the third element, Represents a diagonal matrix. They represent The first, second, and third elements;

[0177] The S5 steps are as follows:

[0178] S501. Based on the torque controller of the designed robot system, calculate the control torque required for the robot joint movement;

[0179] S502. The calculated control torque command is sent to the integrated controller of the robot system via the CAN bus.

[0180] S503, the integrated controller converts the received control signals into drive signals for the robot joint motors, controlling the speed and direction of the robot joint motors, and moving the robot joints. In other words, the robot is guided by visual information to move to the desired pixel position.

[0181] Example 3

[0182] This embodiment implements a fast-converging, calibration-free vision servo control system for robots, including:

[0183] The perspective projection model construction and linear parameterization module is used to construct a perspective projection model containing unknown camera intrinsic and extrinsic parameters, obtain a new perspective projection model by incorporating an invertible scaling matrix into the perspective projection model, and linearly parameterize the new perspective projection model.

[0184] The auxiliary design module is used to design auxiliary matrices and auxiliary vectors based on the linearly parameterized perspective projection model; at the same time, it designs sliding variables based on pixel error information.

[0185] The adaptive law design module is used to design adaptive laws for online updating of unknown parameter information based on auxiliary matrices, auxiliary vectors, and sliding variables.

[0186] The torque controller design module is used to design the torque controller for a robot vision servo system based on the parameter information and sliding variables estimated by the adaptive law.

[0187] The torque feedback module is used to feed back the calculated torque value based on the designed torque controller. In the mathematical model of a degree-of-freedom robot system.

[0188] Example 4

[0189] The processor in this embodiment is used to run a program. When the program runs, it executes the robot uncalibrated visual servo control method for achieving fast convergence as described in Embodiment 1.

[0190] Example 5

[0191] The processor in this embodiment is used to run a program. When the program runs, it executes the robot uncalibrated visual servo control method for achieving fast convergence as described in Embodiment 2.

[0192] Example 6

[0193] Based on the adaptive law and torque controller designed in Example 2, the stability of the entire system is analyzed as follows:

[0194] (a) Sliding Variable and estimated parameters Convergence analysis:

[0195] In order to analyze the sliding variable in (16) The dynamic characteristics of, Taking the first derivative, we get:

[0196] (twenty three)

[0197] in, .

[0198] Choosing Lyapunov functions Its first derivative can be expressed as:

[0199] (twenty four)

[0200] Substitute (17) and (23) into (24), and from Lyapunov function first derivative It can be changed to:

[0201] (25)

[0202] Lyapunov function in formula (25) first derivative It can be seen that, It has an upper bound, which means that the sliding variable... , It is bounded. Therefore, the adaptive law and controller proposed in this invention can guarantee the stability of the system, and under the drive of the adaptive law (17), the estimated camera parameters can be guaranteed to converge to their true values, thereby improving the control performance of the system; in addition, under the drive of the controller (22), the sliding variable can reach the sliding surface to realize the desired motion of the robot.

[0203] (II) Pixel Error Convergence Analysis:

[0204] When the sliding variable reaches the sliding surface, that is At this point, we have:

[0205] (26)

[0206] Choosing Lyapunov functions Its first derivative can be expressed as:

[0207] (27)

[0208] From the above pixel error convergence analysis, it can be seen that, according to the sliding variable of formula (16) The torque controller designed based on the parameter information estimated by the adaptive law of formula (17) This allows the robot to move to the desired pixel position, i.e., the pixel error. .

[0209] Example 7

[0210] To verify the effect of the adaptive law-based torque controller designed in the embodiment on pixel errors in the robot vision servo system. To assess the effectiveness and feasibility of the control system, a numerical simulation of a robot vision servoing system was built on Matlab Simulink. The simulation platform employed a 7-DOF robot (see attached diagram). Figure 2 As shown, the robot's joint positions, velocities, and end-effector positions are obtained through robot kinematics and dynamics. Pixel information of the robot's end-effector position is obtained through numerical simulation.

[0211] The theoretical derivation in this embodiment is based on a single feature point fixed on the robot's end effector. It's important to note that the proposed theory can be extended to scenarios with multiple feature points simply by slightly expanding the matrix dimension. Since a single feature point cannot completely constrain the end effector's position, the position may differ under different operating conditions. Therefore, in the simulation, four non-collinear feature points are fixed on the robot's end effector for visual servo control. The desired pixel positions of the four feature points are set as follows: .

[0212] Simulate the intrinsic parameter matrix of a real-world RealSense camera, and then use the desired intrinsic parameter matrix. The settings are as follows:

[0213]

[0214] Similarly, to simulate the extrinsic parameter matrix in reality, the desired extrinsic parameter matrix will be... The settings are as follows:

[0215]

[0216] Initial intrinsic parameter matrix With the initial extrinsic matrix Set them to:

[0217]

[0218] Invertible scaling matrix Set to:

[0219]

[0220] The robot's initial joint positions are set to rad. The relevant parameter settings for the adaptive law and torque controller are as follows: , ,in Represents the identity matrix, initial value ,in Represents the identity matrix. This represents a diagonal matrix.

[0221] Simulation results of uncalibrated vision servo control for robots are attached. Figure 3 ~Appendix Figure 5 As shown. From the appendix Figure 3 The response and attachment of parameter estimation in Figure 4 The convergence results of the parameter estimation error norm show that the adaptive law proposed in this invention can achieve rapid estimation of unknown camera parameters and converge to their true values. It should be noted that the estimated parameter values ​​are scaled down, while the parameter estimation response graph describes the true camera parameter values ​​in the form of scale matrix reconstruction. Furthermore, from the appendix... Figure 5 The response results of the average pixel error show that the torque controller proposed in this invention can achieve rapid convergence of pixel errors, that is, control the robot to move to the desired position. Therefore, the effectiveness of the proposed uncalibrated visual servo control method for robots can be verified from the simulation results.

Claims

1. A method for achieving fast convergence of calibration-free vision servo control for robots, characterized in that... Includes the following steps: S1. Construct a perspective projection model containing unknown camera intrinsic and extrinsic parameters. Based on the perspective projection model, incorporate an invertible scaling matrix to obtain a new perspective projection model. Linearly parameterize the new perspective projection model. S2. Based on the linearly parameterized perspective projection model, design auxiliary matrices and auxiliary vectors; simultaneously, design sliding variables based on pixel error information. S3. Based on the auxiliary matrix, auxiliary vector, and sliding variable, design an adaptive law for online updating of unknown parameter information; S4. Based on the parameter information and sliding variables estimated by the adaptive law, design the torque controller for the robot vision servo system. S5. Based on the designed torque controller, the calculated torque value is fed back to... In the mathematical model of a degree-of-freedom robot system; The specific process of designing the auxiliary matrix and auxiliary vector in step S2 is as follows: S201, Designing an auxiliary regression matrix and auxiliary vector The specific expression is as follows: ； in, Forgetting factor, Used to adjust incentive levels; S202, Define another auxiliary variable The expression for this variable is: ； in, express The estimated value, Indicates the estimation error; S203, Define another auxiliary matrix As shown below: ； Initial conditions ,in , It is the identity matrix; S204. Based on the expressions in S201 and S203, the following relationship is obtained: ； in, This represents the residual term that converges to 0, i.e. ,and ; S205. Based on the expressions in S202 and S204, define another auxiliary vector. The specific expression is as follows: ； The specific process of designing the sliding variable in step S2 is as follows: S206, Define pixel error The specific expression is as follows: ； in, Indicates the current pixel position. Indicates the desired pixel position; S207. Based on the definition of pixel error, define the sliding variable. The expression is as follows: ； in, Indicates pixel error rate. , , , ;parameter The magnitude of the hyperbolic tangent function determines The degree of approximation, The smaller the value, the better for the sign function. The closer the approximation, the better; Let be a diagonal matrix, where They represent The first, second, and third elements; in the sign function In the middle, when hour, ;when , ;when , .

2. The robot calibration-free vision servo control method for achieving fast convergence according to claim 1, characterized in that... The specific process of constructing the perspective projection model containing unknown intrinsic and extrinsic parameters of the camera in step S1 is as follows: S101. In robot visual servoing technology with an eye-to-hand configuration, the perspective projection model used to represent the relationship between the position of a feature point fixed at the robot's end effector in the world coordinate system and its position in the image coordinate system is: ； in, This indicates the position of the feature point in the image coordinate system. , Representing the coordinates in the image coordinate system shaft and Components of the axis, This represents the depth of the feature point in the camera coordinate system. This represents the camera intrinsic parameter matrix. This represents the extrinsic parameter matrix of the camera. This indicates the position of the feature point in the world coordinate system. They represent in the world coordinate system axis, shaft and Components of the axis, Indicates time, Indicates transpose. express 3D real vector Represents positive real numbers. express A real matrix, express A real matrix, express A dimensional real vector; The specific form of the intrinsic parameter matrix is ​​as follows: ； in, Indicates the camera's focal length. and Representing the coordinates in the image coordinate system Axial direction and The number of pixels per unit distance along the axial direction. express shaft and Deflection between axes Indicates the coordinates of the principal point. , ; The specific form of the camera extrinsic matrix is ​​as follows: ； in, Represents the rotation matrix. Represents the translation vector. , and They represent axis, shaft and Translation components of the axis; S102, Define the scaling intrinsic parameter matrix ,in This represents an invertible scaling matrix. The definition of is: ； in, ; Define a new matrix ,matrix The specific form is: ； in, express The first line, express The second line, express The 3rd line, Set it to a small constant value; Depth information is represented as: ； in, Representation matrix The third line; S103, will Substituting the perspective projection model, we get: 。 3. The method for achieving fast convergence of calibration-free visual servo control for robots according to claim 2, characterized in that... Step S1 linearizes the new perspective projection model by parameterizing it: Specifically, it allows... A new camera model is obtained by linearly parameterizing the new perspective projection model: ； in, , This is a regression matrix, and its specific form is as follows: ； in, express The first element, express The second element, express The third element, Indicates inclusion The subvectors of the first 3 elements; in the new perspective projection model , express The 4th element, Representation matrix The element in the 3rd row and 4th column.

4. The method for achieving fast convergence of calibration-free visual servo control for robots according to claim 1, characterized in that... S3 Step Adaptive Law The expression is: ； in, For auxiliary vectors and For sliding variables, Represents a positive constant learning gain; , , and Both represent design parameters; the adaptive law updates unknown parameter information online. .

5. The method for achieving fast convergence of calibration-free visual servo control for robots according to claim 2, characterized in that... The specific process of designing the torque controller for a robot vision servoing system using steps S4 is as follows: S401. First, establish the relationship between the image velocity of the feature points and the velocity of the feature points in the world coordinate system. The expression is: ； in, Indicates image speed. Represents a depth-independent interaction matrix. This represents the velocity of the feature point in the world coordinate system; S402. Establish the relationship between the velocity of feature points in the world coordinate system and the robot joint velocity using robot forward kinematics, as shown in the following expression: ； in, The robot Jacobian matrix representing feature points; Indicates the speed of the robot's joints; S403. Substituting the expression in S402 into the expression in S401, we get: ； in, ; S404, Introducing a The mathematical model of a degree-of-freedom robot system is expressed as follows: ； in, These represent the robot's joint position, joint angular velocity, and joint angular acceleration, respectively. The inertia matrix of the robot system is represented by... This represents the centrifugal torque of the robot system. This represents the gravitational torque of the robot system; This represents the control torque of the robot system; express A real matrix, express 3D real vector; S405. Design a torque controller based on the estimated parameters and the sliding variable containing image information, as shown in the following expression: ； in, express The false rebellion, express The estimated value; , Represents a diagonal matrix. They represent The first element, the second element, and the third element, express The derivative; , Indicates pixel error rate. Represents a diagonal matrix. They represent The first element, the second element, and the third element, Represents a diagonal matrix. They represent The first, second, and third elements.

6. The method for achieving fast convergence of calibration-free visual servo control for robots according to claim 1, characterized in that... The S5 steps are as follows: S501. Based on the torque controller of the designed robot system, calculate the control torque required for the robot joint movement; S502. The calculated control torque command is sent to the integrated controller of the robot system via the CAN bus. S503, the integrated controller converts the received control signals into drive signals for the robot joint motors, controlling the speed and direction of the robot joint motors, and moving the robot joints. In other words, the robot is guided by visual information to move to the desired pixel position.

7. A control system for a robot calibration-free vision servo control method for achieving fast convergence according to claim 1, characterized in that... include: The perspective projection model construction and linear parameterization module is used to construct a perspective projection model containing unknown camera intrinsic and extrinsic parameters, obtain a new perspective projection model by incorporating an invertible scaling matrix into the perspective projection model, and linearly parameterize the new perspective projection model. The auxiliary design module is used to design auxiliary matrices and auxiliary vectors based on the linearly parameterized perspective projection model. Simultaneously, a sliding variable is designed based on pixel error information; The adaptive law design module is used to design adaptive laws for online updating of unknown parameter information based on auxiliary matrices, auxiliary vectors, and sliding variables. The torque controller design module is used to design the torque controller for a robot vision servo system based on the parameter information and sliding variables estimated by the adaptive law. The torque feedback module is used to feed back the calculated torque value based on the designed torque controller. In the mathematical model of a degree-of-freedom robot system.

8. A processor, characterized in that... The processor is used to run a program, which executes the robot uncalibrated visual servo control method for achieving fast convergence according to any one of claims 1 to 6.

Images