Mechanical arm visual servo tracking method based on adaptive neural network compensation

Abstract

The invention relates to the technical field of mechanical arm control, and particularly provides a mechanical arm visual servo tracking method based on adaptive neural network compensation, which performs adaptive neural network compensation on a traditional mechanical arm control rate according to an image feature error between a real-time image feature and an ideal image feature. And a compensated mechanical arm controller is obtained, the control joint speed of the mechanical arm is obtained according to the controller, and the control joint speed is applied to control the mechanical arm to track the dynamic target. According to the method, on the basis that the control rate of the mechanical arm servo system is not increased, the adaptive neural network is added to compensate the system tracking error, the tracking precision of the mechanical arm is greatly improved, human intervention is reduced, and the control complexity of the mechanical arm is simplified.

Description

Technical Field

[0001] The invention relates to the technical field of robot arm control, and specifically provides a robot arm visual servo tracking method based on adaptive neural network compensation. Background Art

[0002] The visual servo control of the robot arm is a comprehensive technology that integrates machine vision, automatic control and computer technologies. This technology realizes the servo motion control of the robot arm through visual signals, which can effectively improve the motion accuracy and robustness of the robot arm and enhance the interaction ability of the robot arm with complex environments. According to the different motion states of the target, the robot arm visual servo control algorithm can be divided into static target visual positioning control and dynamic target visual tracking control. The traditional dynamic target visual servo tracking control is based on the static visual servo control rate design, and uses filtering methods such as Kalman filtering or extended Kalman filtering to estimate the motion state of the dynamic target in real time. The specific calculation method is: real-time acquisition of the camera's motion speed, and the use of the image interaction matrix to calculate the image feature changes caused by the target motion. However, since the visual servo system is a control system with a low control frequency, the above calculation method will have a large system delay, resulting in inaccurate estimated image feature errors caused by target motion, resulting in low tracking accuracy and other problems. The control frequency of the robot arm visual servo system is limited by the camera sampling frequency and the complexity of the image processing algorithm, so it is difficult to increase the control frequency of the visual servo system. Therefore, it is of great significance to effectively improve the visual servo tracking accuracy and reduce the tracking error without increasing the control frequency of the visual servo system.

[0003] In traditional dynamic target visual tracking control based on filtering algorithms, due to factors such as control system delay and environmental noise, in dynamic target visual tracking tasks, when using the feedback camera speed combined with the filtering method to estimate the image feature changes caused by the target motion, there will be a relative lag resulting in reduced tracking accuracy. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a robot arm visual servo tracking method based on adaptive neural network compensation, which introduces adaptive neural network compensation into the traditional robot arm control rate. The compensation content is based on the actual joint speed and image feature error, which can improve the compensation accuracy and further improve the tracking accuracy of the robot arm visual servo system.

[0005] The present invention provides a robot arm visual servo tracking method based on adaptive neural network compensation, wherein a camera is provided at the end of the robot arm, and the camera can follow the movement of the robot arm, including: S1: Collect the real-time image of the target through the camera and extract the real-time image features of the real-time image ; S2: Extract reference image features based on the ideal image of the target , based on real-time image features and reference image features Calculate image feature error ; S3: Based on image feature error , Adaptive neural network compensation is introduced into the robot servo system, and the robot controller is designed as: ; in, represents the camera velocity in the camera coordinate system, represents the estimated value of the image interaction matrix, express The generalized inverse matrix of represents the first weight coefficient, represents the image feature error, represents the estimated value of the change in image features caused by target motion, represents the second weight coefficient, represents the output of the adaptive neural network, represents the input of the adaptive neural network, , Represents the coefficient matrix of the adaptive neural network; S4: Obtain the camera velocity in the camera coordinate system based on the robot controller , depending on the camera speed Calculate the control joint speed of the robot arm and control the robot arm to move according to the control joint speed.

[0006] Preferably, the coefficient matrix of the adaptive neural network is The adaptive rate followed is: ; in, Represents the image feature subscript index, Represents the activation function of an adaptive neural network node.

[0007] Preferably, the estimated value of the image feature change caused by the target motion The calculation method is: ; in, Indicates the current moment, Represents the camera speed in the world coordinate system, Indicates the time length corresponding to each control cycle of the robot arm.

[0008] Preferably, the camera speed in world coordinates The solution is: Get the actual joint speed of the robot at the current moment, and calculate the camera speed in the world coordinate system based on the actual joint speed .

[0009] Preferably, the calculation method for controlling the joint speed is: Based on the camera velocity in the camera coordinate system The robot end velocity is calculated by the transformation operator from the camera coordinate system to the robot end coordinate system. According to the robot end velocity, the control joint velocity of the robot is calculated by the inverse matrix of the Jacobian matrix of the robot.

[0010] Compared with the prior art, the present invention can achieve the following beneficial effects: The embodiment of the present invention adopts a neural network to compensate for the existing robot arm control rate, and effectively improves the visual servo tracking accuracy of the robot arm without increasing the control rate of the robot arm visual servo control system; and the neural network compensation adopted by the present invention is based on the actual joint speed of the robot arm and the image feature error at the previous moment, and adopts an adaptive control method, which can adaptively adjust the weight coefficient of the neural network, and then adaptively adjust the compensation value to achieve real-time compensation, reduce human intervention in the compensation process, and is easy to operate. In addition, the adaptive real-time compensation adjustment can more efficiently improve the tracking accuracy.

[0011] Each time the adaptive neural network performs compensation, it updates the compensation value in real time based on the feedback of the current actual joint speed of the robotic arm and the image feature error at the previous moment, with higher compensation accuracy, which can more efficiently improve the tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of a robot arm visual servo tracking method provided according to an embodiment of the present invention; Figure 2 is an image feature error curve diagram generated by the robot arm tracking the target when the adaptive neural network compensation is not performed according to an embodiment of the present invention; Figure 3 It is an image feature error curve diagram generated by a robot arm tracking a target after adaptive neural network compensation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are represented by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, the detailed description thereof will not be repeated.

[0014] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0015] like Figure 1 As shown, an embodiment of the present invention provides a robot arm visual servo tracking method based on adaptive neural network compensation, which is used to control the robot arm to track a dynamic target. The specific contents of the method include: When the robot arm is tracking a dynamic target, a camera is set at the end of the robot arm, which is fixed on the robot arm and keeps consistent with the movement state of the robot arm. The camera can collect the target image of the dynamic target in real time according to the control instructions.

[0016] S1: The embodiment of the present invention is used Indicates the current moment of the target's motion, Indicates the current position of the target, and the corresponding Indicates the next moment after the current moment. express The target position corresponding to the moment, and so on.

[0017] The camera takes a real-time image of a target at the current moment according to the above-mentioned time frequency, identifies the target in the real-time image and extracts the real-time image features of the image. .

[0018] Image features refer to multiple points on a representative target in the photographed target. For example, the embodiment of the present invention takes a moving square as the target and the four corners of the square as image feature points. Extracting image features based on the image belongs to the prior art.

[0019] S2: For the target, it has an ideal image. The ideal target image can be understood as that when acquiring the ideal image of the target, the target is completely in the camera field of view, and in the ideal image, the target is not deformed. For example, the target in the embodiment of the present invention is a square, and in its ideal image, the target is still a standard square that can be fully displayed. The ideal image is unique and can be pre-collected before the experiment.

[0020] Extract the reference image features of the target based on the ideal image , reference image features is the image feature in the error-free state. Based on the real-time image feature obtained in S1 and the reference image features obtained in S2 Extract image feature error Image feature error The calculation method is an existing mature technology.

[0021] S3: When performing visual servo control of the robot arm, since the visual servo system is a control system with a low control rate and will carry system delay, the image feature error estimation value caused by the target motion is inaccurate, resulting in the problem of low tracking accuracy. In addition, the control rate of the visual servo system is limited by the camera sampling frequency and the complexity of the image processing algorithm, and it is difficult to improve it. Therefore, the embodiment of the present invention is based on the image feature error. , an adaptive neural network compensation is introduced into the control rate of the robot arm servo system to achieve control rate compensation and make up for the error caused by the control rate. The embodiment of the present invention refers to the compensated robot arm controller expression as Formula 1, which is as follows: ; in, represents the camera velocity in the camera coordinate system, represents the estimated value of the image interaction matrix, express The generalized inverse matrix of represents the first weight coefficient, represents the image feature error, represents the estimated value of the change in image features caused by target motion, represents the second weight coefficient, represents the output of the adaptive neural network, represents the input of the adaptive neural network, , Represents the coefficient matrix of the adaptive neural network.

[0022] In the robot controller, That is, neural network compensation, where As the input of the adaptive neural network, it is related to the image feature error Related. The coefficient matrix of the adaptive neural network The adaptive rate followed is: ; in, Represents the image feature subscript index, represents the activation function of the adaptive neural network node. The coefficient matrix The adaptive rate followed is Formula 2.

[0023] The estimated value of the image feature change caused by target motion in Formula 1 The calculation method is: ; in, Indicates the current moment, Represents the camera speed in the world coordinate system, Indicates the time length corresponding to each control cycle of the robot arm. The calculation formula is formula 3. Figure 1 middle It is a delay operator, which can obtain the last moment when it works. Image feature error corresponding to the moment , that is, .

[0024] The joints of the robot arm are equipped with joint sensors. The actual joint speed of the robot arm can be obtained based on the joint sensors, and the camera speed in the world coordinate system can be calculated. When using the Jacobian matrix of the robot , calculate the end velocity of the robot arm, and according to the end velocity of the robot arm, use the operator that transforms the robot arm end coordinate system to the camera coordinate system , calculate the camera speed in the world coordinate system .

[0025] Combined with Figure 1 , solving the camera velocity in the world coordinate system Then, substitute it into Formula 3 to obtain the estimated value of the image feature change caused by the target motion. . Solve the estimated value of the image feature change caused by the target motion go through After the filtering algorithm is processed, it can be substituted into Formula 1 to solve the camera speed in the camera coordinate system Attached Figure 1 The KF in is Filtering algorithm.

[0026] S4: After establishing the robot arm controller, obtain the camera speed in the camera coordinate system based on the robot arm controller , depending on the camera speed , through the transformation operator from the camera coordinate system to the robot end coordinate system Calculate the end velocity of the robot arm, based on the end velocity of the robot arm, through the inverse matrix of the Jacobian matrix of the robot arm Calculate the control joint speed of the robot arm. Finally, control the robot arm to move according to the control joint speed to achieve robot arm control.

[0027] It can be seen from the above controller that its solution is based on the actual joint speed of the robot and the image feature error at the previous moment. Therefore, the controller can realize real-time update iteration, so that the camera speed of the robot is solved. More accurate, which can improve the tracking accuracy of the robotic arm.

[0028] Based on the controller given by formula 1, the embodiment of the present invention uses the Lyapunov function to determine its stability, further ensuring that the stability of the designed robotic arm controller meets the requirements. The selected Lyapunov function is: ; in, , , Represents the matrix Therefore, the above Lyapunov function can also be transformed into: ; Then the time derivative function of the transformed Lyapunov function is: ; Defining symbols ,in, Representing the interaction matrix of the image, the Lyapunov function after the time derivative function can also be expressed as: ; According to the approximation principle of neural network: ; The above formula can be expressed as: ; in, Representation Matrix No. row, according to the adaptive rate , you can get: ; The activation function selected in the embodiment of the present invention is the Gaussian distribution function. According to the definition of the Gaussian distribution function: ; in, and Respectively represent matrices and The minimum and maximum eigenvalues ​​of Representation Matrix 1 norm, so the above formula can be expressed as: ; therefore, ,if and: ; Therefore, the proposed robot controller is locally uniformly ultimately bounded.

[0029] In order to verify the effectiveness of the robot controller, the embodiment of the present invention randomly selects a rectangular trajectory for tracking experimental testing. The test results are as follows: Figure 2 and Figure 3 As shown in the accompanying drawings , They represent the image feature errors in the tracking direction. The experimental results show that the real-time compensation method can improve the tracking accuracy. direction increased by 4 times, and The direction is improved by 8 times, which proves the effectiveness and practicality of the method proposed in this paper.

[0030] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. Those skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0031] The above specific implementations of the present invention do not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A robot arm visual servo tracking method based on adaptive neural network compensation, wherein a camera is provided at the end of the robot arm, and the camera can follow the movement of the robot arm, characterized in that: include: S1: Collecting a real-time image of a target through the camera and extracting real-time image features of the real-time image ; S2: Extract reference image features based on the ideal image of the target , according to the real-time image features and the reference image features Calculate image feature error ; S3: Based on the image feature error , Adaptive neural network compensation is introduced into the robot servo system, and the robot controller is designed as: ； in, represents the camera velocity in the camera coordinate system, represents the estimated value of the image interaction matrix, express The generalized inverse matrix of represents the first weight coefficient, represents the image feature error, represents the estimated value of the change in image features caused by target motion, represents the second weight coefficient, represents the output of the adaptive neural network, represents the input of the adaptive neural network, , Represents the coefficient matrix of the adaptive neural network; S4: According to the robot controller, obtain the camera speed in the camera coordinate system , according to the camera speed The control joint speed of the robot arm is calculated, and the robot arm is controlled to move according to the control joint speed.

2. The method for visual servo tracking of a robotic arm based on adaptive neural network compensation according to claim 1, characterized in that: The coefficient matrix of the adaptive neural network is The adaptive rate followed is: ； in, Represents the image feature subscript index, Represents the activation function of an adaptive neural network node.

3. The robot arm visual servo tracking method based on adaptive neural network compensation as claimed in claim 1, characterized in that: The estimated value of the image feature change caused by the target motion The calculation method is: ； in, Indicates the current moment, Represents the camera speed in the world coordinate system, Indicates the time length corresponding to each control cycle of the robot arm.

4. The method for visual servo tracking of a robot arm based on adaptive neural network compensation as claimed in claim 3, characterized in that: The camera speed in the world coordinate system The solution is: Get the actual joint speed of the robot at the current moment, and calculate the camera speed in the world coordinate system based on the actual joint speed .

5. The method for visual servo tracking of a robot arm based on adaptive neural network compensation according to claim 1, characterized in that: The calculation method of controlling the joint speed is: According to the camera velocity in the camera coordinate system , the robot end velocity is calculated by a transformation operator from a camera coordinate system to a robot end coordinate system, and based on the robot end velocity, the control joint velocity of the robot arm is calculated by an inverse matrix of the Jacobian matrix of the robot arm.

Images