Industrial robot joint friction parameter compensation method based on machine vision

Abstract

The invention discloses an industrial robot joint friction parameter compensation method based on machine vision. The three-dimensional pose information of the target feature points is extracted based on the image sequence, the tail end position and the pose change are accurately captured, the actual driving force is obtained, and a dynamic comparison mechanism between the theoretical driving force and the actual driving force is established, so that the state deviation is extracted, the load torque and the friction torque are separated out, and the driving precision is improved. Compared with a traditional control strategy depending on a fixed friction model coefficient, the dynamic friction compensation torque is obtained in a real-time image obtaining mode, non-linear friction interference changing along with time, temperature and abrasion can be tracked and compensated in a self-adaptive mode, and the method has the advantages that the dynamic friction compensation torque is obtained in a real-time image obtaining mode. The problems that an existing model lags behind and is large in compensation error are solved, and the friction estimation precision and the compensation capacity are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control, and in particular to a compensation method for joint friction parameters of an industrial robot based on machine vision. BACKGROUND

[0002] With the development of industrial automation and intelligent manufacturing, industrial robots are widely used in precise assembly, micro-operation, force control cooperation and other high-precision tasks. In order to achieve accurate control of the end effector, the robot control system usually plans trajectories and controls joint driving based on kinematics and dynamics models. However, in actual operation, there are generally nonlinear friction factors in each joint of the robot, such as static friction, dynamic friction, viscous friction, etc. These factors significantly interfere with the system performance, especially in low-speed, small-amplitude, force control and other working conditions. SUMMARY

[0003] The present application overcomes the shortcomings of the prior art and provides a compensation method for joint friction parameters of an industrial robot based on machine vision.

[0004] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: a compensation method for joint friction parameters of an industrial robot based on machine vision, comprising the following steps:

[0005] Receiving a robot end image frame collected by an imaging device at a current orientation angle to obtain three-dimensional pose data of the end at the corresponding time;

[0006] Based on the three-dimensional pose data of the end, the actual motion state of each target joint is calculated;

[0007] The preset motion state is compared with the actual motion state to generate a state deviation between the theoretical driving force and the actual driving force of each target joint;

[0008] Based on the state deviation, a dynamic compensation signal is generated and superimposed on the control command to generate an output control signal after compensation.

[0009] In a preferred embodiment of the present application, the timestamp sequence of the image frame is [t1, t2, …, tn], and an image is collected for each timestamp to construct an image set.

[0010] In a preferred embodiment of the present application, the acquisition of the three-dimensional pose data comprises:

[0011] The image set is preprocessed to obtain the feature point positions in the image, the feature point pairs of consecutive two image frames are compared to obtain the position difference of the same feature point, and the three-dimensional coordinates of the feature point in the actual space are calculated;

[0012] According to the change of the same feature point in consecutive two image frames, the attitude of the feature point is obtained;

[0013] The feature point position and the posture are combined to form three-dimensional pose data.

[0014] In a preferred embodiment of the present application, the feature point is a robot end position.

[0015] In a preferred embodiment of the present application, the motion state includes: angle data, angular velocity data and angular acceleration data.

[0016] In a preferred embodiment of the present application, the acquisition of the actual motion state includes:

[0017] The three-dimensional pose of the end is taken as an input quantity to calculate the angle value of the current target joint;

[0018] According to the joint angle corresponding to the two frames of images acquired before and after And The angular velocity and the angular acceleration of each joint are obtained by using the difference approximation method.

[0019] In a preferred embodiment of the present application, the generation of the state deviation of the theoretical data and the actual data of each target joint includes:

[0020] The angle data, the angular velocity data and the angular acceleration data are taken as input quantities to obtain the actual driving force, and the state deviation is obtained by comparing with the theoretical driving force Wherein represents the actual driving force, represents the theoretical driving force;

[0021] If is approximately equal to 0, it indicates that the actual motion is close to the ideal situation; if is greater than 0, it indicates that the actual driving force is greater than the theoretical driving force, and there is a friction torque.

[0022] In a preferred embodiment of the present application, the state deviation is a friction compensation torque.

[0023] In a preferred embodiment of the present application, the compensation signal is superimposed on the control instruction in the following manner: the friction compensation torque and the theoretical driving force are superimposed and synthesized in the instruction level every period, and the period range is 1ms-5ms.

[0024] The present application solves the defects in the background art, and has the following beneficial effects:

[0025] (1) The application provides a kind of compensation method of industrial robot joint friction parameter based on machine vision, based on image sequence extraction target feature point three-dimensional pose information, accurate capture end position and attitude change, obtain actual driving force, and establish the dynamic comparison mechanism between theoretical driving force and actual driving force, to extract state deviation, and then separate load torque and friction torque, realize the online modeling and estimation of friction influence, compared with the control strategy of traditional dependence fixed friction model coefficient, the dynamic friction compensation torque of the present application is obtained by real-time image acquisition, which can adaptively track and compensate the nonlinear friction disturbance varying with time, temperature and wear, solve the problem of existing model lag, large compensation error, effectively improve the friction estimation precision and compensation ability.

[0026] (2) The application provides a kind of compensation method of industrial robot joint friction parameter based on machine vision, by separating the friction torque and load torque in the total driving force of robot joint in real time, the feature makes the control system can identify and extract the dynamic disturbance component caused by nonlinear friction effect, so as to realize accurate friction modeling and compensation, make friction compensation have pertinence and real-time, avoid the problem of control over compensation or response distortion caused by mixing friction and load, compared with the way of relying on offline friction modeling or adjusting total error in prior art, it is difficult to deal with nonlinear fluctuation caused by friction due to temperature rise, lubrication degradation or wear change, the present application dynamically adjusts friction compensation torque by real-time acquisition of image of robot target joint position, so that friction compensation signal can be dynamically and periodically superimposed into control instruction, form adaptive control path automatically corrected with running state, effectively improve the stability and response consistency of robot in low speed section, micro-motion operation and high precision task.

[0027] (3) The application provides a kind of compensation method of industrial robot joint friction parameter based on machine vision, by superimposing the friction compensation signal generated in real time into the original control instruction in periodic way, so that the control instruction can quickly respond to the real-time change of friction disturbance, and can continuously correct nonlinear friction effect in each control period, so as to significantly improve the control precision, eliminate the trajectory deviation and oscillation phenomenon caused by compensation lag or discontinuity in traditional compensation mode, ensure the stability and tracking accuracy in low speed movement process, compared with the existing technology, the compensation amount is updated in static or low frequency mode, which cannot capture the instantaneous change of friction characteristics in time, resulting in insufficient or excessive compensation, the present application effectively avoids the performance degradation caused by compensation delay and model mismatch.

[0028] (4) The application provides a compensation method for joint friction parameters of an industrial robot based on machine vision, three-dimensional pose information of an end effector is extracted through a vision image sequence, space-time comparison is performed on feature points of continuous image frames, a kinematics error accumulation problem caused by temperature drift and mechanical clearance of a traditional joint encoder is directly avoided, an end positioning drift problem caused by nonlinear characteristics of joint friction under complex working conditions is solved, and the robot has stronger stability and flexibility under a dynamic scene, compared with the prior art, has stronger deployment flexibility and lower sensor cost, and is convenient for direct upgrading and integration in an existing robot structure. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0030] Figure 1 It is a flowchart of a compensation method for joint friction parameters of an industrial robot based on machine vision. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0032] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0033] In the description of the present application, it needs to be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the scope of protection of the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0034] In the description of the present application, it needs to be understood that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0035] SUMMARY

[0036] Regarding the nonlinear friction interference such as static friction, dynamic friction, viscous effect and Stribeck effect commonly existing inside the joint, the "creep", "oscillation", "positioning drift" and other phenomena will be caused at low speed or starting stage, which seriously affects the trajectory tracking performance.

[0037] The prior art usually takes two compensation ideas: one is to install a torque sensor or an external force / torque sensor to directly measure the friction and perform real-time compensation, but this way has high hardware cost, complex structure and is difficult to deploy in lightweight or space-limited occasions;

[0038] The second is to rely on offline calibration to obtain fixed friction parameters, and to compensate statically through feedforward or feedback, but since the friction characteristics will change dynamically with temperature, lubrication state and mechanism wear, the static model is difficult to accurately reflect the real friction in time, resulting in gradual invalidation of compensation accuracy.

[0039] In recent years, the maturity of high frame rate machine vision technology makes it possible to obtain the end three-dimensional pose through image sequence, but the existing visual positioning researches are mostly focused on trajectory detection and obstacle avoidance, and the scheme of how to fuse visual perception with friction online modeling and adaptive compensation is not perfect, and there is a lack of complete solution path for joint nonlinear friction interference.

[0040] Therefore, the application provides a compensation method for joint friction parameters of an industrial robot based on machine vision to solve the above problems.

[0041] As shown in Figure 1 A compensation method for joint friction parameters of an industrial robot based on machine vision, comprising the following steps:

[0042] Receiving the robot end image frame collected by the imaging device at the current orientation angle to obtain the end three-dimensional pose data at the corresponding moment;

[0043] Based on the three-dimensional pose data of the end, the actual motion state of each target joint is calculated;

[0044] The preset motion state is compared with the actual motion state, and the state deviation of the theoretical driving force and the actual driving force of each target joint is generated;

[0045] Based on the state deviation, a dynamic compensation signal is generated and superimposed to the control instruction to generate an output control signal after compensation.

[0046] In the application, the timestamp sequence of the image frame is [t1, t2, …, tn], and the image of each timestamp is collected to construct an image set.

[0047] Trigger the camera to collect at a preset frame rate (such as 100Hz), and automatically stamp the current image frame with a unique timestamp tn at each collection, and save the image frame with timestamp in the ring buffer or high-speed storage array according to the time sequence to form a set of continuous image set {I(t1), I(t2), …, I(tn)};

[0048] In the subsequent processing process, by strictly ensuring the front and rear order of the image frame, the calculation error of speed and acceleration caused by frame loss or disorder is avoided, and all image-based calculations can be accurately corresponded to the same physical moment, thereby providing a reliable time reference for the time sequence solution of joint motion state.

[0049] Through accurate timing management and inter-frame alignment of the terminal image frames, the numerical accuracy in subsequent differential calculation speed and acceleration is ensured, and the precision of inverse kinematics back solution and friction observation is directly improved; compared with the prior art without time stamp or loose frame rate control, the scheme avoids the joint state estimation drift problem caused by image frame disorder or acquisition delay, further ensures the real-time and reliability of the friction compensation signal, and significantly enhances the adaptive compensation capability of the robot to nonlinear friction disturbance under variable speed, jitter or vibration working conditions.

[0050] In the application, the acquisition of the three-dimensional pose data comprises:

[0051] The image set is preprocessed to obtain the feature point positions in the images, the feature point pairs of the two consecutive images are compared to obtain the position difference values of the same feature points, and the three-dimensional coordinates of the feature points in the actual space are calculated;

[0052] The pose of the feature points is obtained according to the changes of the same feature points in the two consecutive images;

[0053] The feature point positions and poses are combined to form the three-dimensional pose data.

[0054] Specifically, the image set continuously collected is preprocessed by performing gray processing, filtering and denoising and contrast enhancement on each image to improve the reliability of subsequent feature detection;

[0055] In each preprocessed image, the two-dimensional pixel coordinate set of the feature points is located by a feature detection algorithm, wherein the feature points refer to the robot terminal; the same feature point pairs in the two adjacent images are matched, the displacements of the feature points in the pixel plane are calculated, the pixel displacements are mapped to the coordinate changes of the corresponding feature points in the three-dimensional space by using the camera intrinsic parameters K and binocular stereo disparity or PnP algorithm, and the three-dimensional coordinates of each feature point at the corresponding time are obtained;

[0056] The rotation matrix of the terminal is calculated by rigid registration of the three-dimensional coordinate sets of the same group of feature points in the consecutive frames, so that the spatial positions and overall orientations of the points are combined to form the final six-degree-of-freedom three-dimensional pose.

[0057] It should be noted that through the three-dimensional pose information of the feature points, the system can accurately reconstruct the real-time three-dimensional position and pose of the robot terminal only by visual data without any contact type sensor, avoiding the cumulative error caused by mechanical gap or sensor drift of the traditional encoder; the accuracy and stability of the end pose measurement are improved, and high-fidelity input is provided for subsequent inverse kinematics calculation and friction parameter separation;

[0058] Meanwhile, by continuous frame difference and feature tracking, the influence of visual interference such as light change, occlusion and jitter on pose estimation is effectively inhibited.

[0059] In the application, the motion state comprises: angle data, angular velocity data and angular acceleration data.

[0060] The angle data represents the spatial position of each joint at a certain time; it is the most basic data for describing the robot pose and the end execution point trajectory; it corresponds to the output of inverse kinematics and is the basis for judging whether the end motion reaches the target.

[0061] The angular velocity data represents the change speed of the angle per unit time, reflecting the speed of the motion; it is particularly important for friction modeling, because most friction models are highly sensitive to speed, and the static friction is significant when the speed is close to 0.

[0062] The angular acceleration data represents the change rate of the angular velocity per unit time, revealing the inertia and dynamic response characteristics of the motion.

[0063] In the application, the acquisition of the actual motion state comprises:

[0064] The three-dimensional pose of the end is taken as the input quantity to calculate the angle value of the current target joint;

[0065] According to the joint angles corresponding to the two frames of images acquired before and after And , the angular velocity and angular acceleration of each joint are obtained by using the difference approximation method.

[0066] Specifically, the three-dimensional pose data of the feature point at time t is represented as: Where xt is the three-dimensional position at time t, and Rt is the attitude rotation matrix at time t (such as a 3x3 rotation matrix);

[0067] Angle calculation: input Pt into the known robot inverse kinematics function to obtain all target joint angles of the t-th frame ;

[0068] The calculation of the angular velocity uses the difference method: the difference between the angle values of the consecutive frames t and t-1 is divided by the inter-frame time interval Δt to obtain the angular velocity ;

[0069] The calculation of the angular acceleration also uses the difference method: .

[0070] Wherein, the acquisition of the robot inverse kinematics function comprises:

[0071] For each joint, the DH parameters are established: link length, twist angle, offset, joint angle;

[0072] The transformation matrix Ti of each section is multiplied to obtain the total transformation of the end:

[0073] Input end transformation , inverse solution .

[0074] In the present application, the state deviation of the theoretical data and the actual data of each target joint is generated, including:

[0075] The angle data, angular velocity data and angular acceleration data are taken as input quantities to obtain the actual driving force, and the state deviation is obtained by comparing the actual driving force with the theoretical driving force , wherein represents the actual driving force, represents the theoretical driving force;

[0076] If is approximately equal to 0, it indicates that the actual movement is close to the ideal situation; if is greater than 0, it indicates that the actual driving force is greater than the theoretical driving force, and there is a friction torque;

[0077] The state deviation is the friction compensation torque.

[0078] Specifically, the target joint angle, angular velocity and angular acceleration obtained are taken as inputs: , , ;

[0079] Based on the inverse dynamics model of the robot (such as based on Euler-Lagrange or Newton-Euler method), the above motion state is input to calculate the actual driving force acting on the joint: , wherein represents the joint mass-inertia matrix, represents the Coriolis force / centrifugal force term, represents the gravity term.

[0080] The actual driving force is compared with the theoretical driving force to obtain a difference value, which is the state deviation, i.e. the friction torque.

[0081] It should be noted that by capturing the changes in the end position and attitude, the actual driving force is obtained, and a dynamic comparison mechanism between the theoretical driving force and the actual driving force is established, so as to extract the state deviation, and then separate the load torque and the friction torque, realize the online modeling and estimation of the friction influence, compared with the traditional control strategy relying on fixed friction model coefficient, the present application obtains dynamic friction compensation torque by real-time image acquisition, can adaptively track and compensate the nonlinear friction interference changing with time, temperature and wear, solves the problems of existing model lag and large compensation error, effectively improves the friction estimation precision and compensation ability.

[0082] In the present application, the compensation signal is superimposed on the control command in the following way: the friction compensation torque and the theoretical driving force are superimposed and synthesized at the instruction level in a periodic manner, wherein the period ranges from 1 ms to 5 ms.

[0083] By superimposing the real-time generated friction compensation signal on the original control command in a periodic manner, the control command can quickly respond to the real-time changes of the friction disturbance, and the nonlinear friction effect can be continuously corrected in each control period, thereby significantly improving the control accuracy, eliminating the trajectory deviation and oscillation phenomenon caused by compensation lag or discontinuity in the traditional compensation method, and ensuring the smoothness and tracking accuracy during low-speed movement. Compared with the prior art, the compensation amount is updated in a static or low-frequency manner, which cannot capture the instantaneous changes of the friction characteristics in time, resulting in insufficient or excessive compensation. The present application effectively avoids the performance degradation caused by compensation delay and model mismatch.

[0084] Based on the ideal embodiments of the present application, the above description can be varied and modified without deviating from the technical concept of the present application. The technical scope of the present application is not limited to the contents of the specification, and must be determined by the scope of the claims.

Claims

1. A method for compensating for friction parameters of joints of an industrial robot based on machine vision, characterized in that, The method comprises the following steps: receiving a robot end image frame collected by an imaging device at a current orientation angle to obtain end three-dimensional pose data at a corresponding time; calculating actual motion states of each target joint based on the three-dimensional pose data of the end; calling a preset motion state and actual motion state comparison to generate state deviations of theoretical driving forces and actual driving forces of each target joint; generating a dynamic compensation signal based on the state deviations and superimposing the compensation signal on a control instruction to generate an output control signal after compensation.

2. The method of claim 1, wherein the method further comprises: The time stamp sequence of the image frame is [t1, t2, …, tn], and an image is collected for each time stamp to construct an image set.

3. The method of claim 1, wherein the method further comprises: The three-dimensional pose data acquisition comprises: preprocessing the image set to obtain feature point positions in the image, comparing feature point pairs of two consecutive images to obtain position differences of the same feature points, and calculating three-dimensional coordinates of the feature points in the actual space; obtaining the attitude of the feature point according to the change of the same feature point in the two consecutive images; combining the feature point position and the attitude to form the three-dimensional pose data.

4. The method of claim 3, wherein the method further comprises: The feature point is the position of the robot end.

5. The method of claim 1, wherein: The motion state comprises angle data, angular velocity data, and angular acceleration data.

6. The method of claim 5, wherein: The actual motion state acquisition comprises: taking the three-dimensional pose of the end as an input quantity to calculate the angle value of the current target joint; According to the joint angles corresponding to the two images before and after acquisition and , the angular velocity and angular acceleration of each joint are obtained by using the difference approximation method.

7. The method of claim 6, wherein the method further comprises: the state deviation between the theoretical data and the actual data of each target joint comprises: The angle data, angular velocity data and angular acceleration data are taken as input quantities to obtain the actual driving force, and the state deviation is obtained by comparing the actual driving force with the theoretical driving force wherein represents the actual driving force, represents the theoretical driving force; If approximately equal to 0, it means that the actual motion is close to the ideal case; if greater than 0, it means that the actual driving force is greater than the theoretical driving force, and there is a friction torque.

8. The method of claim 7, wherein the method further comprises: The state deviation is a friction compensation torque.

9. The method of claim 1, wherein: The compensation signal is superimposed on the control instruction in the following manner: the friction compensation torque and the theoretical driving force are superimposed and synthesized at the instruction level to generate an output, and the period range is 1 ms to 5 ms.

Images