A binocular vision-based rocket tank detection dual-robot path planning method

Abstract

The application discloses a double-robot path planning method for rocket tank weld detection based on binocular vision, which comprises the following steps: firstly, a measuring instrument and a weighted moving average filter are used to calibrate the kinematic parameters of a robot; then, a Monte Carlo method is used to solve the double-robot collaborative space, so as to determine the range of the double-robot collaborative working interval and the rocket tank products that can be detected; the running area of each joint is determined by using the collaborative working space, the running range of each joint is reset, the double-vision technology is used to reduce the clamping error of the rocket tank to be detected and identify the model of the rocket tank to be detected; in addition, a polynomial interpolation method is used to interpolate and plan the double-robot path, so as to automatically generate the rocket tank path trajectory and the trajectory number corresponding to the model; finally, the communication between the robot control system and the detection software is established, so as to realize the automatic calling of the rocket tank detection path trajectory and the automatic process detection.

Description

TECHNICAL FIELD

[0001] The application relates to a dual-robot cooperative path planning method, in particular to a dual-robot path planning method for rocket tank weld detection based on binocular vision. BACKGROUND

[0002] Rocket tank detection currently adopts a ray detection technology, and the welds of the detected products are relatively complex, so a dual-robot is used as a motion mechanism, the dual-robot is individually taught by a teach pendant, and the dual-robot is cooperatively operated through signal interaction of the two robots. The dual-robot path planning mode using the teach pendant has low efficiency, and collision and scratches are easily caused to the detected products during the teaching process. Each detected product needs to be individually taught.

[0003] In the prior art, a ray digital imaging detection device and a detection method based on multi-robot cooperation are adopted, three-dimensional modeling software and a three-dimensional model of a detected object are used to generate a robot cooperative path trajectory based on offline programming software. This method ignores the kinematic error of the robot, the installation error of the detected product and the error between the three-dimensional detected product and the actual product. In addition, the size of each detected product after processing cannot be guaranteed to be absolutely consistent, and there are various differences. In addition, the ray detection is not continuous during the detection process, and the detection method is based on continuous detection of each point. Therefore, if the path planning of the robot does not consider these problems, there will be a large detection error, and the parts are easily scratched, which may even cause life danger.

[0004] Patent document CN 111360812 B (application number: 201811597859.X) discloses a DH parameter calibration method and device for an industrial robot based on camera vision. The camera is used to obtain the projection point position of the laser beam emitted from the flange at the end of the robot on the plane and the corresponding robot joint angle. A nonlinear optimization method is used to solve the DH parameter error of the robot, and an accurate DH parameter model is obtained. The purpose of improving the absolute positioning accuracy of the mechanical arm is achieved. In this method, the position coordinates collected by the camera are used as the calibration reference in the calibration process using the camera. However, the image distortion exists in the camera during the collection process, which leads to a large error in the collected position coordinates, and cannot meet the requirement of accurate robot DH parameter error compensation.

[0005] In the paper "Research on Key Technologies of Sorting System Based on Binocular Vision", two horizontally placed cameras are used to complete the clamping and classification of different small-sized workpieces. Compared with the paper, the application establishes two mutually perpendicular vision acquisition devices, and uses the calibration method of eyes outside the hand. Not only the model recognition of large-sized detected products is realized, but also the weld position recognition of the detected products is realized, which meets the requirements of automatic recognition and weld position acquisition for automatic detection of large-sized products. SUMMARY

[0006] The technical problem solved by the present application is that based on the above technical deficiencies, a dual-robot path planning method for rocket tank detection based on binocular vision is provided, which uses measuring instruments and neural network algorithms to solve robot errors and improve detection accuracy and efficiency.

[0007] The technical solution of the present application is a dual-robot path planning method for rocket tank detection based on binocular vision, comprising:

[0008] Based on the D-H method, a robot kinematics model is established, and measuring instruments and weighted moving average filtering are used to compensate and calibrate the kinematics parameters in the robot kinematics model.

[0009] According to the obtained compensated and calibrated robot kinematics model, the Monte Carlo method is used to solve the dual-robot collaborative space to determine the dual-robot collaborative working range, so as to determine the detectable rocket tank model, and the region of each joint is determined by using the collaborative working space, and the operation range of each joint is reset;

[0010] The position of the binocular vision acquisition device is calibrated by using measuring instruments, and the coordinate transformation between binocular vision and dual-robot is established;

[0011] According to the obtained coordinate transformation between binocular vision and robot, the position collection of the feature points of the rocket tank to be detected is completed, and the model recognition of the rocket tank is completed according to the binocular vision detection;

[0012] According to the obtained robot kinematics model, the collaborative path planning is carried out, and the program number corresponding to the rocket tank model is generated;

[0013] The communication between the robot control system and the detection software is established to realize the automatic calling of the corresponding rocket tank detection path and realize the automatic process detection.

[0014] The robot kinematics model is established based on the D-H method, and the kinematics parameters in the robot kinematics model are compensated and calibrated by using measuring instruments and weighted moving average filtering, comprising:

[0015] The kinematics parameters of the robot are defined, and the robot kinematics model is established based on the D-H method;

[0016] According to the determined kinematics parameters, the homogeneous transformation relationship between adjacent joint coordinate systems is represented by rotation and displacement, and the robot end pose conversion matrix is obtained;

[0017] The weighted moving average filtering method is used to change the expected weight on the basis of moving average filtering, and the expected weight value is assigned to the robot end position error value collected at different times. The weight of the current time data is assigned to the maximum value, and the weight value is gradually decreased in the other direction of the cache area. The robot end position error value is the difference between the theoretical robot end position parameter obtained by the end position conversion matrix and the actual robot end position parameter obtained by the measuring instrument. The robot kinematics parameters are calibrated by using the obtained robot end position error value.

[0018] According to the obtained robot end position error value, the inverse kinematics is solved by using the algebraic solving method, the variables are separated one by one, and the kinematics parameter error value of each joint of the robot is obtained. The kinematics model of the robot established based on the D-H method is compensated by using the obtained kinematics parameter error value of each joint of the robot, and the equation is obtained by sequentially left multiplying the matrix to simplify the equation as follows:

[0019] [ 0 T1] -1 0 T7= 1 T2 2 T3 3 T4 4 T5 5 T6 6 T7

[0020] [ 1 T2] -1 [ 0 T1] -1 0 T6= 2 T3 3 T4 4 T5 5 T6。

[0021] The kinematics parameters include joint rotation angle θ i , connecting rod offset distance d i , connecting rod length a i , and the included angle α i between joint shafts.

[0022] The robot end position conversion matrix is:

[0023]

[0024]

[0025] i T i+2 = i T i+1 · i+1 Ti+2 ;

[0026] wherein i T i+1 represents a pose conversion matrix from the coordinate system of the i th joint to the i+1 coordinate system;

[0027] The position calibration of the binocular vision acquisition device by the measuring instrument and the establishment of the coordinate conversion between the binocular vision and the double robots include:

[0028] The binocular vision acquisition device is applied, and an eye-to-hand mode is adopted, one vision sensor is fixed on the upper end of the object to be measured, and the other vision sensor is fixed on the right end of the object to be measured, the binocular vision acquisition device is corrected and position calibrated by the measuring instrument, after the correction and calibration are completed, the spatial image is collected by the binocular vision acquisition device;

[0029] According to the obtained spatial image and the robot, the coordinate conversion is established, and the relationship between the pixel coordinates in the image and the specified robot coordinates (u, v) and (X W ,Y W ,Z W ) is represented by the following formula:

[0030]

[0031] Wherein, (x, y) and (X C ,Y C ,Z C ) are used to represent image coordinates, camera coordinates; the coordinates of any point P in space from the robot coordinates (X W ,Y W ,Z W ) to the image pixel coordinates (u, v), f x =f / d x , f y =f / d y represent the scaling size of the focal length f in the direction of the two axes of the pixel coordinates; M in is composed of the internal parameters after camera calibration; M out is composed of the rotation matrix R and the translation matrix t of the external parameters after camera calibration; P w represents a point in space.

[0032] According to the obtained coordinate conversion between the binocular vision and the robot, the position collection of the feature points of the rocket tank to be measured is completed, and the model recognition of the rocket tank is completed according to the binocular vision detection, including:

[0033] A mark point is added at the starting point of the detection of the weld of the rocket tank to be measured;

[0034] The binocular vision acquisition device transmits the acquired mark point position parameters to the robot control system to perform position error compensation and correction.

[0035] An image recognition library is established according to the size and structure of the rocket tank, and the tank model is identified by comparing the images acquired by vision acquisition.

[0036] According to the obtained robot kinematics model, cooperative path planning is performed to generate a program number corresponding to the rocket tank model, including:

[0037] According to the rocket tank model, the path of the master robot is planned by interpolation using a quintic polynomial to obtain a quintic polynomial path and first and second derivatives;

[0038] The trajectory constraint conditions of the dual robots are set to establish a tight coordination master-slave following, and the running path of the slave robot is automatically generated according to the path of the master robot;

[0039] The running path of the dual robots is saved to generate a robot program number corresponding to the rocket tank model.

[0040] According to the rocket tank model, the path of the master robot is planned by interpolation using a quintic polynomial to obtain a quintic polynomial path and first and second derivatives, including:

[0041] θ(t)=a0+a1t+a2t 2 +a3t 3 +a4t 4 +a5t 5

[0042]

[0043]

[0044] where θ(t), are the joint angle, joint angular velocity and joint angular acceleration at time t, respectively, a0, a1, a2, a3, a4 and a5 are undetermined coefficients of the quintic polynomial, and the following can be obtained by giving the constraint conditions on the angle and acceleration of the starting point and target point:

[0045] a0=θ0

[0046]

[0047]

[0048]

[0049]

[0050]

[0051] θ0, θt respectively represent the joint angle at the starting point and the joint angle at the target point, f θ0, θt respectively represent the joint angle at the starting point and the joint angle at the target point, θ0, θt respectively represent the joint angle at the starting point and the joint angle at the target point, θ0, θt respectively represent the joint angle at the starting point and the joint angle at the target point, f t represents the time from the starting point to the target point.

[0052] Compared with the prior art, the present application has the following advantages and positive effects:

[0053] 1. In the present application, the installation of double robots, the geometric error of the robot caused by machining, and the influence of the end load on the deformation of the robot end force are considered at the same time, so that the final robot kinematics calibration accuracy is higher.

[0054] 2. The polynomial interpolation method is adopted, so that the robot path running time is shorter, and the running is stable, there is no large vibration, and the influence of robot running on the detection result can be reduced;

[0055] 3. The binocular vision technology is adopted, which can reduce the error of the measured parts assembly and the influence of the differences between the sizes of the measured products,

[0056] 4. The visual recognition technology is adopted, which can automatically identify the corresponding measured product model and automatically call the corresponding robot path trajectory number according to the established control system. Reduce the operation error of the operator. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The flowchart of the technical scheme of the present application.

[0058] Figure 2 The Monte Carlo method algorithm step graph (a) and the cooperative space graph (b) of the present application.

[0059] Figure 3 The principle diagram of the spatial position parameter collection of the binocular vision of the present application.

[0060] Figure 4 The process diagram of the binocular vision feature point position parameter collection and the box type identification of the present application.

[0061] Figure 5 The communication diagram of the robot control system and the detection software of the present application.

[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described and discussed below with reference to the accompanying drawings. Obviously, what is described here is only a part of the examples of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0063] like Figure 1 As shown in the flowchart of the technical solution of the present invention, it includes the following steps:

[0064] (1) A robot kinematic model is established based on the DH method, and the kinematic parameters in the robot kinematic model are compensated and calibrated by measuring instruments and weighted moving average filtering.

[0065] (2) Based on the obtained compensated and calibrated robot kinematic model, the Monte Carlo method is used to solve the dual-robot cooperative space, determine the range of the dual-robot cooperative working area, thereby determining the detectable rocket tank model, using the cooperative working space to determine the area of ​​each joint, and resetting the operating range of each joint point.

[0066] (3) Use measuring instruments to calibrate the position of the binocular vision acquisition device and establish the coordinate transformation between binocular vision and dual robots;

[0067] (4) Based on the coordinate transformation between the binocular vision and the robot, the position acquisition of the feature points of the rocket propellant tank under test is completed, and the model identification of the rocket propellant tank is completed based on the binocular vision detection.

[0068] (5) Based on the obtained robot kinematics model, a polynomial interpolation algorithm is used to perform cooperative path planning and generate the program number corresponding to the rocket tank model.

[0069] (6) Establish communication between the robot control system and the detection software to realize automatic calling of the corresponding rocket tank detection path and realize automated process detection.

[0070] Step (1) further includes the following steps:

[0071] (1.1) Define the robot's kinematic parameters and establish the robot's kinematic model based on the DH method;

[0072] (1.2) Based on the determined kinematic parameters, the homogeneous transformation relationship between adjacent joint coordinate systems is represented by rotation and displacement to obtain the robot end-effector pose transformation matrix;

[0073] (1.3) using a weighted moving average filtering method, on the basis of moving average filtering, the expected weight is changed, the expected weight value is assigned to the robot end position error value collected at different times, the weight of the current time data is assigned to the maximum value and the weight value is gradually decreased to the other direction of the cache area, wherein the robot end position error value is the difference between the theoretical robot end position parameter obtained by the end position conversion matrix and the actual robot end position parameter obtained by the measuring instrument, and the robot kinematics parameters are calibrated by using the obtained robot end position error value;

[0074] (1.4) according to the obtained robot end position error value, solving by robot inverse kinematics, wherein algebraic solving method is used to complete inverse kinematics solving, separating variables one by one, obtaining the kinematics parameter error value of each joint of the robot; then the kinematics model of the robot based on D-H method is compensated by using the obtained kinematics parameter error value of each joint of the robot, and the equation is obtained by matrix left multiplication in sequence as follows:

[0075]

[0076] According to the double robot path planning method for rocket tank detection based on binocular vision of step (1.1), characterized in that, the kinematics parameters include joint rotation angle θ i , link offset distance d i , link length a i , and the angle between joint axes α i .

[0077] According to the double robot path planning method for rocket tank detection based on binocular vision of step (1.2), characterized in that, the robot end position conversion matrix is:

[0078]

[0079] i T i+2 = i T i+1 · i+1 T i+2 ;

[0080] Wherein i T i+1 represents the position conversion matrix from the coordinate system of the i-th joint to the i+1 coordinate system;

[0081] The step (2) further comprises the following steps:

[0082] (2.1) determining the kinematics parameters of the double robot after calibration and the running range of each joint;

[0083] (2.2) using Monte Carlo method to solve the collaborative space;

[0084] (2.3) resetting the operating range of each joint of the dual robots according to the collaborative workspace;

[0085] The step (3) further comprises the following steps:

[0086] (3.1) using binocular vision acquisition device, adopting eye-to-hand mode, one vision sensor is fixed on the upper end of the object to be measured, and the other vision sensor is fixed on the right end of the object to be measured, the binocular vision acquisition device is corrected and positionally calibrated through a measuring instrument, after the correction and calibration are completed, the binocular vision acquisition device is used to collect spatial images;

[0087] (3.2) establishing coordinate conversion between the obtained spatial images and the robot, the relationship between the pixel coordinates in the images and the specified robot coordinates (u, v) and (X W ,Y W ,Z W ) is expressed by the following formula:

[0088]

[0089] Wherein, (x, y) and (X C ,Y C ,Z C ) are used to represent image coordinates, camera coordinates; the coordinates of a point P in space from robot coordinates (X W ,Y W ,Z W ) to image pixel coordinates (u, v), f x =f / d x , f y =f / d y represent the scaling size of focal length f in the direction of two axes of pixel coordinates; M in is composed of internal parameters after camera calibration; M out is composed of rotation matrix R and translation matrix t of external parameters after camera calibration; P w represents a point in space.

[0090] The step (4) further comprises the following steps:

[0091] (4.1) adding a mark point at the starting point of the detection of the rocket tank weld;

[0092] (4.2) transmitting the position parameters of the collected mark point to the robot control system through the binocular vision acquisition device, and performing position error compensation and correction;

[0093] (4.3) According to the size and structure of the rocket tank, an image recognition library is established, and the image collected by vision is compared to complete the identification of the tank model.

[0094] The step (5) further comprises the following steps:

[0095] (5.1) According to the rocket tank model, the path of the main robot is interpolated and planned by a quintic polynomial, and the quintic polynomial path and the first and second derivative formulas are as follows:

[0096] θ (t) = a0+a1t+a2t 2 +a3t 3 +a4t 4 +a5t 5

[0097]

[0098]

[0099] Where θ (t), respectively, the joint angle, joint angular velocity and joint angular acceleration at time t, a0, a1, a2, a3, a4, a5 are the undetermined coefficients of the quintic polynomial, which can be obtained by giving the constraints on the angle and acceleration of the starting point and target point as follows:

[0100] a0=θ0

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] Where θ0, θ f respectively represent the joint angle at the starting point and the joint angle at the target point, respectively represent the joint angular velocity at the starting point and the joint angular velocity at the target point, respectively represent the joint angular acceleration at the starting point and the joint angular velocity at the target point, t f represents the time from the starting point to the target point;

[0107] (5.2) Set the trajectory constraint condition of the double robot, and establish a tight coordination of master-slave following. According to the path of the main robot, the running path of the slave robot is automatically generated;

[0108] (5.3) Save the double robot running path, and generate the robot program number corresponding to the rocket tank model

[0109] The step (6) further comprises the following steps:

[0110] (6.1) The PROFINET bus communication mode commonly used by the robot system is adopted to build the S7 protocol, and information interaction with the detection software is realized;

[0111] (6.2) According to the experimental test, the double robot path planning method based on binocular vision for rocket tank detection can make the double robot achieve higher trajectory accuracy and positioning accuracy, reduce the influence of rocket tank assembly error on detection, improve the detection accuracy and detection efficiency, and reduce the influence of human bad operation on detection.

[0112] As shown in Figure 2 The cloud chart generated by simulating the working space of the double robot by the Monte Carlo method is shown, which includes the working space of the master robot, the working space of the slave robot and the collaborative working space of the double robot, the detectable rocket tank model can be determined through the working space of the robot and the detection process of the rocket tank, and the region of each joint is determined by using the collaborative working space, and the operation range of each joint is reset.

[0113] As shown in Figure 3 , 4 The principle and working flow chart of the spatial position parameter collection of binocular vision are shown. The spatial coordinate parameters of a certain feature point can be solved through the coordinate parameters collected by two vision sensors and the included angle of the two vision sensors, and the robot is combined to complete the in-place detection.

[0114] As shown in Figure 5 The communication principle of the robot control system and the detection software is shown. It includes the operation layer, the control layer and the device layer, the operation layer mainly realizes man-machine interaction, satisfies parameter and state display, the control layer mainly realizes the communication of the operation layer and the device layer, data acquisition, image acquisition, etc., and the device layer specifically realizes the fixed function of the detection process.

[0115] The above describes the principles and implementation modes of the present application by using specific examples, and the above example is only used to help understand the method and core idea of the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description should not be understood as a limitation of the present application.

Claims

1. A dual robot path planning method for binocular vision-based rocket tank detection, characterized in that, The application relates to a method for detecting a rocket tank type by using a double robot system. The method comprises the following steps: A robot kinematics model is established, and a measuring instrument and a weighted moving average filtering method are used to compensate and calibrate kinematics parameters in the robot kinematics model, including the following steps: A robot kinematics parameter is defined, and a robot kinematics model is established based on a D-H method; A robot end pose conversion matrix is obtained by using rotation and displacement to represent a homogeneous transformation relationship between adjacent joint coordinate systems; The inverse kinematics is solved by using an algebraic solving method, and each joint kinematics parameter error value of the robot is obtained by separating variables one by one; The robot kinematics model established based on the D-H method is compensated by using the obtained robot joint kinematics parameter error value, and the equation is simplified by using matrix left multiplication one by one to obtain the following equation:

2. The dual robot path planning method for binocular vision-based rocket tank detection according to claim 1, wherein, The robot end pose conversion matrix is as follows: The position of the binocular vision acquisition device is calibrated by using the measuring instrument, and the coordinate conversion between the binocular vision and the double robot is established, including the following steps: The binocular vision acquisition device is applied, and an eye-to-hand mode is adopted, that is, one vision sensor is fixed on the upper end of the object to be detected, and the other vision sensor is fixed on the right end of the object to be detected; The binocular vision acquisition device is calibrated and positionally calibrated by using the measuring instrument; [ 0 T1] -10 T7= 1 T2 2 T3 3 T4 4 T5 5 T6 6 T7 [ 1 T2] -1 [ 0 T1] -10 T6= 2 T3 3 T4 4 T5 5 T6.

3. The binocular vision-based path planning method for dual-robot rocket tank detection according to claim 2, wherein, The kinematic parameters include joint rotation angle θ i , connecting rod offset distance d i , connecting rod length a i , the included angle between joint axes α i .

4. The dual-robot path planning method for binocular vision-based rocket tank detection according to claim 3, wherein, After the calibration and the positional calibration are completed, the binocular vision acquisition device is used to acquire space images. wherein i T i+1 denotes the pose transformation matrix from the coordinate system of the i-th joint to the i+1 coordinate system.

5. The binocular vision-based path planning method for dual-robot rocket tank detection according to claim 1, wherein, ​ ​ The spatial image is used to establish a coordinate transformation between the image and the robot. The relationship between the pixel coordinates in the image and the specified robot coordinates (u, v) and (X W , Y W , Z W ) is given by the equations: where (x, y) and (X C ,Y C ,Z C ) are image coordinates, camera coordinates; the coordinates of a point P in space from robot coordinates (X W ,Y W ,Z W ) to image pixel coordinates (u, v), f x = f / d x , f y = f / d y represent the scaling size of the focal length f in the direction of the two axes of the pixel coordinates; M in is composed of the internal parameters after camera calibration; M out is composed of the rotation matrix R and the translation matrix t of the external parameters after camera calibration; P w represents a point in space.

6. The dual robot path planning method for binocular vision-based rocket tank detection according to claim 1, wherein, The position collection of the feature points of the rocket tank to be detected is completed according to the obtained coordinate conversion between binocular vision and the robot, and the model recognition of the rocket tank is completed according to the binocular vision detection, including: A mark point is added at the starting point of the detection of the weld of the rocket tank to be detected; The position parameter of the collected mark point is transmitted to the robot control system by the binocular vision acquisition device for position error compensation and correction; An image recognition library is established according to the size and structure of the rocket tank, and the recognition of the model of the tank is completed by comparing the images acquired by vision.

7. The binocular vision-based dual-robot path planning method for rocket tank detection according to claim 1, wherein: According to the obtained kinematic model of the robot, the cooperative path planning is performed, and the program number corresponding to the model of the rocket tank is generated, including: According to the model of the rocket tank, the path of the master robot is planned by interpolation using a quintic polynomial, and the quintic polynomial path and the first and second derivatives are obtained; The trajectory constraint conditions of the dual robots are set, and the master-slave following is established in a tight coordination; the running path of the slave robot is automatically generated according to the path of the master robot; The running paths of the dual robots are saved, and the robot program number corresponding to the model of the rocket tank is generated.

8. The dual-robot path planning method for the rocket tank detection based on binocular vision according to claim 7, characterized in that: According to the model of the rocket tank, the path of the master robot is planned by interpolation using a quintic polynomial, and the quintic polynomial path and the first and second derivatives are obtained, including: θ(t) = a0+ a1t + a2t 2 + a3t 3 + a4t 4 + a5t 5 where θ(t), are the joint angle, joint angular velocity, joint angular acceleration at time t, respectively, a0, a1, a2, a3, a4, a5 are undetermined coefficients of the 5th polynomial, and the following can be obtained by giving the constraints on the angle and acceleration of the starting point and target point: a0=θ0 where θ0, θ f respectively represent the joint angle at the start point time and the joint angle at the target point time, respectively represent the joint angular velocity at the start point time and the joint angular velocity at the target point time, respectively represent the joint angular acceleration at the start point time and the joint angular velocity at the target point time, t f represents the time taken from the start point to the target point.

Images