Industrial robot calibration measurement device and method based on multi-view vision

Abstract

The invention discloses an industrial robot calibration and measurement device and method based on multi-view vision. According to the invention, a multi-camera cooperative system of large-view guidance and high-precision measurement is constructed, and automatic measurement of the motion distance of the industrial robot is realized. The actual distance LT between the feature points on the black and white lattice targets E and F at the two ends of the length scale serves as the absolute reference, the Euclidean distance Lo of the length scale under the base coordinate system of the industrial robot is measured through multiple postures, a series of high-confidence-coefficient distance errors are obtained, the errors directly and objectively reflect the tail end positioning precision defect of the industrial robot, and the accuracy of the industrial robot is improved. And the comprehensive influence of all kinematics parameter errors is naturally included.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision and industrial robot calibration, and in particular to an industrial robot calibration and measurement device and method based on multi-vision. Background Art

[0002] As the driving force behind industrial automation and intelligentization, industrial robots can liberate labor and significantly improve production efficiency, and are currently widely used in numerous industries. Industrial robots are widely favored by the modern manufacturing industry due to their high flexibility, spacious workspace, excellent human-machine interaction, low manufacturing costs, and strong versatility. However, faced with the growing demand for complex, high-precision, and high-intensity processing, industrial production is placing increasing demands on the positioning accuracy of industrial robots.

[0003] While existing mainstream calibration equipment (such as coordinate measuring machines and laser trackers, among other high-precision devices) has achieved remarkable results in improving the absolute positioning accuracy of industrial robots, its application has significant drawbacks: A single device can cost hundreds of thousands of yuan, and the calibration process requires specialized personnel, resulting in high maintenance costs. Furthermore, these devices are generally difficult to move and lack flexibility, among other limitations.

[0004] Chinese patent publication number CN106903687A discloses a laser ranging-based industrial robot calibration system and method. Its core components include a calibration head with three orthogonal laser ranging sensors, whose measurement lines intersect at 90° angles, a calibration base, and a processor. This invention, which combines laser ranging sensors with reflectors, enables both online and offline robot calibration, offering advantages such as system simplicity, low cost, and stability and reliability.

[0005] The above-mentioned existing technical solutions have the following drawbacks: They require customized calibration bases for different robots, resulting in poor versatility, limited calibration range by the base's layout, and inefficient calibration due to manual instruction. Furthermore, lasers are susceptible to interference from external environments such as dust, and calibration accuracy is highly dependent on the precise installation and stability of the reflector.

[0006] The utility model with announcement number CN221604422U relates to an industrial robot calibration system. Its core devices include: a three-axis inclination sensor, three industrial cameras, three backboards and other devices. It uses the principles of visual positioning and direct inclination measurement, and has the advantages of low cost, strong dynamics and comprehensive functions.

[0007] The above-mentioned existing technical solutions have the following defects: the above-mentioned device has high requirements for on-site installation and debugging, the color distinction between the backplate and the sensor is easily affected by the external environment, and there is also the problem of low calibration efficiency.

[0008] Therefore, for the absolute positioning accuracy requirements of industrial robots, designing precise calibration devices and efficient calibration methods has very important theoretical significance and application value. Summary of the Invention

[0009] The problem to be solved by the present invention is to address the above-mentioned shortcomings in the existing technology and provide an industrial robot calibration and measurement device and method based on multi-vision, which can effectively solve the problems in the existing industrial robot calibration technology such as the difficulty in balancing high precision and large field of view, low degree of automation, high equipment cost and poor flexibility, and provide a new technical solution for the further development of robot calibration technology.

[0010] The above-mentioned object of the present invention is achieved through the following technical solutions:

[0011] A multi-vision-based industrial robot calibration measurement method includes the following steps:

[0012] Step 1. Two binocular vision systems work together, with the first binocular vision system used for wide field of view guidance and positioning, and the second binocular vision system used for high-precision measurement;

[0013] Step 2. Fix the binocular vision system at the end of the industrial robot, control the industrial robot to move to a first posture, and obtain first coordinates of two feature points on the calibration target through the first binocular vision system;

[0014] Step 3. Convert the first coordinates to the industrial robot's base coordinate system. Based on the coordinate information of the feature point in the base coordinate system, control the industrial robot to move to a second posture. Use the second binocular vision system to obtain the second coordinates of the feature point and simultaneously collect the joint angle of the industrial robot at this time. At the same time, convert the second coordinates to the industrial robot's base coordinate system.

[0015] Step 4. Repeatedly changing the posture of the calibration target and collecting multiple sets of the first coordinates and the second coordinates, and converting the second coordinates of each set of feature points into the industrial robot base coordinate system, and then calculating the measured distance between the feature points in each set;

[0016] Step 5. Using the actual distance between the feature points as a reference, optimize the error between the measured distance and the actual distance, iteratively solve the kinematic model parameters of the industrial robot, and complete the calibration.

[0017] An industrial robot calibration and measurement device based on multi-camera vision, comprising:

[0018] industrial robots with programmable end effectors;

[0019] A first binocular vision system, fixed to the end effector, for guiding positioning with a large field of view;

[0020] A second binocular vision system, fixed to the end effector, for high-precision measurement;

[0021] A calibration assembly comprising a rigid ruler of known length and calibration targets disposed at both ends thereof, the calibration targets having characteristic points recognizable by the first binocular vision system and the second binocular vision system;

[0022] a driving mechanism connected to the rigid scale and configured to drive the calibration component to change its spatial posture;

[0023] a control unit, communicatively connected to the industrial robot, the first binocular vision system, the second binocular vision system, and the driving mechanism, for:

[0024] Controlling the movement of the industrial robot so that the first binocular vision system acquires the first coordinates of the feature point;

[0025] Controlling the movement of the industrial robot so that the second binocular vision system acquires the second coordinates of the feature point;

[0026] Based on the first and second coordinates of the feature points under multiple posture changes, the second coordinates of each group of feature points are converted into the base coordinate system of the industrial robot in turn, and then the measured distance between the feature points in each group is calculated and compared with the actual distance to optimize the kinematic parameters of the industrial robot.

[0027] The beneficial effects of the present invention are:

[0028] 1. This invention innovatively constructs a multi-camera collaborative system of "large field of view guidance + high-precision measurement" to realize automatic measurement of the movement distance of industrial robots.

[0029] 2. The present invention also innovatively uses the actual distance L between the characteristic points on the black and white grid targets E and F at both ends of the length scale T As an absolute reference, the Euclidean distance L in the industrial robot base coordinate system is measured through multi-pose. o , a series of high-confidence distance errors are obtained, which directly and objectively reflect the positioning accuracy defects of the industrial robot end and naturally include the comprehensive influence of all kinematic parameter errors;

[0030] 3. The present invention uses optimization algorithms to identify parameters and builds a highly automated closed-loop system: based on a fixed distance L T As an absolute reference, the end distance L in the industrial robot base coordinate system is generated through multi-eye vision collaboration o, using the distance error under N posture changes to drive the iterative optimization of kinematic parameters, forming a closed-loop calibration of "visual guidance-robot motion control-high-precision measurement-error feedback-parameter optimization", which significantly improves the efficiency and accuracy of calibration and greatly reduces the impact of manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a general diagram of a multi-vision industrial robot calibration and measurement device provided by an embodiment of the present invention;

[0032] Figure 2 is a diagram of a calibration instrument assembly device provided by an embodiment of the present invention;

[0033] Figure 3 This is a diagram of an industrial robot body and multi-camera combination device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention, rather than limiting the present invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, rather than all structures.

[0035] like Figure 1 As shown, an embodiment of the present application provides an industrial robot calibration and measurement device based on multi-vision, including: two wide-field-of-view cameras 9, 13 (corresponding to wide-field-of-view camera A and wide-field-of-view camera B), two high-precision cameras 10, 12 (corresponding to high-precision camera C and high-precision camera D), a plane light source 11, two annular light sources 8, two black-and-white grid targets 6, 7 (corresponding to black-and-white grid target E and black-and-white grid target F), a length scale 1, a motor 3, a tripod bracket 5, an industrial computer 16 and a measurement and control box 18.

[0036] like Figure 3 As shown, the two annular light sources are respectively fixed on the two wide-field-of-view cameras to provide uniform, high-brightness active illumination for the target; the two wide-field-of-view cameras A and B form a binocular vision system, called binocular vision system No. 1.

[0037] In one embodiment, the wide field of view camera selected is Hikvision's MV-CS200-10UM and the matching lens MVL-KF7528M-12MP, which can achieve a large field of view of approximately 500mm×350mm;

[0038] In order to reduce the interference of ambient light, the ring light source selected is the high-brightness ring light source P-RLG-82-90-R / BGW, so as to ensure that the wide-field camera can quickly and accurately locate the position information of the target during the large-scale search phase.

[0039] The planar light source is placed in the middle of the two high-precision cameras to provide stable, low-shadow uniform lighting to the high-precision cameras to ensure that the feature points are not overexposed or have local dark areas in the imaging; when the system is working, the camera and the light source are always in working state; the two high-precision cameras constitute another binocular vision system, called the No. 2 binocular vision system.

[0040] In one embodiment, the plane light source is the surface light source P-HFL-180-150-R / BGW; the high-precision camera is Hikvision's MV-CH250-90GM and the matching lens SA8520M-10MP, which can achieve micron-level positioning.

[0041] Furthermore, the two wide-field-of-view cameras, the two high-precision cameras, the two annular light sources, and the planar light source are fixed to the end of the industrial robot 15 via a connecting plate 14 .

[0042] like Figure 2 As shown, the two black and white grid calibration targets are respectively fixed at the two ends of the length scale; the black and white grid target is a 2×2 black and white grid, and the middle intersection is defined as the feature point of the black and white grid target.

[0043] In a certain embodiment, in order to reduce the reasons such as the expansion and contraction of the scale caused by the motor load and the external ambient temperature, the length scale is made of carbon fiber or Invar; in order to reduce the deformation caused by temperature changes and ensure that the intersection of the black and white grid does not drift due to the ambient temperature, the two black and white grid calibration targets are made of ceramic materials with a small thermal expansion coefficient.

[0044] Furthermore, the motor is fixed to the upper end of the tripod via a flange 4. The length scale is fixed to the output shaft of the motor 3 via a connecting rod 2 and can rotate with the motor 3. The motor 3 is connected to the measurement and control box 18. The measurement and control box, the two wide-field cameras, and the two high-precision cameras are each connected to an industrial computer 16 on a table 17.

[0045] In one embodiment, the motor selected is MHMF022L1U2M from Panasonic's MINAS A6 series; the driver in the measurement and control box is MADLN15NE from Panasonic's MINAS A6N series; and the industrial computer selected is Advantech's IDS-3110.

[0046] Furthermore, in the device of the above embodiment, the internal and external parameters of the No. 1 binocular vision system and the No. 2 binocular vision system are known; the coordinate transformation matrices of the reference coordinate systems of the No. 1 and No. 2 binocular vision systems relative to the six-coordinate system of the industrial robot joint are known, and are respectively denoted as and ; Among them, the transformation matrix and The subscripts "1, 2" refer to the reference coordinate system of the first binocular vision system and the reference coordinate system of the second binocular vision system, respectively. The transformation matrix and The superscript R in the figure refers to the six-coordinate system of the industrial robot joint.

[0047] The present application also provides a multi-vision-based industrial robot calibration method. The ABB IRB1410 industrial robot is used for calibration. During calibration, a tripod-supported length scale is positioned close to the industrial robot, ensuring that both black and white grid targets E and F are within the working range of the industrial robot and facing the industrial robot.

[0048] The steps of this method are as follows:

[0049] Step 1: The industrial computer gives the industrial robot motion instructions. When the images captured by the wide-field-of-view cameras A and B fixed at the end of the industrial robot both contain the black and white grid targets E and F, the industrial robot stops moving and calculates the coordinates of the feature points of the black and white grid targets E and F in the No. 1 binocular vision system.

[0050] Furthermore, according to the triangulation positioning principle of the binocular system, the coordinates of the feature points of the black and white grid targets E and F in the binocular vision system No. 1 are calculated and recorded as ( , , )and( , , );in,( , , )and( , , ) The subscripts "1A" and "1B" of the two coordinates refer to the coordinates of the black and white grid targets E and F under the No. 1 binocular vision system.

[0051] Step 2: Convert the coordinates of the two feature points of the black and white grid targets E and F in the No. 1 binocular vision system to the industrial robot base coordinate system.

[0052] Furthermore, the industrial computer calculates the two feature points of the black and white grid targets E and F ( , , )and( , , ) are converted to the industrial robot base coordinate system and recorded as ( , , )and( , , );

[0053] The conversion formulas are:

[0054] [ , , ,1] T = [ , , ,1] T

[0055] [ , , ,1] T = [ , , ,1] T

[0056] in,( , , )and( , , The subscripts "1A0" and "1B0" of the two coordinates refer to the coordinates of the two feature points E and F of the black and white grid targets in the No. 1 binocular vision system after coordinate transformation in the industrial robot base coordinate system; It is the theoretical transformation matrix from the joint six-coordinate system to the base coordinate system, calculated based on the nominal kinematic parameters of the robot; the transformation matrix is The superscript “0” refers to the industrial robot base coordinate system, and the transformation matrix is The subscript R also refers to the six-coordinate system of the industrial robot joints.

[0057] Step 3: Based on the coordinate information of the feature points of the black and white grid calibration target E in the industrial robot's base coordinate system, the industrial computer sends instructions to control the movement of the industrial robot, allowing high-precision cameras C and D to simultaneously capture images of the feature points of the black and white grid target E, calculate the coordinates of the feature points of the black and white grid target E in the No. 2 binocular vision system at this time, and collect the six joint angles of the industrial robot at this time.

[0058] Specifically: Based on the feature point coordinates ( , , ) results, the industrial computer uses the coordinate transformation matrix of the reference coordinate system of the No. 2 binocular vision system relative to the coordinate system of the industrial robot joint 6 , sending instructions to control the movement of the industrial robot; making the second binocular vision system approach the feature point of the black and white grid target E and stop, so that the high-precision cameras C and D can simultaneously capture images of the feature point of the black and white grid target E;

[0059] Furthermore, according to the triangulation positioning principle of the binocular system, the coordinates of the feature points of the black and white grid target E in the second binocular vision system are calculated and recorded as ( , , );in,( , , The subscript "2C" in the coordinates refers to the coordinates of the black and white grid target E under the No. 2 binocular vision system;

[0060] At the same time, the six joint angles of the industrial robot are recorded and recorded as ( );in,( ) refers to the six joint angles under the calibration target E.

[0061] Step 4: Based on the coordinate information of the feature points of the black and white grid calibration target F in the industrial robot's base coordinate system, the industrial computer sends instructions to control the movement of the industrial robot, allowing high-precision cameras C and D to simultaneously capture images of the feature points of the black and white grid target F, calculate the coordinates of the feature points of the black and white grid target F in the No. 2 binocular vision system at this time, and collect the six joint angles of the industrial robot at this time.

[0062] Specifically: Based on the feature point coordinates ( , , ) results, the industrial computer uses the coordinate transformation matrix of the reference coordinate system of the No. 2 binocular vision system relative to the coordinate system of the industrial robot joint 6 , sending instructions to control the movement of the industrial robot; making the second binocular vision system approach the feature points of the black and white grid target F and stop, so that the high-precision cameras C and D can simultaneously capture images of the feature points of the black and white grid target F;

[0063] Furthermore, according to the triangulation positioning principle of the binocular system, the coordinates of the feature points of the black and white grid target F in the second binocular vision system are calculated and recorded as ( , , );in,( , , The subscript "2D" in the image refers to the coordinates of the black and white grid target F under the No. 2 binocular vision system;

[0064] At the same time, the six joint angles of the industrial robot are recorded and recorded as ( );in,( ) refers to the six joint angles under the calibration target F.

[0065] Step 5: Convert the coordinates of the two feature points of the black and white grid targets E and F under the No. 2 binocular vision system to the industrial robot base coordinate system.

[0066] Furthermore, the industrial computer calculates the two feature points of the black and white grid targets E and F ( , , )and( , , ) is converted to the industrial robot base coordinate system, and the result is recorded as ( , , )and( , , ); in,( , , )and( , , The subscripts "2C0" and "2D0" in the figure refer to the coordinates of the two feature points of the black and white grid targets E and F in the industrial robot base coordinate system after the coordinate transformation under the No. 2 binocular vision system;

[0067] The conversion formula is:

[0068] [ , , ,1] T = [ , , ,1] T

[0069] [ , , ,1] T = [ , , ,1] T

[0070] and The actual kinematic transformation matrix from the joint six-coordinate system to the base coordinate system is calculated based on the real-time joint angles. The transformation matrix contains the six joint angles obtained in real time by the industrial robot and the kinematic parameters to be optimized.

[0071] The transformation matrix is The superscript “0” refers to the industrial robot base coordinate system, and the transformation matrix is The subscript "R" also refers to the six-coordinate system of the industrial robot joint, and the other subscript "E" refers to the calibration target E;

[0072] The transformation matrix is The superscript “0” refers to the industrial robot base coordinate system, and the transformation matrix is The subscript "R" also refers to the six-coordinate system of the industrial robot joint, and the other subscript "F" refers to the calibration target F.

[0073] Step 6: The industrial computer calculates the distance between the two measured feature points in the industrial robot base coordinate system.

[0074] Furthermore, the two characteristic points ( , , )and( , , ) in the industrial robot base coordinate system is L o ; L o The subscript “o” represents the measured distance;

[0075] The calculation formula is:

[0076] .

[0077] Step 7, controlling the motor to rotate and change the posture of the length scale;

[0078] Step 8: Repeat steps 1 to 7 for a total of N (N>5) times;

[0079] Step 9: The industrial computer uses the optimized algorithm, and the optimization target is N L o and L T The sum of squares of errors between them is minimized;

[0080] Among them, L T (L T ≥200mm and less than twice the arm span of the industrial robot) is the actual distance between the feature points of the black and white grid targets E and F. It is a known fixed value, and its subscript "T" represents the real distance; It is the measured distance between the two feature points on the two black and white grid targets E and F in the base coordinate system of the industrial robot, and the subscript "o" represents the measured distance.

[0081] Then the kinematic model parameters of the industrial robot are obtained, and the kinematic parameters are updated through an iterative optimization algorithm until the error converges, thereby completing the kinematic parameter calibration of the industrial robot.

[0082] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-vision based industrial robot calibration measurement method, characterized in that: The following steps are involved: Step 1. Two binocular vision systems work together, with the first binocular vision system used for wide field of view guidance and positioning, and the second binocular vision system used for high-precision measurement; Step 2. Fix the binocular vision system at the end of the industrial robot, control the industrial robot to move to a first posture, and obtain first coordinates of two feature points on the calibration target through the first binocular vision system; Step 3. Convert the first coordinates to the industrial robot's base coordinate system. Based on the coordinate information of the feature point in the base coordinate system, control the industrial robot to move to a second posture. Use the second binocular vision system to obtain the second coordinates of the feature point and simultaneously collect the joint angle of the industrial robot at this time. At the same time, convert the second coordinates to the industrial robot's base coordinate system. Step 4. Repeatedly changing the posture of the calibration target and collecting multiple sets of the first coordinates and the second coordinates, and converting the second coordinates of each set of feature points into the industrial robot base coordinate system, and then calculating the measured distance between the feature points in each set; Step 5. Using the actual distance between the feature points as a reference, optimize the error between the measured distance and the actual distance, iteratively solve the kinematic model parameters of the industrial robot, and complete the calibration.

2. The method according to claim 1, characterized in that The calibration target is a black and white grid target fixed at both ends of the length scale, the characteristic point is the central intersection of the black and white grids, and the length scale is driven by a motor to rotate to change its posture.

3. The method according to claim 2, characterized in that The length scale is made of carbon fiber or Invar, and the black and white grid target is made of ceramic material to reduce thermal expansion error.

4. The method according to any one of claims 1 to 3, characterized in that The first binocular vision system consists of two wide-field-of-view cameras and is equipped with a coaxial ring light source; the second binocular vision system consists of two high-precision cameras and is equipped with a planar light source to eliminate shadows.

5. The method according to claim 4, characterized in that The field of view of the first binocular vision system is ≥500mm×350mm, and the positioning accuracy of the second binocular vision system reaches micron level.

6. The method according to claim 1, characterized in that The coordinate conversion is achieved by converting the feature point coordinates from the camera coordinate system to the industrial robot base coordinate system based on the intrinsic parameters, extrinsic parameters of the binocular vision system and a known transformation matrix with the industrial robot joint coordinate system.

7. The method according to claim 1, characterized in that The optimization is to minimize the N-times measured distance L o The actual distance L T The sum of squared errors, where N>5 and L T ≥200mm and less than twice the arm span of an industrial robot.

8. The method according to claim 1, characterized in that The posture change of the calibration target is achieved by motor drive. The motor is connected to a measurement and control box, and the measurement and control box communicates with an industrial computer to synchronously control the movement of the industrial robot and image acquisition.

9. The method according to claim 1, characterized in that The coordinates of the feature points are calculated using the triangulation principle, and the sub-pixel feature points of the black and white grid target are extracted using an image processing algorithm.

10. An industrial robot calibration and measurement device based on multi-camera vision, characterized in that: include: industrial robots with programmable end effectors; A first binocular vision system, fixed to the end effector, for guiding positioning with a large field of view; A second binocular vision system, fixed to the end effector, for high-precision measurement; A calibration assembly comprising a rigid ruler of known length and calibration targets disposed at both ends thereof, the calibration targets having characteristic points recognizable by the first binocular vision system and the second binocular vision system; a driving mechanism connected to the rigid scale and configured to drive the calibration component to change its spatial posture; a control unit, communicatively connected to the industrial robot, the first binocular vision system, the second binocular vision system, and the driving mechanism, for: Controlling the movement of the industrial robot so that the first binocular vision system acquires the first coordinates of the feature point; Controlling the movement of the industrial robot so that the second binocular vision system acquires the second coordinates of the feature point; Based on the first and second coordinates of the feature points under multiple posture changes, the second coordinates of each group of feature points are converted into the base coordinate system of the industrial robot in turn, and then the measured distance between the feature points in each group is calculated and compared with the actual distance to optimize the kinematic parameters of the industrial robot.

Images