A calibration and measurement device and method for industrial robots based on multi-view vision

Abstract

This invention discloses a calibration and measurement device and method for industrial robots based on multi-view vision. The invention constructs a multi-camera collaborative system of "wide field of view guidance + high-precision measurement" to achieve automatic measurement of the movement distance of industrial robots. The invention also uses the actual distance between feature points on the black and white grid targets E and F at both ends of a length scale. L T As an absolute benchmark, its Euclidean distance in the industrial robot base coordinate system is measured through multiple orientations. L o A series of high-confidence distance errors were obtained, which directly and objectively reflect the positioning accuracy defects of the industrial robot end effector and naturally include the comprehensive influence of all kinematic parameter errors.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and industrial robot calibration technology, and in particular to an industrial robot calibration and measurement device and method based on multi-view vision. Background Technology

[0002] As the main force in industrial automation and intelligence, industrial robots can liberate labor and significantly improve production efficiency, and are now widely used in many industries. Industrial robots are favored by modern manufacturing industries due to their high flexibility, spacious workspace, excellent human-machine interaction capabilities, low manufacturing costs, and strong versatility. However, facing the ever-increasing demands for high-difficulty, high-precision, and high-strength processing, industrial production is placing increasingly higher requirements on the positioning accuracy of industrial robots.

[0003] While existing mainstream calibration equipment (such as coordinate measuring machines and laser trackers) has achieved significant results in improving the absolute positioning accuracy of industrial robots, its application has obvious drawbacks: the cost of a single device can reach hundreds of thousands of yuan, the calibration process requires professional personnel, and equipment maintenance costs are high. In addition, these devices are generally not easy to move and have poor flexibility.

[0004] Chinese Patent Publication No. CN106903687A discloses an industrial robot calibration system and method based on laser ranging. Its core components include a calibration head containing three orthogonal laser ranging sensors whose measurement lines are at 90° to each other and intersect at a single point, a calibration base, and a processor. This invention utilizes a combination of laser ranging sensors and reflectors to achieve online and offline robot calibration, offering advantages such as system simplicity, low cost, and stability and reliability.

[0005] The aforementioned existing technical solutions have the following drawbacks: the devices require customized calibration mounts for different robots, resulting in poor versatility and a calibration range limited by the layout of the calibration mounts. Furthermore, calibration requires manual teaching and control, leading to low efficiency. In addition, lasers are easily affected by external environmental interference such as dust, and the calibration accuracy is highly dependent on the precise installation and stability of the reflector.

[0006] The utility model with announcement number CN221604422U relates to an industrial robot calibration system. Its core devices include: a three-axis tilt sensor, three industrial cameras, three backplates, etc. It utilizes the principles of visual positioning and direct tilt measurement, and has advantages such as low cost, strong dynamics, and comprehensive functions.

[0007] The above-mentioned existing technical solutions have the following drawbacks: the above devices have high requirements for on-site installation and debugging, the color differentiation of the back panel and sensors is easily affected by the external environment, and the calibration efficiency is also low.

[0008] Therefore, the design of precise calibration devices and efficient calibration methods is of great theoretical and practical significance in meeting the requirements for absolute positioning accuracy of industrial robots. Summary of the Invention

[0009] The problem to be solved by the present invention is to address the above-mentioned shortcomings of the existing technology by providing an industrial robot calibration and measurement device and method based on multi-view vision. It can effectively solve the problems of difficulty in achieving high precision and large field of view, low degree of automation, high equipment cost and poor flexibility in the existing industrial robot calibration technology, and provide a new technical solution for the further development of robot calibration technology.

[0010] The above-mentioned objective of the present invention is achieved through the following technical solution:

[0011] A calibration and measurement method for industrial robots based on multi-view vision includes the following steps:

[0012] Step 1. Two binocular vision systems work together, with the first binocular vision system used for wide field of view guidance and positioning, and the second binocular vision system used for high-precision measurement.

[0013] Step 2. Fix the binocular vision system at the end of the industrial robot and control the industrial robot to move to the first posture. Obtain the first coordinates of two feature points on the calibration target through the first binocular vision system.

[0014] Step 3. Transform the first coordinates to the industrial robot base coordinate system. Based on the coordinate information of the feature point in the base coordinate system, control the industrial robot to move to the second posture. Obtain the second coordinates of the feature point through the second binocular vision system and simultaneously collect the joint angle of the industrial robot at this time. At the same time, transform the second coordinates to the industrial robot base coordinate system.

[0015] Step 4. Repeatedly change the posture of the calibration target and collect multiple sets of the first coordinates and the second coordinates, and transform the second coordinates of each set of feature points to the industrial robot base coordinate system, and then calculate the measurement distance between feature points in each set;

[0016] Step 5. Using the actual distance between the feature points as a benchmark, optimize the error between the measured distance and the actual distance, iteratively solve the kinematic model parameters of the industrial robot, and complete the calibration.

[0017] A calibration and measurement device for industrial robots based on multi-view vision, comprising:

[0018] Industrial robots with programmable end effectors;

[0019] A first binocular vision system, fixed to the end effector, is used for wide-field-of-view guided positioning;

[0020] A second binocular vision system, fixed to the end effector, is used for high-precision measurement;

[0021] The calibration component includes a rigid scale of known length and calibration targets at both ends thereof, the calibration targets having feature points that can be recognized by the first binocular vision system and the second binocular vision system;

[0022] A drive mechanism, connected to the rigid scale, is used to drive the calibration component to change its spatial attitude;

[0023] The control unit, which is communicatively connected to the industrial robot, the first binocular vision system, the second binocular vision system, and the drive mechanism, is used for:

[0024] Control the movement of the industrial robot so that the first binocular vision system acquires the first coordinates of the feature point;

[0025] The movement of the industrial robot is controlled so that the second binocular vision system acquires the second coordinates of the feature points.

[0026] Based on the first and second coordinates of the feature points under multiple posture changes, the second coordinates of each group of feature points are sequentially transformed into the industrial robot base coordinate system. Then, the measured distance between the feature points in each group is calculated and compared with the actual distance to optimize the kinematic parameters of the industrial robot.

[0027] The beneficial effects of this invention are as follows:

[0028] 1. This invention innovatively constructs a multi-camera collaborative system of "wide field of view guidance + high-precision measurement" to realize automatic measurement of the movement distance of industrial robots.

[0029] 2. This invention also innovatively uses the actual distance L between feature points on the black and white grid targets E and F at both ends of the length scale. T As an absolute benchmark, its Euclidean distance L in the industrial robot base coordinate system is measured through multiple poses. o A series of high-confidence distance errors were obtained, which directly and objectively reflect the positioning accuracy defects of the industrial robot end effector and naturally include the comprehensive influence of all kinematic parameter errors.

[0030] 3. This invention applies an optimization algorithm for parameter identification, constructing a highly automated closed-loop system: based on a fixed distance L T Using multi-view vision collaboration as the absolute reference, the end-effector distance L in the industrial robot's coordinate system is generated. oBy using the distance error under N posture changes to drive the iterative optimization of kinematic parameters, a closed-loop calibration of "visual guidance - robot motion control - high-precision measurement - error feedback - parameter optimization" is formed, which significantly improves the efficiency and accuracy of calibration and greatly reduces the impact of human intervention. Attached Figure Description

[0031] Figure 1 This is a general diagram of a multi-view vision industrial robot calibration and measurement device provided in an embodiment of the present invention;

[0032] Figure 2 This is a diagram of the calibration instrument combination device provided in an embodiment of the present invention;

[0033] Figure 3 This is a diagram of an industrial robot body and multi-camera combination device provided in an embodiment of the present invention. Detailed Implementation

[0034] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not the entire structure.

[0035] like Figure 1 As shown in the figure, this application provides an industrial robot calibration and measurement device based on multi-view vision, including: two wide field-of-view cameras 9 and 13 (corresponding to wide field-of-view camera A and wide field-of-view camera B), two high-precision cameras 10 and 12 (corresponding to high-precision camera C and high-precision camera D), a planar light source 11, two ring light sources 8, two black and white grid targets 6 and 7 (corresponding to black and white grid target E and black and white grid target F), a length scale 1, a motor 3, a tripod 5, an industrial control computer 16, and a measurement and control box 18.

[0036] like Figure 3 As shown, the two ring light sources are respectively fixed on the two wide field-of-view cameras to provide uniform and high-brightness active illumination for the target; the two wide field-of-view cameras A and B form a binocular vision system, referred to as the No. 1 binocular vision system.

[0037] In one embodiment, the wide-field-of-view camera used is the Hikvision MV-CS200-10UM, along with the matching lens MVL-KF7528M-12MP, which can achieve a wide field of view of approximately 500mm × 350mm;

[0038] To reduce interference from ambient light, a high-brightness ring light source, P-RLG-82-90-R / BGW, is selected as the ring light source to ensure that the wide-field camera can quickly and accurately locate the target's position information during the large-scale search phase.

[0039] The planar light source is placed in the middle of the two high-precision cameras to provide stable, low-shadow, uniform illumination for the high-precision cameras to ensure that feature points are not overexposed or have local dark areas in the image. When the system is working, both the camera and the light source are always in working condition. The two high-precision cameras form another binocular vision system, called the second binocular vision system.

[0040] In one embodiment, the planar light source is a surface light source P-HFL-180-150-R / BGW; the high-precision camera is a Hikvision MV-CH250-90GM with a matching lens SA8520M-10MP, which can achieve micron-level positioning.

[0041] Furthermore, the two wide-field-of-view cameras, the two high-precision cameras, the two ring light sources, and the planar light source are fixed to the end of the industrial robot 15 via a connecting plate 14.

[0042] like Figure 2 As shown, the two black and white checkered targets are fixed at both ends of the length scale; the black and white checkered targets are 2×2 black and white checkered targets, and the intersection point in the middle is defined as the feature point of the black and white checkered target.

[0043] In one embodiment, to reduce motor load and scale expansion / contraction caused by external ambient temperature, the length scale is made of carbon fiber or Invar steel; to reduce deformation caused by temperature changes and to ensure that the intersection of the black and white grids does not drift due to ambient temperature, the two black and white grid calibration targets are made of ceramic material with a low coefficient of thermal expansion.

[0044] Furthermore, the motor is fixed to the upper end of the triangular bracket via flange 4; the length scale is fixed to the output shaft of the motor 3 via connecting rod 2 and can rotate with the motor 3; the motor 3 is connected to the measurement and control box 18. The measurement and control box, the two wide-field cameras, and the two high-precision cameras are respectively connected to the industrial control computer 16 on the table 17.

[0045] In one embodiment, the motor is selected from the Panasonic MINAS A6 series MHMF022L1U2M; the driver in the control box is selected from the Panasonic MINAS A6N series MADLN15NE; and the industrial computer is selected from the Advantech industrial computer IDS-3110.

[0046] Furthermore, in the device of the above embodiment, the intrinsic and extrinsic parameters of both the first and second binocular vision systems are known; the coordinate transformation matrices of the reference coordinate systems of the first and second binocular vision systems relative to the six-axis coordinate system of the industrial robot joints are also known, and are denoted as follows: and ; where, transformation matrix and The subscripts "1" and "2" in the text refer to the reference coordinate system of the first binocular vision system and the reference coordinate system of the second binocular vision system, respectively, and the transformation matrix. and The superscript R in the figure refers to the six-axis coordinate system of the industrial robot joint.

[0047] This application embodiment also provides an industrial robot calibration method based on multi-view vision. The calibration object selected in this application is an ABB IRB1410 industrial robot. During calibration, the tripod supports the length scale close to the industrial robot, so that the black and white grid targets E and F are both within the working range of the industrial robot, and the black and white grid targets E and F are both facing the industrial robot.

[0048] The steps of this method are as follows:

[0049] Step 1: The industrial control computer sends motion commands to the industrial robot. When the images captured by the wide-field-of-view cameras A and B fixed at the end of the industrial robot both contain black and white grid targets E and F, the industrial robot stops moving and the coordinates of the feature points of the black and white grid targets E and F in the first binocular vision system are calculated.

[0050] Furthermore, based on the triangulation principle of the binocular system, the coordinates of the feature points of the black and white grid targets E and F in the first binocular vision system are calculated and denoted as ( , , )and( , , );in,( , , )and( , , The subscripts “1A” and “1B” of the two coordinates refer to the coordinates of the black and white grid targets E and F under the No. 1 binocular vision system, respectively.

[0051] Step 2: Transform the coordinates of the two feature points of the black and white grid targets E and F in the binocular vision system to the industrial robot coordinate system.

[0052] Furthermore, the industrial control computer calculates the two feature points of the black and white grid targets E and F. , , )and( , , All are converted to the industrial robot coordinate system and denoted as ( , , )and( , , );

[0053] The conversion formulas are as follows:

[0054] [ , , ,1] T = [ , , ,1] T

[0055] [ , , ,1] T = [ , , ,1] T

[0056] in,( , , )and( , , The subscripts “1A0” and “1B0” in the two coordinates refer to the coordinates of two feature points of the black and white grid targets E and F, respectively, in the industrial robot base coordinate system after coordinate transformation in the No. 1 binocular vision system. It is the theoretical transformation matrix calculated based on the robot's nominal kinematic parameters, transforming from the joint six-coordinate system to the base coordinate system; the transformation matrix is... The superscript "0" refers to the industrial robot's base coordinate system, and the transformation matrix is... The subscript R also refers to the six-axis coordinate system of the industrial robot joint.

[0057] Step 3: Based on the coordinate information of the feature points of the black and white grid target E in the industrial robot's base coordinate system, the industrial control computer sends instructions to control the movement of the industrial robot, so that the high-precision cameras C and D can simultaneously acquire images of the feature points of the black and white grid target E, calculate the coordinates of the feature points of the black and white grid target E in the second binocular vision system, and acquire the six joint angles of the industrial robot at this time.

[0058] Specifically: by using feature point coordinates ( , , As a result, the industrial control computer uses the coordinate transformation matrix of the reference coordinate system of the second binocular vision system relative to the joint 6 coordinate system of the industrial robot. The system sends commands to control the movement of the industrial robot, causing the second binocular vision system to approach and stop near the feature points of the black and white grid target E, allowing high-precision cameras C and D to simultaneously acquire images of the feature points of the black and white grid target E.

[0059] Furthermore, based on the triangulation principle of the binocular system, the coordinates of the feature points of the black and white grid target E in the second binocular vision system are calculated and denoted as ( , , );in,( , , The subscript "2C" in the coordinate system refers to the coordinates of the black and white grid target E in the second binocular vision system.

[0060] At the same time, record the angles of the six joints of the industrial robot at this moment and denot them as ( );in,( The subscript in ) refers to the six joint angles under the calibration target E.

[0061] Step 4: Based on the coordinate information of the feature points of the black and white grid target F in the industrial robot's base coordinate system, the industrial control computer sends instructions to control the movement of the industrial robot, so that the high-precision cameras C and D can simultaneously acquire images of the feature points of the black and white grid target F, calculate the coordinates of the feature points of the black and white grid target F in the second binocular vision system, and acquire the six joint angles of the industrial robot at this time.

[0062] Specifically: based on feature point coordinates ( , , As a result, the industrial control computer uses the coordinate transformation matrix of the reference coordinate system of the second binocular vision system relative to the joint 6 coordinate system of the industrial robot. The system sends commands to control the movement of the industrial robot, causing the second binocular vision system to approach and stop near the feature points of the black and white grid target F, allowing high-precision cameras C and D to simultaneously acquire images of the feature points of the black and white grid target F.

[0063] Furthermore, based on the triangulation principle of the binocular system, the coordinates of the feature points of the black and white grid target F in the second binocular vision system are calculated and denoted as ( , , );in,( , , The subscript "2D" in the figure refers to the coordinates of the black and white grid target F in the second binocular vision system;

[0064] At the same time, record the angles of the six joints of the industrial robot at this moment and denot them as ( );in,( The subscript ) indicates the six joint angles under the calibration target F.

[0065] Step 5: Convert the coordinates of the two feature points of the black and white grid target E and F under the second binocular vision system to the industrial robot base coordinate system.

[0066] Furthermore, the industrial control computer calculates and identifies two feature points (E and F) on the black and white grid targets. , , )and( , , Converting to the industrial robot coordinate system, the result is denoted as ( , , )and( , , ); in,( , , )and( , , The subscripts “2C0” and “2D0” in the text refer to the coordinates of two feature points of the black and white grid targets E and F in the industrial robot base coordinate system after coordinate transformation in the second binocular vision system.

[0067] The conversion formula is as follows:

[0068] [ , , ,1] T = [ , , ,1] T

[0069] [ , , ,1] T = [ , , ,1] T

[0070] and All of these are based on the actual kinematic transformation matrix calculated from the joint six-coordinate system to the base coordinate system, which is derived from the real-time joint angles. The transformation matrix contains the six joint angles obtained in real time by the industrial robot and the kinematic parameters to be optimized.

[0071] Wherein, the transformation matrix is The superscript "0" refers to the industrial robot's base coordinate system, and the transformation matrix is... The subscript "R" refers to the six-axis coordinate system of the industrial robot joint, and the other subscript "E" refers to the calibration target E.

[0072] Wherein, the transformation matrix is The superscript "0" refers to the industrial robot's base coordinate system, and the transformation matrix is... The subscript "R" refers to the six-axis coordinate system of the industrial robot joint, and the other subscript "F" refers to the calibration target F.

[0073] Step 6: The industrial control computer calculates the distance between the two measured feature points in the industrial robot's coordinate system.

[0074] Furthermore, the two feature points obtained by the industrial control computer through calculation and measurement ( , , )and( , , The distance L in the industrial robot coordinate system o ; L o The subscript "o" represents the measured distance;

[0075] The calculation formula is as follows:

[0076] .

[0077] Step 7: Control the motor to rotate and change the posture of the length scale;

[0078] Step 8: Repeat steps 1 to 7 for a total of N (N>5) times;

[0079] Step 9: The industrial control computer uses an optimized algorithm, with the optimization objective being N L... o and L T The sum of squares of the errors between them is minimized;

[0080] Among them, L T (L) T (≥200mm and less than twice the arm span of an industrial robot) is the actual distance between feature points of black and white grid targets E and F, which is a known fixed value, and its subscript "T" indicates the true distance; It is the distance between two feature points on two black and white grid targets E and F in the industrial robot base coordinate system, where the subscript "o" represents the measured distance.

[0081] Then, the kinematic parameters of the industrial robot are obtained, and the kinematic parameters are updated through iterative optimization algorithms until the error converges, thereby completing the calibration of the industrial robot's kinematic parameters.

[0082] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A calibration and measurement method for industrial robots based on multi-view vision, characterized in that, Includes the following steps: Step 1. Two binocular vision systems work together, with the first binocular vision system used for wide field of view guidance and positioning, and the second binocular vision system used for high-precision measurement. Step 2. Fix the binocular vision system at the end of the industrial robot and control the industrial robot to move to the first posture. Obtain the first coordinates of two feature points on the calibration target through the first binocular vision system. Step 3. Transform the first coordinates to the industrial robot base coordinate system. Based on the coordinate information of the feature point in the base coordinate system, control the industrial robot to move to the second posture. Obtain the second coordinates of the feature point through the second binocular vision system and simultaneously collect the joint angle of the industrial robot at this time. At the same time, transform the second coordinates to the industrial robot base coordinate system. Step 4. Repeatedly change the posture of the calibration target and collect multiple sets of the first coordinates and the second coordinates, and transform the second coordinates of each set of feature points to the industrial robot base coordinate system, and then calculate the measurement distance between feature points in each set; Step 5. Using the actual distance between the feature points as a benchmark, optimize the error between the measured distance and the actual distance, iteratively solve the kinematic model parameters of the industrial robot, and complete the calibration.

2. The method according to claim 1, characterized in that, The calibration target is a black and white grid target fixed at both ends of a length scale. The feature point is the intersection of the center of the black and white grid. The length scale is rotated by a motor to change its posture.

3. The method according to claim 2, characterized in that, The length scale is made of carbon fiber or Invar steel, and the black and white checkered target is made of ceramic material to reduce thermal expansion error.

4. The method according to any one of claims 1-3, characterized in that, The first binocular vision system consists of two wide-field cameras and is equipped with a coaxial ring light source; the second binocular vision system consists of two high-precision cameras and is equipped with a planar light source to eliminate shadows.

5. The method according to claim 4, characterized in that, The field of view of the first binocular vision system is ≥500mm×350mm, and the positioning accuracy of the second binocular vision system reaches the micrometer level.

6. The method according to claim 1, characterized in that, The coordinate transformation is achieved by transforming the feature point coordinates from the camera coordinate system to the industrial robot base coordinate system based on the intrinsic and extrinsic parameters of the binocular vision system and the known transformation matrix with respect to the joint coordinate system of the industrial robot.

7. The method according to claim 1, characterized in that, The optimization is to minimize the distance measured N times. L o Distance from actual distance L T The sum of squared errors, where N > 5 and L T ≥200mm and less than twice the arm span of an industrial robot.

8. The method according to claim 1, characterized in that, The attitude change of the calibration target is achieved by a motor drive. The motor is connected to the measurement and control box, which communicates with the industrial control computer to synchronously control the movement of the industrial robot and image acquisition.

9. The method according to claim 1, characterized in that, The coordinates of the feature points are calculated using the principle of triangulation, and sub-pixel-level feature points of the black and white grid target are extracted using image processing algorithms.

10. A calibration and measurement device for industrial robots based on multi-view vision, used to implement the method of claim 1, characterized in that, include: Industrial robots with programmable end effectors; A first binocular vision system, fixed to the end effector, is used for wide-field-of-view guided positioning; A second binocular vision system, fixed to the end effector, is used for high-precision measurement; The calibration component includes a rigid scale of known length and calibration targets at both ends thereof, the calibration targets having feature points that can be recognized by the first binocular vision system and the second binocular vision system; A drive mechanism, connected to the rigid scale, is used to drive the calibration component to change its spatial attitude; The control unit, which is communicatively connected to the industrial robot, the first binocular vision system, the second binocular vision system, and the drive mechanism, is used for: Control the movement of the industrial robot so that the first binocular vision system acquires the first coordinates of the feature point; The movement of the industrial robot is controlled so that the second binocular vision system acquires the second coordinates of the feature points. Based on the first and second coordinates of the feature points under multiple posture changes, the second coordinates of each group of feature points are sequentially transformed into the industrial robot base coordinate system. Then, the measured distance between the feature points in each group is calculated and compared with the actual distance to optimize the kinematic parameters of the industrial robot.

Images