A visual-based automatic calibration method and system for optical processing of industrial robots

Abstract

An automatic calibration method and system for visual-based industrial robot optical processing, used for measuring the coordinates of a mark target paper in optical processing. The automatic calibration method mainly includes the following steps: step S1, the camera is mounted on the processing grinding head at the end of the mechanical arm, the internal and external parameters of the camera are calibrated, and the pose transformation relationship between the camera and the mechanical arm is approximately determined by using a hand-eye calibration method. Step S2, correct the pose of the camera relative to the robot. Step S3, paste the mark target paper on the processed surface of the optical element, and establish the base coordinates at the geometric center. Step S4, manually operate the robot to shoot the mark target paper, and at the same time, the position information of the robot in the base coordinate system is sent to the upper computer in real time. The upper computer identifies the target paper position and performs coordinate transformation, and outputs the coordinates of the target paper in the base coordinate system. Compared with the existing optical processing calibration method, the method has higher efficiency and better accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial robot calibration in optical processing, and particularly relates to a visual-based industrial robot optical processing automatic calibration method and system. BACKGROUND

[0002] In the last stage of optical processing of off-axis aspheric surface and parabolic mirror surface, the mirror surface image detected by using a laser interferometer with a compensator or a holographic film will have a large distortion, and thus needs to be corrected for distortion before being used as a reference for subsequent processing. The existing distortion correction method is to paste a plurality of mark target papers on the mirror surface before light detection, measure the positions of the target papers with distortion on the surface image after light detection, and then input the two position parameter data into a distortion correction software to complete the distortion correction.

[0003] The existing method for measuring the actual position of the target paper has two kinds. One method is to use an industrial robot to mount a measuring head, and an operator operates the robot to align the measuring head with the target paper, and then records the position of the target paper. This method has problems of low efficiency and insufficient accuracy. The other method is to use a laser tracker to measure. In this method, the target ball cannot be held by hand when measuring a large-diameter optical element, and needs to be placed on an extended rod, which is very difficult to operate. Moreover, the laser tracker needs to be preheated for a period of time before use, and the efficiency is also low. SUMMARY

[0004] To overcome the deficiencies of the existing calibration method, the present application aims to develop a visual calibration device and method. Specifically, a visual-based industrial robot optical processing automatic calibration method and system is provided, in which a camera is mounted on the end mill of a mechanical arm to shoot and identify the target paper, and the actual position of the target paper is calculated according to the pose of the robot. The present application can effectively improve the accuracy and efficiency of the calibration process.

[0005] The technical scheme provided by the present application is as follows:

[0006] A visual-based industrial robot optical processing automatic calibration method, comprising the following steps:

[0007] Step S1: mounting a camera on the end mill of a mechanical arm, calibrating the internal and external parameters of the camera using a hand-eye calibration method, and approximately determining the pose transformation relationship between the camera and the mechanical arm;

[0008] Step S2: correcting the pose of the camera relative to the industrial robot using an end correction method to improve the measurement accuracy of the system;

[0009] Step S3: pasting mark target papers on the processed surface of an optical element, fixing the position of the optical element, establishing a base coordinate system at the geometric center thereof, and importing the base coordinate system into the robot system;

[0010] Step S4: manually operating the industrial robot to shoot the mark target paper attached to the optical element, while sending the position information of the industrial robot in the base coordinate system to the host computer in real time, the host computer identifying the position of the mark target paper and performing coordinate transformation, and outputting the position of the mark target paper in the base coordinate system.

[0011] Further, step S2 comprises the following steps:

[0012] First, multiply the conversion matrix X obtained by the hand-eye calibration method described in step S1 by the machining grinding head TCP value to obtain the camera TCP value, input the camera TCP value into the robot system and select the camera as the tool, the machining grinding head TCP is the tool grinding head center point value, and the camera TCP value is the camera center point value.

[0013] Second, attach the calibration target paper to a hard foam board, place the calibration board on the hard foam board, and calculate the external parameters of the camera relative to the hard foam board after the camera shoots the calibration board, and adjust the pose of the hard foam board to make it parallel to the camera coordinate system according to the external parameters.

[0014] Then, operate the robot to shoot the calibration target paper, identify the origin coordinates of the calibration target paper using shape matching technology in image recognition, and perform coordinate transformation to calculate the coordinates of the calibration target paper in the robot system.

[0015] Finally, select the calibration sharp head as the tool in the robot system, move the robot to make the calibration sharp head above the coordinate point of the calibration target paper in the robot system, slowly lower it, and make the calibration sharp head leave a mark on the calibration target paper, measure the coordinate difference between the mark and the origin of the calibration target paper using a vernier caliper, modify the camera TCP according to the coordinate difference, and complete the calibration.

[0016] The application also provides a visual-based industrial robot optical machining automatic calibration system, which executes the visual-based industrial robot optical machining automatic calibration method, and comprises a host computer, a camera, a laser distance sensor and a camera support frame.

[0017] The application has the following advantages over the existing manual calibration method:

[0018] 1. During calibration, only the camera needs to be moved to the approximate position above the target paper to take a clear photo, which is simple and efficient.

[0019] 2. It uses visual recognition to identify the target paper position and corrects the calibration results by hand and eye, resulting in higher calibration accuracy. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the automatic calibration device.

[0021] Figure 2 This is a schematic diagram of the correction principle.

[0022] Figure 3 This is a diagram of coordinate transformation.

[0023] Figure 4 This is a schematic diagram of the calibration process of an automatic calibration device. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0025] like Figure 1 As shown, the vision-based automatic calibration system for industrial robots includes a camera, a calibration tip, a laser displacement sensor, and a camera support.

[0026] like Figures 2-4 As shown, the vision-based automatic calibration method for industrial robots includes the following steps:

[0027] Step S1: Figure 1 The camera fixture is mounted on the machining head at the end of the robotic arm to calibrate the camera's internal and external parameters. The hand-eye calibration method is used to roughly determine the pose transformation relationship between the camera and the robotic arm.

[0028] Step S2: Correct the hand-eye calibration results. First, multiply the transformation matrix X obtained from the hand-eye calibration with the grinding head TCP value to obtain the camera TCP value, input it into the robot system, and select the camera as the tool. According to the calibration principle, the transformation matrix T between the camera coordinate system and the calibration plate coordinate system... cThe rotation part of the transformation matrix is set to a unit matrix, i.e. it is necessary to ensure that the coordinate system of the correction target paper is parallel to the coordinate system of the camera, and therefore the correction target paper is pasted on a hard foam board, and the calibration board is also placed on the hard foam board, and the camera calculates the external parameters of the camera relative to the hard foam board after shooting the calibration board, and adjusts the pose of the hard foam board to make it parallel to the coordinate system of the camera. The robot is operated to shoot the correction target paper, the shape matching technology in image recognition is used to identify the origin coordinates of the marked target paper, and coordinate transformation is performed to calculate the coordinates of the correction target paper in the robot system. The correction sharp is selected as a tool in the robot system, the robot is moved to make the correction sharp above the position of the correction target paper in the coordinates of the robot system, and is slowly lowered to leave a mark on the correction target paper. The coordinate difference between the mark and the origin of the correction target paper is measured using a vernier caliper, and the TCP of the camera is modified according to the value to complete the correction.

[0029] The correction principle in step S2 is as follows:

[0030] As shown in Figure 2 , the conversion matrix X of the hand-eye calibration is the conversion matrix of the camera relative to the grinding head coordinate system, and multiplying the position value of the grinding head TCP by X can obtain the TCP value of the camera. According to the correction principle diagram, P w = T x T c P o , wherein P o is the position of the correction target paper in the coordinate system of the calibration board; T c is the conversion matrix between the coordinate system of the camera and the coordinate system of the calibration board; and T x is the conversion matrix between the coordinate system of the camera and the coordinate system of the robot. The above formula is transformed to obtain P o =P w T x1 T c1 , wherein T c1 and T x1 are inverse matrices of T c and T x , respectively.

[0031] Let the error matrix of T x1 be ΔT x1 ,

[0032] then P o =(P Δw T x1 +ΔT x1 )T c1 (1),

[0033] Let P o1 =T c1 T x1 P Δw, the actual coordinate in the coordinate system of the calibration plate corresponding to the measured coordinate, the measured coordinate with error in the coordinate system of the robot, and the difference between the measured coordinate and the actual coordinate is ΔP o = P o -P o1 , formula (1) can be simplified as

[0034] ΔP o = ΔT x1 T c1 (2)

[0035] For simplicity, T c1 is set to be a unit matrix, and the matrix is written as a homogeneous transformation form, so that

[0036]

[0037] p c1 is the translation part in T c1 ;

[0038] ΔR x1 and Δp x1 are the rotation part and the translation part of ΔT x1 , respectively;

[0039] R o and p o are the rotation part and the translation part of ΔP o , respectively;

[0040] Substituting formula (2) can obtain

[0041]

[0042] In the system, since the surface of the detected mark target paper is mostly not a plane, and from the hand-eye calibration result, the value of ΔR x1 is within 0.43°, so it is not very meaningful to solve it, and the focus is to correct the translation part. From formula (3), the rotation part of ΔP o is equal to the rotation part of the error matrix, and also has an impact on the translation, so ΔR x1 p c1 This part of the translation error caused by the rotation error is combined into Δp x1 and corrected together.

[0043] Step S3: Paste the mark target paper on the machined surface of the optical element, fix the position of the optical element, establish the base coordinate system at the geometric center thereof and import it into the robot system.

[0044] Step S4: The robot is manually operated to take pictures of the marking target paper attached to the optical element. At the same time, the robot system is programmed to send the robot's position information in the base coordinate system to the host computer in real time. The host computer identifies the position of the marking target paper, performs coordinate transformation, and outputs the position of the marking target paper in the base coordinate system.

[0045] The principle of coordinate transformation in step S4 is as follows:

[0046] like Figure 3 As shown, O w For the robot world coordinate system, O t For the robot tool coordinate system, O c Let T be the camera coordinate system. w X is the transformation matrix between these three coordinate systems, and T is the transformation matrix between these three coordinate systems. c2 Let O be the transformation matrix between the pixel coordinate system and the camera coordinate system. Then, the spatial point P lies in O. w The coordinates P below w =T w XT c2 P o T can be obtained by reading the position coordinates from the robot system and then performing a homogeneous transformation. w The transformation matrix X can be determined using hand-eye calibration. c2 This represents the translation and scaling relationship between the pixel coordinate system and the camera coordinate system. Its value can be obtained from the camera intrinsic parameters and the distance from the camera to point P, as shown in the formula:

[0047]

[0048] Where f is the camera focal length; Z c Let be the z-coordinate of point P in the camera coordinate system; dx and dy are the pixel and distance scales in the x and y directions, respectively; u o and v o These represent the translation distances in the x and y directions, respectively.

[0049] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A vision-based optical processing auto-calibration method for industrial robots, characterized in that, The automatic calibration method comprises the following steps: Step S1: mount the camera on the machining grinding head at the end of the mechanical arm, calibrate the internal and external parameters of the camera using the hand-eye calibration method, and approximately determine the pose transformation relationship between the camera and the mechanical arm; Step S2: use the end correction method to correct the pose of the camera relative to the industrial robot, and improve the measurement accuracy of the system; including: First, multiply the conversion matrix X obtained by the hand-eye calibration method in step S1 by the machining grinding head TCP value to obtain the camera TCP value, input the camera TCP value into the robot system and select the camera as the tool, the machining grinding head TCP is the tool grinding head center point value, and the camera TCP value is the camera center point value; Second, paste the correction target paper on a hard foam board, place the calibration board on the hard foam board, and calculate the external parameters of the camera relative to the hard foam board after the camera shoots the calibration board, and adjust the pose of the hard foam board to make it parallel to the camera coordinate system according to the external parameters; Then, operate the robot to shoot the correction target paper, identify the origin coordinates of the correction target paper using shape matching technology in image recognition, and calculate the coordinates of the correction target paper in the robot system through coordinate transformation; Finally, select the correction sharp head as the tool in the robot system, move the robot to make the correction sharp head above the coordinate point of the correction target paper in the robot system, slowly lower it, and make the correction sharp head leave a mark on the correction target paper, measure the coordinate difference between the mark and the origin of the correction target paper using a vernier caliper, modify the camera TCP according to the coordinate difference, and complete the correction; Step S3: paste the mark target paper on the machined surface of the optical element, fix the position of the optical element, establish the base coordinate system at the geometric center thereof and import it into the robot system; Step S4: manually operate the industrial robot to shoot the mark target paper pasted on the optical element, and simultaneously send the position information of the industrial robot in the base coordinate system to the upper computer in real time, the upper computer identifies the position of the mark target paper and performs coordinate transformation, and outputs the position of the mark target paper in the base coordinate system.

2. An automatic calibration system for optical machining of an industrial robot based on vision, wherein the automatic calibration system performs the automatic calibration method for optical machining of an industrial robot based on vision according to claim 1, and wherein the automatic calibration system comprises an upper computer, a camera, a laser distance sensor, and a camera support frame; the upper computer processes the pictures of the target paper shot by the camera, identifies the position of the target paper, performs coordinate transformation with the real-time position coordinates of the industrial robot received, and outputs the position of the target paper; the camera support frame supports the camera and the laser distance sensor, and fixes them on the machining grinding head; and the laser distance sensor, the camera, and the industrial robot are electrically connected to the industrial computer where the upper computer is located. ​

Images