Mechanical arm tail end position calibration method based on visual closed loop

Abstract

The invention discloses a mechanical arm tail end position calibration method based on a visual closed loop, and the method comprises the steps: firstly obtaining the oblique projection point coordinates of a central point above the tail end of a mechanical arm on a target plane by using an industrial camera, estimating the vertical projection point position of the industrial camera, and calculating the height from the industrial camera to the target plane; before the mechanical arm is used, the position information of the mechanical arm is obtained through an image obtained by the industrial camera, the current zero point error of the mechanical arm is calculated according to the vertical projection point position of the industrial camera and the height from the industrial camera to a target plane, and automatic iteration compensation adjustment is conducted on the zero point. After the zero compensation of the mechanical arm, during the operation of the mechanical arm, the position information of the mechanical arm is obtained through an image obtained by the industrial camera, the current position error of the mechanical arm is calculated, and the actual position of the tail end of the mechanical arm is compensated and calibrated. According to the method, manual measurement errors are effectively avoided, the data acquisition efficiency is remarkably improved, and the calibration efficiency of the mechanical arm is improved.

Description

Technical Field

[0001] The invention belongs to the field of robot control and image processing, and in particular relates to a robot arm end calibration method based on a visual closed loop. Background Art

[0002] Industrial robots widely use serial structures, which are similar to human arms, have multiple degrees of freedom, each with a large rotation range, and are compact. As robots can be programmed offline or taught online, the manufacturing industry is moving towards intelligent digitalization, giving robots a broad future for application. Robots combined with the above technologies provide more solutions for industrial production and manufacturing, freeing workers from repetitive and heavy labor.

[0003] Industrial robots are key equipment in industrial automation and intelligence. With the rapid development of technologies such as computers, robot control and artificial intelligence, a variety of intelligent robots are emerging and applied in various industries. In practical applications, robots are developing towards high precision, high stability and high flexibility. Domestic and foreign researchers have conducted many in-depth studies on the accuracy and performance of industrial robots, mainly focusing on compensation methods and calibration techniques.

[0004] During the use of industrial robots, as the use time increases and they wear and age, the tightness of the parts will continue to change, and the parameters will also change. At this time, the posture deviation will cause losses to industrial production. In the production environment, the solution is generally to replace or repair parts, which has a large time and labor cost and does not meet future production requirements.

[0005] At present, most methods to solve the above problems are to build an error model to compensate for the errors of industrial robots and recalibrate them. However, due to many uncontrollable factors in the industrial production environment, such as temperature changes, vibrations, and mechanical wear, these factors will affect the accuracy and stability of the compensation model, making it difficult to maintain the compensation effect for a long time and the real-time performance is poor. The calibration method based on model parameters for error compensation needs to be further improved in terms of practicality. Summary of the invention

[0006] In view of the shortcomings of the prior art mentioned above, the present invention provides a method for calibrating the end position of a robotic arm based on a visual closed loop, which combines industrial camera visual detection, robotic arm body zero point compensation and end position real-time compensation to form a visual closed-loop feedback control to achieve the purpose of end position calibration, solve the problem of inaccurate end position of the robotic arm on the target plane in the robotic arm motion space during long-term use due to aging, wear, environmental changes and other factors, while taking into account real-time and stability.

[0007] The method comprises the following steps:

[0008] S1. Use an industrial camera to obtain the coordinates of the oblique projection point of the center point above the end of the robot arm on the target plane, estimate the position of the vertical projection point of the industrial camera, and calculate the height of the industrial camera from the target plane.

[0009] S2. Before using the robotic arm, obtain the position information of the robotic arm through the image obtained by the industrial camera, calculate the current zero point error of the robotic arm using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and perform automatic iterative compensation adjustment on the zero point.

[0010] S3. When the robot is in operation, after the zero point compensation of the robot is completed, the position information of the robot is obtained through the image acquired by the industrial camera based on the visual closed loop. The current position error of the robot is calculated using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and the actual position of the end of the robot is compensated and calibrated.

[0011] Furthermore, the step S1 of obtaining the oblique projection point coordinates of the center point above the end of the robot arm on the target plane is specifically implemented by calculating the vertical projection point coordinates of the industrial camera on the target plane and the height from the target plane. First, the industrial camera is calibrated, and the Zhang Zhengyou calibration method is used to obtain the camera's internal parameter matrix mtx to perform distortion correction on the image obtained by the industrial camera.

[0012] Move the robot arm to the top of the five preset reference points on the target plane in sequence, and align the center point below the end of the robot arm precisely with the reference point. When the robot arm is stable above the reference point, obtain the distortion-corrected image through the industrial camera. Identify the coordinates of the oblique projection point A of the center point above the end of the robot arm on the target plane.

[0013] Furthermore, in step S1, the vertical projection point position of the industrial camera is estimated, and the height of the industrial camera from the target plane is calculated as follows: the target plane is xoy, the triangle FAC is perpendicular to the plane xoy, the line segment ED is on the triangle FAC, and forms a triangle EAD with point A. The length of the line segment ED is the distance h from the vertex E above the end of the robot arm to the plane xoy end ; Point A is the oblique projection point of the center point E above the end of the robot arm in the view of the industrial camera F on the target plane xoy; Point B is on the plane xoy, and Point C is the vertical projection point of the industrial camera F on the plane xoy, where the straight line AB is parallel to the x-axis, the line segment BC is parallel to the y-axis, and the line segment AB is perpendicular to the line segment BC; Angle α is the angle formed by the intersection of the line segment AF and the line segment AC; Angle β is the angle formed by the line segment AC and the line segment AB. Point G is on the line segment AB, and the line segment DG is parallel to the line segment CB. x ADis the distance between point A and point D in the x direction on the plane xoy, y AD is the distance between point A and point D in the y direction on plane xoy. The distance between industrial camera F and plane xoy is h cam .

[0014] Using the similarity of triangle EAD and triangle FAC, combined with the coordinates of the oblique projection point A, the coordinates of the center point position D below the end of the robot arm are calculated. Each reference point and the corresponding oblique projection point construct a straight line, and a total of 5 straight lines are constructed. Due to the existence of actual errors, the 5 straight lines will not intersect at the same point. The least squares method is used for the coordinates of the intersection of all two straight lines to obtain the optimal estimate of the vertical projection point C of the industrial camera F on the plane xoy: (x cam ,y cam ). The least squares method is used as shown in Formula 1: On the plane xoy, find the target point C so that it intersects with all the intersection points (x i ,y i ) has the smallest distance square sum:

[0015]

[0016] According to the known coordinates of points A, C, and D, the height h of the end of the robotic arm end And triangle EAD is similar to triangle FAC, and the length of line segment AC is distance AC , the length of line segment AD is distance AD , then h cam Calculated by the following formula 2:

[0017]

[0018] Furthermore, in step S2, before the robot arm is used, the vertical projection point position C of the industrial camera and the height h of the industrial camera from the target plane are used to calculate the value of the vertical projection point position C of the industrial camera and the height h of the industrial camera from the target plane. cam , calculate the current zero point error of the robot arm and perform automatic iterative compensation adjustment on the zero point.

[0019] Furthermore, the specific process of step S2 is: control the robot arm to move to the reference point position coordinate P on the target plane, and calculate the coordinate of the center point D below the end of the robot arm. The center point coordinate D below the end is subtracted from the reference point position coordinate P. In this state, the positioning error of the robot arm end in the x and y directions is (Δx, Δy).

[0020] The positioning accuracy error (Δx, Δy) in the x and y directions of the robot is associated with the joint angle of the robot through the Jacobian matrix. The robot has three rotating joints whose axes are parallel to each other. They are positioned and oriented in the x and y directions in the target plane. The other joint is a mobile joint, which is used to complete the movement of the end piece perpendicular to the plane. The position of the wrist reference point is determined by the angles θ1 and θ2 of the two rotating joints. Its kinematic equation is shown in Formula 3 below:

[0021]

[0022] By taking the derivative of both ends of the equation with respect to time t, the relationship between the operation speed and the joint speed is shown in the following equation 4:

[0023]

[0024] In the above formula, the first matrix on the right side of the equal sign is the Jacobian matrix, that is:

[0025]

[0026] That is to say in, is the operating velocity vector of the end of the robot arm; Represents the velocity vector from the joint To operating speed The linear mapping of .

[0027] For the robot arm, the zero point compensation (Δθ1, Δθ2) of the robot arm joint angle is performed as shown in the following formula 6:

[0028]

[0029] Control the robot arm to move to the zero point position on the target plane. If it is within the allowable error range, complete the zero point compensation, otherwise continue to perform zero point compensation.

[0030] Furthermore, the specific implementation process of step S3 is as follows: based on the completion of the zero point compensation of the robotic arm, the position information of the robotic arm is obtained through the image obtained by the industrial camera during the operation of the robotic arm, the current position error of the robotic arm is calculated using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and the actual position of the end of the robotic arm is compensated and calibrated, as follows:

[0031] Further, according to the steps described above: obtain the coordinates of the center point D below the end of the robotic arm: (x real ,y real ). When the end of the robot moves to a certain point, the target position that the end is expected to reach is P: (x goal ,y goal), to obtain the end of the robot based on the visual closed loop, the specific length needs to be calibrated, that is, the pixel length that needs to be corrected in the x direction is x real -x goal ; The pixel length that needs to be corrected in the y direction is y real -y goal . Let the actual length corresponding to the unit pixel be L pixel2real , then the actual length L that needs to be corrected at the end of the robot in the x direction is real_x , the actual length L that needs to be corrected in the y direction real_y As shown in Formula 7 below:

[0032]

[0033] The positioning error value is fed back to the control system to compensate and calibrate the position of the end of the robotic arm in real time, thereby completing the compensation calibration of the actual position of the end of the robotic arm.

[0034] Compared with the prior art, the invention has the following beneficial effects: using an industrial camera to obtain the coordinates of the oblique projection point of the center point above the end of the robot arm on the target plane, and estimating the vertical projection point position of the industrial camera by the least squares method.

[0035] 1. The present invention proposes a method combining image recognition and least squares method to determine the vertical projection point position of an industrial camera on a target plane, and on this basis determine the height of the industrial camera from the target plane. This method effectively avoids manual measurement errors and significantly improves the efficiency of data acquisition.

[0036] 2. The present invention proposes a method combining image recognition and Jacobian matrix to realize visual closed loop and perform automatic iterative zero-point compensation on the robot body. Before the robot is used, the positioning accuracy error in the x and y directions of the robot is associated with the joint angle of the robot through the Jacobian matrix, and the zero-point compensation of the joint angle of the robot is performed automatically. This method realizes automated operation and significantly improves the efficiency of robot calibration.

[0037] 3. The present invention proposes a method for feeding back the actual position of the end of the robotic arm through image recognition, realizing a visual closed loop and compensating and calibrating the actual position of the end of the robotic arm. After the robotic arm moves to any target position above the target plane, the actual position of the end is fed back through image recognition, and the positioning error is fed back to the control system to compensate and calibrate the end position of the robotic arm in real time. This method, combined with image recognition, has a high degree of real-time performance and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the method for calibrating the end position of a robotic arm based on a visual closed loop proposed by the present invention;

[0039] Figure 2is a schematic diagram of a similar spatial geometric representation of the relationship between the robot arm and the target plane in the present invention;

[0040] Figure 3 It is a schematic diagram of reference points for selecting image target space of the present invention;

[0041] Figure 4 It is a schematic diagram of the vertical projection points of the industrial camera on the target plane obtained by taking the intersection points of multiple groups of intersection lines of the present invention;

[0042] Figure 5 It is a top view plane model diagram of the SCARA robot arm of the present invention;

[0043] Figure 6 It is a histogram of the error between the actual position and the target position of the end of the robot arm before and after correction of the robot arm of the present invention. DETAILED DESCRIPTION

[0044] The purpose of the present invention is to provide a method for calibrating the end position of a robot arm based on a visual closed loop, such as Figure 1 As shown, the following steps are included:

[0045] S1. First, calibrate the industrial camera to remove image distortion. Use the industrial camera to obtain the coordinates of the oblique projection point of the center point above the end of the robot arm on the target plane. Estimate the vertical projection point position of the industrial camera by the least squares method, and calculate the height of the industrial camera from the target plane based on the principle of similar triangles.

[0046] S2. Before using the robotic arm, obtain the position information of the robotic arm through the image obtained by the industrial camera, calculate the current zero point error of the robotic arm using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and perform automatic iterative compensation adjustment on the zero point.

[0047] S3. When the robot is in operation, based on the completion of zero-point compensation of the robot, the position information of the robot is obtained through the image acquired by the industrial camera, and the current position error of the robot is calculated using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and the actual position of the end of the robot is compensated and calibrated.

[0048] The present invention will be further described in detail below in conjunction with the embodiments. The following embodiments are for explanation of the present invention, and the present invention is not limited to the following examples.

[0049] Embodiment 1:

[0050] A visual closed-loop calibration method based on a Go robot. The center point above the end of the robotic arm is calculated through the blue feature area at the top of the end of the robotic arm, hereinafter referred to as the blue feature area. The top view of the robotic arm is as follows: Figure 5 As shown, the following steps are included:

[0051] First, calibrate the industrial camera of the Go robot to obtain de-distorted images.

[0052] Collect 20 pictures with complete chessboards. The chessboards in each picture have different angles, postures, and positions. Calculate the intrinsic parameter matrix mtx and lens distortion coefficient dist of the camera used after calibration to complete the distortion correction of the camera picture.

[0053] The specific process is as follows:

[0054] 1. Collect 20 pictures containing a complete chessboard, each with a different angle, posture and position. The best chessboard position in all pictures should cover the entire camera field of view.

[0055] 2. After collecting chessboard images, use the cv2.findChessboardCorners function in the OpenCV image processing library to identify the corner points in the image set and extract the corner point information.

[0056] 3. Perform camera calibration to obtain the camera's intrinsic parameter matrix and distortion coefficient as shown in equations 1 and 2:

[0057] Internal parameter matrix:

[0058]

[0059] Lens distortion coefficient:

[0060] dist=[-0.23238-0.06048 0.0028-0.001398 0.0975] (2)

[0061] Substitute each original image img into the function cv2.getOptimalNewCameraMatrix, obtain the free scale parameter as new_cameramtx, and finally use the following function cv2.undistort to obtain new_img as the distortion-corrected image.

[0062] Second, take the chessboard as the target plane and determine the coordinates of the vertical projection point C of the industrial camera on the target plane (x cam ,y cam ) and the distance from the industrial camera to the target plane is h cam .

[0063] like Figure 2 As shown, Figure 2The figure is a schematic diagram of the geometric representation of the similar space between the Go robot and the workspace. The industrial camera F acquires the image and extracts the Go board area from the image. Let the coordinates of the upper left corner be (0, 0), the right is the positive direction of the x-axis, the downward is the positive direction of the y-axis, and one grid is 1 unit length. Among the 19×19 grid points on the chessboard, nine "stars" occupy the nine positions of the chessboard, namely, the top, bottom, left, right, top, bottom, right, and center. The chessboard is divided into nine areas, namely P1 (3, 3), P2 (9, 3), and so on.

[0064] Select a reference point and move the robot arm to the reference point on the target plane. The coordinates of the center point D below the end of the robot arm are: (x D ,y D ), the host recognizes the coordinates of the oblique projection point A of the center point of the blue feature area above the end of the robot arm on the target plane through the acquired image: (x pic ,y pic ):

[0065] A straight line is constructed through the center point D at the bottom of the robot end and the oblique projection point A at the end. The vertical projection point of the industrial camera on the target plane falls on this straight line. Select the upper left, upper right, lower left, lower right, and middle star positions as reference points. Figure 3 As shown in the figure, for each reference point, a straight line is constructed, and a total of 5 straight lines are constructed. The least squares method is used for the coordinates of the intersection of any two straight lines to obtain the optimal estimated coordinates of the vertical projection point C of the camera on the target plane (x cam ,y cam )like Figure 4 As shown, the coordinate values ​​are: (476.42, 128.51).

[0066] According to the known height h of the end of the robot arm end , triangle EAD is similar to triangle FAC, then the distance h from the industrial camera to the target plane cam As shown in the following formula 3, the height h is calculated cam 521.2mm:

[0067]

[0068] Third, before using the Go robot arm, the position information of the arm is obtained through the image acquired by the industrial camera, and the positioning error of the current zero point of the arm in the x and y directions is calculated using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and the zero point is automatically iterated and compensated.

[0069] Determine if Figure 3 The robot is controlled to move to the reference point P on the target plane: (x expect ,y exptot). For each reference point position, the coordinates of the oblique projection point A of the center point of the blue feature area on the target plane are identified through the image obtained by the industrial camera: (x pic ,y pic ), using the similarity of triangle EAD and triangle FAC, the coordinates of the center point D below the end of the robotic arm are calculated: (x real ,y real ). The coordinates of point D are subtracted from the coordinates of the reference point P. In this state, the positioning error of the end of the robot in the x and y directions is (Δx, Δy), and the corresponding joint angle is (Δθ1, Δθθ2).

[0070] Based on the above description and known quantities, it can be calculated by the following formula:

[0071] First, the length of line segment AC can be obtained by the coordinates of the vertical projection point C and the oblique projection point A of the industrial camera. AC :

[0072]

[0073] Through the known coordinates of the vertical projection point C and the oblique projection point A, the line segment relationship between the triangles ACB in the plane xoy can be obtained as angle β:

[0074]

[0075] Through the relationship between the line segments and angles in triangle AFC, we can get the tangent value of angle α:

[0076]

[0077] In right triangle AED, since the tangent of angle α and ED are known, the length of AD can be obtained:

[0078]

[0079] Finally, in the right triangle ADG, since the length of AD is known and the angle β is known, the pixel difference x of D relative to A in the x and y directions can be calculated by formulas (8) and (8). AD and AD Values:

[0080] x AD =distance AD *cosβ (8)

[0081] y AD =distance AD *sinβ (9)

[0082] In this similar space geometric representation, the relationship between the coordinates of the center point D below the end of the robot and the coordinates of the oblique projection point A is shown in the following formula 10:

[0083] (x real ,y real )=(x pic +x AD ,y pic +y AD ) (10)

[0084] Subtract the coordinates of the center point below the end from the coordinates of the reference point, and let the actual length corresponding to the unit pixel be L pixel2real , the positioning error (Δx, Δy) of the end of the robot arm in the x and y directions in this state is obtained, as shown in the following formula 11:

[0085]

[0086] The positioning error (Δx, Δy) in the x and y directions of the robot is associated with the joint angle of the robot through the Jacobian matrix. The value of (Δx, Δy) is substituted into the above formula 6 to perform zero-point compensation (Δθ1, Δθ2) on the joint angle of the robot.

[0087] In order to make the compensation result more accurate, select Figure 3 The five points P1, P2, P3, P4, and P5 are known as the first joint length l1: 399 mm, and the second joint length l2: 310 mm. Five groups of compensation angles are obtained, and the zero point compensation values ​​corresponding to the five positions are superimposed. The specific values ​​are shown in Table 1 below:

[0088] Table 1 Parameters and values ​​related to zero point compensation of the robot arm

[0089]

[0090] The final zero point compensation value is obtained by summing the values ​​of Δθ1 and Δθ2 in the above table: joint 1 compensation -0.1875°, joint 2 compensation 0.2179°.

[0091] Finally, the robot arm is controlled to move to the zero point position on the target plane. If it is within the allowable error range, the zero point compensation is completed, otherwise the error compensation is continued. Finally, the robot arm is controlled to move to the zero point with an error of 0.50mm. For the Go robot, the error is acceptable within 1mm, so there is no need to perform the above steps.

[0092] Fourth, when the Go robot's mechanical arm is in operation, the position information of the mechanical arm is obtained through the image acquired by the industrial camera, and the current position error of the mechanical arm is calculated using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and the actual position of the end of the mechanical arm is compensated and calibrated.

[0093] The robot moves on the target plane to any target position P: (x goal ,y goal ), let the actual length corresponding to the unit pixel be L pixel2real , then the actual length L that needs to be corrected at the end of the robot in the x direction is real_x , the actual length L that needs to be corrected in the y direction real_y As shown in formula 12:

[0094]

[0095] The positioning error value (L real_x , L real_y ) directly feeds back to the control system to compensate and calibrate the end position of the robot arm in real time. Δx_old and Δy_old are the lengths of the offset of the target position of the end of the robot arm in the x and y directions before correction, Δx_correct and Δy_correct are the lengths of the offset of the target position of the end of the robot arm in the x and y directions after correction, respectively. Figure 6 The error bar graph of the actual position and target position of the robot end before and after correction for the 9 selected reference points. It can be seen from the figure that the posture error after calibration is basically kept within 1mm, and this method has good calibration capability.

[0096] The end calibration method of the visual closed loop identifies the blue feature area above the end of the robot arm to provide feedback on the actual end position of the robot arm, which is then compared with the target position to obtain the difference, and finally the end position of the robot arm is calibrated.

[0097] In this process, a similar space geometric representation diagram was proposed, and the least squares method was used to determine the position of the vertical projection point of the industrial camera. The height of the industrial camera from the target plane was calculated based on the principle of similar triangles, and visual feedback was performed. Finally, this method was applied in practice and put into production.

[0098] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for calibrating the end position of a robotic arm based on a visual closed loop, characterized in that: The following steps are involved: S1. Use an industrial camera to obtain the coordinates of the oblique projection point of the center point above the end of the robot arm on the target plane, estimate the vertical projection point position of the industrial camera, and calculate the height of the industrial camera from the target plane; S2. Before using the robotic arm, obtain the position information of the robotic arm through the image obtained by the industrial camera, calculate the current zero point error of the robotic arm by using the vertical projection point position of the industrial camera and the height of the industrial camera from the target plane, and perform automatic iterative compensation adjustment on the zero point; S3. After the zero point compensation of the robot arm, during the operation of the robot arm, based on the visual closed loop, the position information of the robot arm is obtained through the image obtained by the industrial camera, the current position error of the robot arm is calculated, and the actual position compensation and calibration of the end of the robot arm are performed.

2. The method for calibrating the end position of a robotic arm based on a visual closed loop according to claim 1, characterized in that: The specific implementation process of obtaining the coordinates of the oblique projection point of the center point above the end of the robotic arm on the target plane is as follows: Use Zhang Zhengyou calibration method to obtain the internal parameter matrix of the industrial camera as mtx and the distortion coefficient as dist, and perform distortion correction on the images obtained by the industrial camera; The robotic arm is moved to the top of five preset reference points on the target plane in sequence, and the center point below the end of the robotic arm is accurately aligned with the position of each reference point. When the robotic arm is stable above the reference point, the coordinates of the oblique projection point A of the center point above the end of the robotic arm on the target plane are identified through the distortion-corrected image.

3. The method for calibrating the end position of a robotic arm based on a visual closed loop according to claim 2, characterized in that: The specific process of estimating the vertical projection point position of the industrial camera and calculating the height of the industrial camera from the target plane is as follows: The target plane is xoy, the triangle FAC is perpendicular to the plane xoy, the line segment ED is on the triangle FAC, and forms a triangle EAD with point A; the length of the line segment ED is the distance h from the vertex E above the end of the robot arm to the plane xoy end ; Point A is the oblique projection point of the center point E above the end of the robot arm in the view of the industrial camera F on the target plane xoy; Point B is on the plane xoy, and Point C is the vertical projection point of the industrial camera F on the plane xoy, where the straight line AB is parallel to the x-axis, the line segment BC is parallel to the y-axis, and the line segment AB is perpendicular to the line segment BC; Angle α is the angle formed by the intersection of the line segment AF and the line segment AC; Angle β is the angle formed by the line segment AC and the line segment AB; Point G is on the line segment AB, and the line segment DG is parallel to the line segment CB; x AD is the distance between point A and point D in the x direction on the plane xoy, y AD is the distance between point A and point D in the y direction on plane xoy; the distance between industrial camera F and plane xoy is h cam ; Using the similarity of triangle EAD and triangle FAC, combined with the coordinates of the oblique projection point A, calculate the coordinates of the center point position D below the end of the robot arm; construct a straight line for each reference point and the corresponding oblique projection point, and construct a total of 5 straight lines. Use the least squares method for the coordinates of the intersection of all two straight lines, and find the target point C on the plane xoy so that it is consistent with all the intersection points (x i ,y i ) is the smallest distance square sum, and the optimal estimated vertical projection point C of the industrial camera F on the plane xoy is obtained: (x cam ,y cam ); According to the known coordinates of points A, C, and D, the height h of the end of the robotic arm end And triangle EAD is similar to triangle FAC, and the length of line segment AC is distance AC , the length of line segment AD is distance AD , get h cam The value of .

4. The method for calibrating the end position of a robotic arm based on a visual closed loop according to claim 3, characterized in that: The specific implementation process of step S2 is as follows: Control the robot arm to move to the reference point position coordinate P on the target plane, and calculate the coordinate of the center point D below the end of the robot arm; the coordinate D of the center point below the end is subtracted from the reference point position coordinate P. In this state, the positioning error of the end of the robot arm in the x and y directions is (Δx, Δy); The positioning accuracy error (Δx, Δy) of the robot in the x and y directions is associated with the joint angle of the robot through the Jacobian matrix. The robot has three rotating joints whose axes are parallel to each other. The positioning and orientation in the x and y directions are performed in the target plane. The other joint is a mobile joint, which is used to complete the movement of the end piece perpendicular to the plane. The position of the wrist reference point is determined by the angles θ1 and θ2 of the two rotating joints; its kinematic equation is as follows: Taking the derivative of both ends of the equation with respect to time t, we can obtain the following relationship between the operation speed and the joint speed: In the above formula, the first matrix on the right side of the equal sign is the Jacobian matrix, that is: For the robot arm, the zero point compensation (Δθ1, Δθ2) of the robot arm joint angle is as follows: Control the robot arm to move to the zero point position on the target plane. If it is within the allowable error range, complete the zero point compensation, otherwise continue to perform zero point compensation.

5. The method for calibrating the end position of a robotic arm based on a visual closed loop according to claim 4, characterized in that: The specific implementation process of step S3 is as follows: Get the coordinates of the center point D below the end of the robot arm (x real ,y real ); When the end of the robot moves to a certain point, the target position that the end is expected to reach is P(x goal ,y goal ); Get the specific length of the end of the robot arm based on the visual closed loop to be calibrated, that is, the pixel length corrected in the x direction is x real -x goal ; The pixel length corrected in the y direction is y real -y goal ; Let the actual length corresponding to the unit pixel be L pixel2real , then the actual length L of the robot end in the x direction is real_x , the actual length L corrected in the y direction real_y As shown below: The actual length corrected is used to compensate and calibrate the end position of the robotic arm in real time, thereby completing the actual position compensation calibration of the end position of the robotic arm.

Images