Large-range robot hand-eye calibration method fusing internal and external vision

Abstract

The invention discloses a large-range robot hand-eye calibration method fusing internal and external vision, and relates to the field of hand-eye calibration, and the method comprises the steps: S1, calibrating an internal vision system; s2, calibrating the photogrammetry system; s3, acquiring an image of the common target by using the internal vision system, and performing three-dimensional reconstruction to obtain common target three-dimensional data {Pi} measured by the internal vision system; respectively acquiring images of the internal target and the common target by using a photogrammetry system, and respectively performing three-dimensional reconstruction to obtain three-dimensional data {Qi} of the internal target and three-dimensional data {Mi} of the common target measured by the photogrammetry system; s4, calculating a conversion matrix Z between the internal target and the internal visual system according to {Pi}, {Qi} and {Mi}; s5, calculating a pose conversion matrix Y between the tail end flange plate and the internal target; and S6, according to Z and Y, a relative transformation matrix X between the tail end flange plate and the internal visual system is solved, and calibration is completed. According to the method, the use of a robot tail end pose relation is avoided, and the hand-eye calibration precision is improved.

Description

Technical Field

[0001] The invention relates to the field of hand-eye calibration, and in particular to a large-scale robot hand-eye calibration method integrating internal and external vision. Background Art

[0002] Robot hand-eye calibration technology is one of the key technologies to realize robot automatic measurement and processing, and is also one of the key research contents of machine vision. However, the traditional hand-eye calibration method introduces the accumulated error of robot arm kinematics, resulting in low hand-eye calibration accuracy.

[0003] There are two main ways to assemble the visual measurement system and the robot: one is to fix the visual measurement system directly to the flange at the end of the robot, forming an "eye on hand" assembly method. In this assembly method, the visual measurement system moves with the robot. At this time, the phase pose of the visual measurement system relative to the end of the robot is fixed. The other is the "eye outside the hand" assembly method, that is, the visual measurement system is fixed to the outside of the robot. At this time, the position and posture of the external visual measurement system relative to the robot base coordinate system is fixed. However, when the above two methods are used to solve the hand-eye matrix, the robot end position and posture will be introduced, thereby introducing the cumulative error of the robot arm, resulting in low accuracy of the final hand-eye calibration. The method of mixed calibration of external binocular vision and internal binocular vision is limited by the field of view of the external binocular vision, which makes the robot unable to move over a large range. Using a close-range photography system as an external vision can not only obtain high-precision three-dimensional coordinates of markers in a large range, but also avoid the problem of failure to measure caused by occlusion and measurement range limitation due to the freedom of shooting angle. Summary of the invention

[0004] In response to the above problems, the present invention proposes a large-scale robot hand-eye calibration method that integrates internal and external vision. The integration of internal and external vision avoids the use of the robot's end posture relationship and solves problems such as the cumulative error of the robotic arm.

[0005] A large-scale robot hand-eye calibration method integrating internal and external vision is described in the following steps:

[0006] S1, fix the internal visual system and perform calibration;

[0007] S2, calibrate the photogrammetry system;

[0008] S3, moving the robot arm to several groups of different postures, and using the internal vision system to collect images of the common target in each posture, and performing three-dimensional reconstruction based on the common target image to obtain the three-dimensional data of the common target measured by the internal vision system {P i}; Fix the robot arm, use the photogrammetry system to collect images of the internal target and the public target at different positions and perform three-dimensional reconstruction respectively, and obtain the three-dimensional data of the internal target measured by the photogrammetry system {Q i} and the three-dimensional data of the public target measured by the photogrammetry system {M i}; The common target is set at a fixed position;

[0009] S4, according to {P i}, {Q i} and {M i} Calculate the transformation matrix Z between the internal target and the internal visual system;

[0010] S5, establishing the coordinate system pose of the flange of the robot manipulator in the photogrammetry system coordinate system, and obtaining the pose of the internal target in the photogrammetry system coordinate system, and calculating the pose conversion matrix Y between the end flange and the internal target;

[0011] S6, according to the transformation matrix Z between the internal target and the internal vision system and the posture transformation matrix Y between the end flange and the internal target, the relative transformation matrix X between the end flange and the internal vision system is solved to complete the robot hand-eye calibration.

[0012] Preferably, in S1, an external calibration plate is used to calibrate the internal visual system, as follows:

[0013] The internal vision system is fixed, and the external calibration plate is moved to several groups of different postures. The internal vision system is used to collect the calibration plate images of each posture, and the Zhang Zhengyou calibration method is used to calibrate the internal vision system.

[0014] Preferably, the S2 is as follows:

[0015] The photogrammetry system is fixed so that the coordinate system of the photogrammetry system remains unchanged relative to the base coordinate system of the robot. The photogrammetry system shoots the calibration rod and several coding points placed in the measurement space to obtain the three-dimensional coordinates of the base coordinate system of the photogrammetry system, the calibration rod and the coding points under the photogrammetry system to complete the calibration.

[0016] Preferably, the S3 is as follows:

[0017] S31, using the internal visual system to collect the public marker point image of the public target, using the Otsu threshold method to perform binarization processing on the public marker point image, and separating the reflective marker point area and the background area; negating the binarized image; calling the Opencv library to identify the circular marker point and extract the sub-pixel from the negated binarized image, and obtain the sub-pixel center coordinates; performing three-dimensional reconstruction on the sub-pixel center coordinates to obtain the three-dimensional data of the public marker point, that is, the three-dimensional data of the public target measured by the internal visual system {P i};

[0018] S32, using the photogrammetry system to respectively collect images of the internal target and the public target and respectively perform three-dimensional reconstruction according to the three-dimensional reconstruction steps of S31, to obtain the three-dimensional data of the internal target measured by the photogrammetry system {Q i} and the three-dimensional data of the public target measured by the photogrammetry system {M i}.

[0019] Preferably, the S4 is specifically as follows:

[0020] S41, based on {P i}Construct the first local coordinate system, and calculate the transformation matrix between the origin coordinate system of the internal vision system and the common target coordinate system under the internal vision system based on the first local coordinate system C F T; and using the transformation matrix {P i}Go to the origin of the visual system and obtain the three-dimensional data of the public target in the internal visual origin coordinate system {P' i};

[0021] S42, {P i} and {M i} Perform rigid body transformation to obtain the transformation matrix between the common target coordinate system under the photogrammetry system and the common target coordinate system under the internal vision system Using the transformation matrix {P i} is converted to the photogrammetry system coordinate system to obtain the three-dimensional data of the public target of internal visual measurement in the photogrammetry system coordinate system {P" i};

[0022] S43, based on {Q i}Construct the second local coordinate system, and calculate the transformation matrix between the origin coordinate system of the photogrammetry system and the internal target coordinate system based on the second local coordinate system

[0023] S44, using the transformation matrix {P" i} is converted to the internal target coordinate system to obtain the public target three-dimensional data {P"' i}, based on {P"' i}Construct the third local coordinate system based on {P' i}、{P” i} and the third local coordinate system to calculate the transformation matrix between the internal target coordinate system and the internal visual origin coordinate system That is, Z; the construction method of the second local coordinate system and the third local coordinate system is the same as the construction method of the first local coordinate system.

[0024] Preferably, the method based on {P i}Construct the first local coordinate system as follows:

[0025] Any selection {P i}, three points P1, P2 and P3 are not on the same straight line; point P1 is taken as the origin; the x-axis of the local coordinate system is constructed with points P1 and P2, and its unit direction vector is expressed as The temporary y-axis of the local coordinate system is constructed with points P1 and P3, and its unit direction vector is expressed as Use the right-hand rule to calculate the unit direction vector of the z-axis Based on the unit direction vector X G and Z G , according to the right-hand rule, calculate the unit direction vector Y in the y-axis direction of the local coordinate system G , expressed as in, represents the vector product; | | represents the magnitude of the vector.

[0026] Preferably, the S5 is specifically as follows:

[0027] S51, using the rotation and translation of the TCP coordinate system to determine the conversion relationship from the photogrammetry system to the target center: control the flange at the end of the robot arm to rotate around the z-axis of the TCP coordinate system for multiple times, and use the photogrammetry system to collect the reflective marker point P z Then control the robot arm to move multiple times along the x-axis and y-axis positive directions of the TCP coordinate system to collect the three-dimensional data P of the reflective marker point x and P y , and for P z Perform circle fitting and calculate P x and P y Perform straight line fitting to determine the transformation matrix between the origin coordinate system of the photogrammetry system and the flange coordinate system of the internal target fitting

[0028] S52, when the robot arm moves along the positive direction of the y-axis of the TCP coordinate system, the three-dimensional data of the internal target is collected while collecting the last set of three-dimensional data of the marker points, a fourth local coordinate system is constructed based on the three-dimensional data of the internal target, and a conversion matrix between the origin coordinate system of the photogrammetry system and the internal target coordinate system is calculated

[0029] S53, based on the thickness of the reflective mark point, calculate the conversion matrix between the end flange coordinate system and the flange coordinate system of the internal target fitting

[0030] S54, according to the formula Calculate the transformation matrix between the end flange coordinate system and the internal target coordinate system That is, Y; where ( ) -1 represents the inverse matrix, .

[0031] Preferably, the relative transformation matrix X between the end flange and the internal vision system is solved using the formula X=YZ; wherein Z represents the transformation matrix between the internal target and the internal vision system; and Y represents the posture transformation matrix between the end flange and the internal target.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention does not use the robot end posture relationship in the entire calibration process, which avoids the influence of the accumulated error of the robot arm on the hand-eye calibration accuracy, and also reduces the influence of errors such as internal target processing and installation on the calibration accuracy, thereby achieving high-precision hand-eye calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention is further described in detail below in conjunction with the accompanying drawings;

[0035] Figure 1 It is a flow chart of a large-scale robot hand-eye calibration method integrating internal and external vision according to an embodiment of the present invention;

[0036] Figure 2 It is a schematic diagram of the basic principle of hybrid calibration of internal targets and internal vision systems in a large-scale robot hand-eye calibration method integrating internal and external vision according to an embodiment of the present invention;

[0037] Figure 3 Schematic diagram of an experimental platform for a large-scale robot hand-eye calibration method integrating internal and external vision according to an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of the basic principle of solving the conversion relationship between the flange and the internal target in a large-scale robot hand-eye calibration method integrating internal and external vision according to an embodiment of the present invention;

[0039] Figure 5 Schematic diagram of the calibration solution principle of a large-scale robot hand-eye calibration method integrating internal and external vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention is further described below through specific implementation modes.

[0041] In this embodiment, the internal vision system and the internal target are respectively arranged on the robot arm, the photogrammetry system (or the camera system, the close-range measurement system) is arranged outside the robot, and the public target is arranged outside the robot. In this embodiment, the internal vision system can be a binocular vision measurement system.

[0042] The calibration process of this embodiment involves multiple coordinate systems, which are expressed as follows:

[0043] {O} Photogrammetry system origin coordinate system

[0044] {E}Internal target coordinate system

[0045] {F} Public target coordinate system under internal vision system

[0046] {C} Internal vision system origin coordinate system

[0047] {G} Common target coordinate system in photogrammetry system

[0048] {B} Public target origin coordinate system in internal target coordinate system

[0049] {H} Flange coordinate system for internal target fitting

[0050] {I} End flange coordinate system

[0051] And the transformation matrix between coordinate systems is expressed as follows:

[0052] The transformation matrix between the internal vision system origin coordinate system and the common target coordinate system under the internal vision system;

[0053] The transformation matrix between the common target coordinate system under the photogrammetry system and the common target coordinate system under the internal vision system;

[0054] The transformation matrix between the origin coordinate system of the photogrammetry system and the internal target coordinate system;

[0055] The transformation matrix between the internal target coordinate system and the internal visual origin coordinate system;

[0056] The transformation matrix between the end flange coordinate system and the internal target coordinate system;

[0057] The transformation matrix between the end flange coordinate system and the flange coordinate system of the internal target fitting;

[0058] The transformation matrix between the origin coordinate system of the photogrammetry system and the flange coordinate system of the internal target fitting.

[0059] like Figure 1 As shown, the large-scale robot hand-eye calibration method integrating internal and external vision of this embodiment specifically includes the following steps.

[0060] S1, fix the internal vision system and perform calibration.

[0061] Specifically, the internal vision system is fixed, the external calibration plate is moved to N groups of different postures, and the internal vision system is used to collect the calibration plate image of each posture, and the Zhang Zhengyou calibration method is used to calibrate the internal vision system.

[0062] S2, calibrate the photogrammetry system.

[0063] The photogrammetry system is fixed so that the coordinate system of the photogrammetry system remains unchanged relative to the base coordinate system of the robot. The photogrammetry system shoots the calibration rod and several coding points placed in the measurement space to obtain the three-dimensional coordinates of the base coordinate system of the photogrammetry system, the calibration rod and the coding points under the photogrammetry system.

[0064] Specifically, Figure 2 As shown, the internal vision system and the external photogrammetry system are respectively installed at a first position of the flange at the end of the robot's mechanical arm and a first position outside the robot; the external common target and the internal target are respectively installed at a second position outside the robot and a second position of the flange at the end of the robot's mechanical arm. Figure 3 This is the hand-eye calibration experimental platform of this embodiment. Move the robot to a suitable posture, in which the internal vision system at the end of the robot can always "see" the external public target in space, and the external photogrammetry system outside the robot can always "see" the internal target at the end of the robot arm and the external public target.

[0065] S3, moving the robot arm to several groups of different postures, and using the internal vision system to collect images of the common target in each posture, and performing three-dimensional reconstruction based on the common target image to obtain the three-dimensional data of the common target measured by the internal vision system {P i}; Fix the robot arm, use the photogrammetry system to collect images of the internal target and the public target at different positions and perform three-dimensional reconstruction respectively, and obtain the three-dimensional data of the internal target measured by the photogrammetry system {Q i} and the three-dimensional data of the public target measured by the photogrammetry system {Mi}; The public target is set at a fixed position. The specific steps are as follows:

[0066] S31, according to the external public target image collected by internal vision, the public mark point image on the public target is binarized by using the Otsu threshold method to separate the reflective mark point area and the background area. The binary image is inverted to obtain the background that can be recognized by Opencv. The Opencv library is called to identify and extract sub-pixel circular mark points, obtain the sub-pixel center coordinates, and then perform three-dimensional reconstruction to obtain the three-dimensional data of the public mark points.

[0067] S32, using an external photogrammetry system to acquire three-dimensional data of internal targets and external common targets.

[0068] S4, based on the three-dimensional data of the public target measured by the internal vision system {P i}、Three-dimensional data of internal targets measured by photogrammetry system {Q i} and the three-dimensional data of the public target measured by the photogrammetry system {M i}Build a hand-eye calibration model and calculate the transformation matrix Z between the internal target and the internal visual system. The specific steps are as follows:

[0069] S41, based on the public target three-dimensional data {P i}Establish a local coordinate system, and convert the measured three-dimensional data into i}Go to the origin of the visual measurement system. Select any 3D data {P i}, three points P1, P2 and P3 are not on the same straight line, and point P1 is used as the origin. The x-axis of the local coordinate system is constructed with points P1 and P2, and its unit direction vector is recorded as Construct the temporary y-axis of the coordinate system with points P1 and P3, and its unit direction vector Finally, using the right-hand rule, we can calculate the unit direction vector of the z-axis Given a unit direction vector X G and Z G Similarly, according to the right-hand rule, the final unit direction vector in the y-axis direction can be calculated The transformation matrix between the internal visual system origin coordinate system and the public target coordinate system under the internal visual system is calculated. And using the transformation matrix Point cloud data {P i}Go to the origin of the visual system and obtain the three-dimensional data of the public target in the internal visual origin coordinate system {P' i}.

[0070] S42, according to the three-dimensional data of the public target measured by the internal visual system {P i} and the common target 3D data obtained by external photogrammetry system {M i} Perform rigid body transformation to register the internal visual measurement point cloud data to the photogrammetry system, and obtain the transformation matrix between the public target coordinate system under the photogrammetry system and the public target coordinate system under the internal visual system Using the transformation matrix The public target in the internal visual coordinate system is converted to the photogrammetry system coordinate system to obtain the three-dimensional data of the public target in the internal visual measurement under the photogrammetry system {P” i}.

[0071] S43, based on the internal target three-dimensional data measured by the external photogrammetry system {Q i}, construct the local coordinate system using the method in S41, and calculate the transformation matrix between the origin coordinate system of the photogrammetry system and the internal target coordinate system by constructing the local coordinate system

[0072] S44, according to the transformation matrix The public target three-dimensional data {P" i} is converted to the internal target coordinate system to obtain the public target three-dimensional data {P"' of the internal visual measurement in the internal target coordinate system i}, and based on {P"' i} to construct a local coordinate system, which is the common target origin coordinate system {B} under the internal target coordinate system. Finally, the three-dimensional data {P' i} and {P"' i} and its constructed local coordinate system {B}, calculate the transformation matrix between the internal target coordinate system and the internal visual origin coordinate system That is, the transformation matrix Z between the internal target and the internal visual system needs to be solved.

[0073] S5, establish the coordinate system pose of the flange of the robot manipulator in the photogrammetry system coordinate system, and obtain the pose of the internal target in the photogrammetry system coordinate system, and calculate the pose conversion matrix Y between the end flange and the internal target. Figure 4 As shown; the specific steps are as follows:

[0074] S51, using the rotation and translation of the TCP coordinate system to determine the conversion relationship from the photogrammetry system to the flange coordinate system of the internal target fitting. Control the flange at the end of the robot to rotate around the z-axis of the TCP coordinate system for multiple times, and use the photogrammetry system to collect the reflective marker point P zThen control the robot to move multiple times along the x-axis and y-axis positive directions of the TCP coordinate system to collect the three-dimensional data P of the landmark points. x and P y , and for P z Perform circle fitting and calculate P x and P y Perform straight line fitting to determine the transformation matrix between the origin coordinate system of the photogrammetry system and the flange coordinate system of the internal target fitting.

[0075] S52, when the robot moves along the positive direction of the y-axis of the TCP coordinate system, the last set of three-dimensional data of the marker points is collected, and the three-dimensional data of the internal target is collected at the same time, and the conversion matrix between the origin coordinate system of the photogrammetry system and the internal target coordinate system is calculated according to the form of constructing the local coordinate system.

[0076] S53, according to the thickness of the pasted reflective marker point, calculate the conversion matrix between the end flange coordinate system and the flange coordinate system of the internal target fitting

[0077] S54, according to the formula Calculate the final transformation matrix between the end flange coordinate system and the internal target coordinate system That is Y.

[0078] S6, according to the transformation matrix Z between the internal target and the internal vision system and the posture transformation matrix Y between the end flange and the internal target, the relative transformation matrix X between the end flange and the internal vision system is solved to complete the robot hand-eye calibration.

[0079] For details, see Figure 5 As shown, based on the transformation matrix Z between the internal target and the visual system, and the posture transformation matrix Y between the end flange and the internal target, the formula X=YZ is used to solve the final relative transformation matrix (or hand-eye matrix) X between the end flange and the internal visual system.

[0080] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A large-scale robot hand-eye calibration method integrating internal and external vision, characterized in that: The steps include: S1, fix the internal visual system and perform calibration; S2, calibrate the photogrammetry system; S3, moving the robot arm to several groups of different postures, and using the internal vision system to collect images of the common target in each posture, and performing three-dimensional reconstruction based on the common target image to obtain the three-dimensional data of the common target measured by the internal vision system {P i }; Fix the robot arm, use the photogrammetry system to collect images of the internal target and the public target at different positions and perform three-dimensional reconstruction respectively, and obtain the three-dimensional data of the internal target measured by the photogrammetry system {Q i } and the three-dimensional data of the public target measured by the photogrammetry system {M i }; The common target is set at a fixed position; S4, according to {P i }, {Q i } and {M i } Calculate the transformation matrix Z between the internal target and the internal visual system; S5, establishing the coordinate system pose of the flange of the robot manipulator in the photogrammetry system coordinate system, and obtaining the pose of the internal target in the photogrammetry system coordinate system, and calculating the pose conversion matrix Y between the end flange and the internal target; S6, according to the transformation matrix Z between the internal target and the internal vision system and the posture transformation matrix Y between the end flange and the internal target, the relative transformation matrix X between the end flange and the internal vision system is solved to complete the robot hand-eye calibration.

2. The large-scale robot hand-eye calibration method integrating internal and external vision according to claim 1 is characterized in that: In S1, an external calibration plate is used to calibrate the internal visual system, as follows: The internal vision system is fixed, and the external calibration plate is moved to several groups of different postures. The internal vision system is used to collect the calibration plate images of each posture, and the Zhang Zhengyou calibration method is used to calibrate the internal vision system.

3. The large-scale robot hand-eye calibration method integrating internal and external vision according to claim 1 is characterized in that: The S2 is specifically as follows: The photogrammetry system is fixed so that the coordinate system of the photogrammetry system remains unchanged relative to the base coordinate system of the robot. The photogrammetry system shoots the calibration rod and several coding points placed in the measurement space to obtain the three-dimensional coordinates of the base coordinate system of the photogrammetry system, the calibration rod and the coding points under the photogrammetry system to complete the calibration.

4. The large-scale robot hand-eye calibration method integrating internal and external vision according to claim 1 is characterized in that: The S3 is specifically as follows: S31, using the internal visual system to collect the public marker point image of the public target, using the Otsu threshold method to perform binarization processing on the public marker point image, and separating the reflective marker point area and the background area; negating the binarized image; calling the Opencv library to identify the circular marker point and extract the sub-pixel from the negated binarized image, and obtain the sub-pixel center coordinates; performing three-dimensional reconstruction on the sub-pixel center coordinates to obtain the three-dimensional data of the public marker point, that is, the three-dimensional data of the public target measured by the internal visual system {P i }; S32, using the photogrammetry system to respectively collect images of the internal target and the public target and respectively perform three-dimensional reconstruction according to the three-dimensional reconstruction steps of S31, to obtain the three-dimensional data of the internal target measured by the photogrammetry system {Q i } and the three-dimensional data of the public target measured by the photogrammetry system {M i }.

5. The large-scale robot hand-eye calibration method integrating internal and external vision according to claim 1 is characterized in that: The S4 is specifically as follows: S41, based on {P i }Construct the first local coordinate system, and calculate the transformation matrix between the origin coordinate system of the internal vision system and the common target coordinate system under the internal vision system based on the first local coordinate system And using the transformation matrix {P i }Go to the origin of the visual system and obtain the three-dimensional data of the public target in the internal visual origin coordinate system {P' i }; S42, {P i } and {M i } Perform rigid body transformation to obtain the transformation matrix between the common target coordinate system under the photogrammetry system and the common target coordinate system under the internal vision system Using the transformation matrix {P i } is converted to the photogrammetry system coordinate system to obtain the three-dimensional data of the public target of internal visual measurement in the photogrammetry system coordinate system {P" i }; S43, based on {Q i }Construct the second local coordinate system, and calculate the transformation matrix between the origin coordinate system of the photogrammetry system and the internal target coordinate system based on the second local coordinate system S44, using the transformation matrix {P" i } is converted to the internal target coordinate system to obtain the public target three-dimensional data {P"' i }, based on {P"' i }Construct the third local coordinate system based on {P' i }、{P” i } and the third local coordinate system to calculate the transformation matrix between the internal target coordinate system and the internal visual origin coordinate system That is, Z; the construction method of the second local coordinate system and the third local coordinate system is the same as the construction method of the first local coordinate system.

6. The large-scale robot hand-eye calibration method integrating internal and external vision according to claim 5 is characterized in that: The based {P i }Construct the first local coordinate system as follows: Any selection {P i }, three points P1, P2 and P3 are not on the same straight line; point P1 is taken as the origin; the x-axis of the local coordinate system is constructed with points P1 and P2, and its unit direction vector is expressed as The temporary y-axis of the local coordinate system is constructed with points P1 and P3, and its unit direction vector is expressed as Use the right-hand rule to calculate the unit direction vector of the z-axis Based on the unit direction vector X G and Z G , according to the right-hand rule, calculate the unit direction vector Y in the y-axis direction of the local coordinate system G , expressed as in, represents the vector product; | | represents the magnitude of the vector.

7. The large-scale robot hand-eye calibration method integrating internal and external vision according to claim 1 is characterized in that: The S5 is specifically as follows: S51, using the rotation and translation of the TCP coordinate system to determine the conversion relationship from the photogrammetry system to the target center: control the flange at the end of the robot arm to rotate around the z-axis of the TCP coordinate system for multiple times, and use the photogrammetry system to collect the reflective marker point P z Then control the robot arm to move multiple times along the x-axis and y-axis positive directions of the TCP coordinate system to collect the three-dimensional data P of the reflective marker point x and P y , and for P z Perform circle fitting and calculate P x and P y Perform straight line fitting to determine the transformation matrix between the origin coordinate system of the photogrammetry system and the flange coordinate system of the internal target fitting S52, when the robot arm moves along the positive direction of the y-axis of the TCP coordinate system, the three-dimensional data of the internal target is collected while collecting the last set of three-dimensional data of the marker points, a fourth local coordinate system is constructed based on the three-dimensional data of the internal target, and a conversion matrix between the origin coordinate system of the photogrammetry system and the internal target coordinate system is calculated S53, based on the thickness of the reflective mark point, calculate the conversion matrix between the end flange coordinate system and the flange coordinate system of the internal target fitting S54, according to the formula Calculate the transformation matrix between the end flange coordinate system and the internal target coordinate system That is, Y; where ( ) -1 Represents the inverse matrix.

8. The large-scale robot hand-eye calibration method integrating internal and external vision according to claim 1 is characterized in that: The relative transformation matrix X between the end flange and the internal vision system is solved using the formula X=YZ, where Z represents the transformation matrix between the internal target and the internal vision system, and Y represents the posture transformation matrix between the end flange and the internal target.

Images