A method, system and medium for automatic hand-eye calibration using robot body features

Abstract

This application discloses an automatic hand-eye calibration method, system, and medium utilizing robot body features. The method includes the following steps: selecting the robot body's geometric straight-line features and establishing a 3D model; acquiring images and joint angle data under multiple configurations; extracting the straight-line projection features of links in the images; establishing PnL projection constraints; solving for camera extrinsic parameters through parameter identification; and performing reprojection verification. The advantage of this invention is that it directly utilizes the robot body's geometric straight-line features as a 3D reference, eliminating the need for external calibration objects such as checkerboards or ArUco, thus eliminating the risks of purchasing, installing, maintaining, and damaging calibration objects. This facilitates direct execution of the calibration process on the production site and allows for convenient recalibration after extrinsic parameter drift.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to an automatic hand-eye calibration method, system, and medium utilizing the characteristics of a robot's body. Background Technology

[0002] In robot vision systems, cameras are typically mounted in a fixed manner near the robot's workspace (eye-to-hand configuration). To achieve accurate grasping, detection, and localization guided by vision, the transformation relationship between the camera coordinate system and the robot's base coordinate system needs to be precisely calibrated, i.e., the hand-eye extrinsic parameter matrix. The accuracy of the hand-eye calibration directly affects the overall performance of the robot vision system.

[0003] To solve the problem of hand-eye calibration of the rigid body transformation matrix between the camera coordinate system and the robot base coordinate system, classical methods are mostly based on the AX=XB or AX=YB model. Representative works include [1] (Tsai RY, Lenz R K. A new technique for fully autonomous and efficient 3 d robotics hand / eye calibration[J]. IEEE Transactions on robotics and automation, 1989, 5(3):345-358.) [2] (Park FC, Martin B J. Robot sensor calibration: solving AX= XBon the Euclidean group[J]. IEEE Transactions on Robotics and Automation, 1994, 10(5): 717-721.) and [3] (Horaud R, Dornaika F. Hand-eye calibration[J]. The international journal of robotics research, 1995, 14(3): 195-210.). These methods are theoretically mature and have high solution efficiency, and they remain the mainstream basic methods in industrial vision systems.Since hand-eye calibration involves solving on the SE(3) manifold, methods based on the probabilistic framework[4] (Wu J, Liu M, Zhu Y, et al. Globally optimal symbolic hand-eye calibration[J]. IEEE / ASME Transactions on Mechatronics,2020, 26(3): 1369-1379.), algebraic polynomial theory[5] (Ha J. Probabilistic framework for hand–eye and robot–world calibration $AX= YB$[J]. IEEE transactions onrobotics, 2022, 39(2): 1196-1211.), and branch-and-bound method[6] (Heller J, Havlena M, PajdlaT. Globally optimal hand-eye calibration using branch-and-bound[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(5): 1027-1033.) have been proposed to improve the accuracy of parameter estimation. However, most existing technologies rely on checkerboard patterns, calibration boards, or specialized fixtures, requiring manual placement of calibration objects and offline calculation after collecting multiple sets of posture data. Once the hand-eye matrix changes due to camera disassembly / reassembly, end-effector replacement, external collisions, or long-term vibrations, recalibration is usually necessary. Otherwise, a mismatch will occur between the visual measurement results and the robot's motion coordinates, significantly affecting the accuracy of grasping, assembly, and positioning. While this problem can be solved through downtime maintenance in industrial production lines, in scenarios such as home services, caregiving, and open environments, users typically lack professional calibration capabilities and find it difficult to frequently deploy calibration boards and perform complex recalibration processes. Once the hand-eye extrinsic parameters drift, the system's deployment efficiency and continuous availability will be severely impacted.

[0004] In recent years, some automated calibration technologies have attempted to reduce human intervention. For example, patent CN107253190B proposes an automated calibration scheme combining a laser generator and a calibration board; patent CN107498558A proposes a fully automated hand-eye calibration scheme based on preset motion paths and calibration board data; and patent CN110695996A proposes an automated hand-eye calibration scheme for industrial robots based on a calibration board and pose planning. While these methods have improved the automation level of hand-eye calibration to some extent, they still generally rely on external conditions such as calibration boards, lasers, or preset motion plans, and their support for low-cost and convenient recalibration after changes in hand-eye extrinsic parameters in open scenarios remains limited.

[0005] Therefore, there is an urgent need for a method that can autonomously complete hand-eye extrinsic parameter calibration in real-world scenarios without the need for external calibration objects, and can directly utilize the robot's own geometric features as a spatial reference, so as to effectively reduce the dependence on external calibration tools and human intervention. Summary of the Invention

[0006] This application provides a method, system, and medium for automatic hand-eye calibration using robot body features. Its advantage is that it directly uses the geometric straight-line features of the robot body as a three-dimensional reference, eliminating the need for external calibration objects such as checkerboards or ArUco, thus eliminating the risks of purchasing, installing, maintaining, and damaging calibration objects. It is also beneficial to directly execute the calibration process on the production site and to conveniently re-perform calibration after the external parameters drift.

[0007] The technical solution of this application is as follows:

[0008] On the one hand, this application provides an automatic hand-eye calibration method utilizing robot body features, comprising the following steps:

[0009] S1: Select the straight line geometric features of the robot body and establish a three-dimensional parametric model of the straight line features;

[0010] S2: Control the robot to move sequentially to several different joint configurations, and simultaneously collect robot joint angle and linear feature image data during the process;

[0011] S3: For each frame of the image data, extract the projection line parameters of the selected geometric line features onto the image plane;

[0012] S4: Project the 3D sampling points in the reference coordinate system onto the image plane to obtain the corresponding pixel coordinates. Calculate the point-to-line distance error between the projected points and the corresponding image lines. With N frames of images and M sampling points per frame, establish an overdetermined system of N×M constraints as the PnL projection constraint. The transformation matrix of the camera coordinate system relative to the reference coordinate system is the unknown parameter to be solved.

[0013] S5: Minimize the transformation matrix of the camera coordinate system relative to the reference coordinate system, and then identify the unknown parameters based on the PnL projection constraint to obtain the transformation matrix of the camera coordinate system relative to the reference coordinate system.

[0014] Furthermore, in step S1, the step of establishing the three-dimensional parametric model is as follows: For an n-degree-of-freedom robot, the selected straight line feature, given the joint angle vector... Under the given conditions, the homogeneous coordinates of the 3D point corresponding to the parameterized coordinates s∈(0,1) on the straight line feature in the reference coordinate system are:

[0015] (1)

[0016] Where T(q) is the transformation matrix from the reference coordinate system to the local coordinate system of the selected straight line feature, determined by the forward kinematics; f(s) is the local coordinate parameterization function of the straight line feature, in the form [L·s, 0, 0, 1]ᵀ, where L is the nominal length of the straight line feature;

[0017] Uniform sampling is performed on s∈(0,1) to generate a set of three-dimensional reference points along the linear feature.

[0018] This invention uses the straight-line geometric features of the robot body as the sole three-dimensional reference, eliminating the need for external calibration objects. The three-dimensional geometry of the features is calculated in real time by forward kinematics, requiring no additional measurements.

[0019] Furthermore, in step S2, the robot is controlled to move to N ≥ 15 different joint configurations, and the image projection direction of the selected straight line feature covers an angle range of at least 120°; the change in joint angle between adjacent configurations is not less than 5°.

[0020] Furthermore, in step S3, the projected line is represented using a normalized homogeneous representation:

[0021] (2)

[0022] Where [u, v]ᵀ represents the image pixel coordinates. .

[0023] Furthermore, step S3 also includes the following steps: for the detected candidate lines, perform quality assessment and screening, and retain the line features that meet the preset quality conditions to participate in subsequent identification;

[0024] The preset quality conditions for the quality assessment include one or more of the following indicators:

[0025] (a) The length of the line segment corresponding to the straight line is greater than a preset threshold;

[0026] (b) The ratio of the major axis to the minor axis of the fitted region is greater than the preset threshold;

[0027] (c) The proportion of interior points obtained based on RANSAC or other robust fitting methods is greater than a preset threshold.

[0028] (d) The consistency between the straight line direction and the projection direction of the target link meets the preset conditions.

[0029] In this invention, the quality assessment and frame selection mechanism for straight line features is as follows: the quality of each frame extraction result is assessed based on methods such as mask principal axis length, major-minor axis ratio, and RANSAC in-point ratio, and only reliable data is accepted as a constraint.

[0030] Furthermore, in step S4, let T be the transformation matrix of the camera coordinate system relative to the reference coordinate system. CB Its rotation component is R, and its translation component is t; projecting the three-dimensional sampling point p(s) in the reference coordinate system onto the image plane, the corresponding pixel coordinates are expressed as:

[0031] (3)

[0032] Where K is a known camera intrinsic parameter matrix, and ~ represents an equation in a homogeneous sense;

[0033] Calculate the point-to-line distance error between the projected point and the corresponding image line l = [a, b, c]ᵀ:

[0034] (4)

[0035] Given N frames of images and M sampling points per frame, an overdetermined system of equations with N×M constraints can be established as follows:

[0036] (5).

[0037] In this invention, parametric sampling modeling of straight line features is performed: M internal points are uniformly sampled on the straight line feature at s∈(0,1), avoiding endpoints, and M independent point-to-line constraints are constructed for each frame, resulting in a significantly larger number of constraints than the endpoint method. PnL constraints under normalized homogeneous straight line representation are implemented: the normalization constraint a²+b²=1 gives the residuals the physical meaning of pixel distance, and constraint construction and parameter identification are performed under the same metric.

[0038] Furthermore, in step S5, a rotation vector is used. Minimize the parameters of the rotation matrix R:

[0039] (6)

[0040] Where Exp SO(3) (·) represents the exponential mapping of SO(3), , Combined with the translation vector t, the transformation matrix of the camera coordinate system relative to the reference coordinate system consists of a six-dimensional vector. Complete expression;

[0041] Based on all the PnL constraints constructed by equation (5), the following parameter identification problem is established:

[0042] (7)

[0043] Where ρ(·) is the cost function, and after convergence, the optimal parameters are obtained. Restore rotation matrix Finally, the transformation matrix of the camera coordinate system relative to the reference coordinate system is obtained:

[0044] (8).

[0045] The above scheme constructs an external parameter identification framework under the robust cost function: the orthogonality constraint is eliminated by minimizing the rotation vector parameterization, and the solution is obtained under the robust cost function to suppress abnormal data interference.

[0046] Furthermore, it also includes step S6: calculating the obtained T CB Substitute the points into the projection equation, reproject the three-dimensional sampling points of the line back into each frame image, and calculate the pixel distance between the reprojected points and the labeled line, which is used as a quantitative verification index. If the reprojection error is within the preset range, the calibration result is considered reliable; otherwise, recalibrate.

[0047] On another front, this application provides an automatic hand-eye calibration system utilizing robot body features, including a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the steps in the method described above.

[0048] In another aspect, this application provides a computer-readable medium storing a computer program that, when executed by a computer, implements the steps in the method described above.

[0049] In summary, the beneficial effects of this application are as follows:

[0050] 1. No external calibration is required, which helps reduce on-site deployment and maintenance costs;

[0051] This invention directly utilizes the geometric straight-line features of the robot body as a three-dimensional reference, eliminating the need for external calibration objects such as checkerboards or ArUco. This eliminates the risks of purchasing, installing, maintaining, and damaging calibration objects, making it easier to execute the calibration process directly on the production site and conveniently re-calibrate after external parameters drift.

[0052] 2. It has natural robustness against partial shading;

[0053] This invention constructs constraints independently frame by frame through steps S3 and S4, with each frame being independent of the others. When a link is occluded by other objects, it is only necessary to discard that frame or switch to other visible straight line features, without affecting the constraint contribution of the remaining valid frames, and the calibration process does not need to be interrupted.

[0054] 3. A linear feature quality screening mechanism helps improve identification stability;

[0055] In step S3, the present invention performs quality assessment and screening on the straight line features extracted from each frame before constraint construction, and only uses high-quality straight line data that meet the preset conditions for parameter identification. This helps to suppress the interference of low-quality image features on the external parameter solution results and improve the stability and reliability of the identification process.

[0056] 4. Introducing a robust cost function helps suppress abnormal frame interference;

[0057] In step S5, the present invention employs robust cost functions such as Huber in parameter identification to automatically reduce the weight of abnormal constraints with large residuals, which helps to maintain the stability of identification results in real-world scenarios.

[0058] 5. Line feature extraction methods are unrestricted and adaptable to different engineering scenarios;

[0059] In step S3, this invention does not limit the method for extracting straight line features. Deep learning semantic detection, traditional edge detection, and manual annotation are all applicable, which is conducive to flexible deployment in scenarios with different levels of automation and accuracy requirements. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the robot body feature hand-eye calibration process of the present invention;

[0061] Figure 2 This is a schematic diagram of the PnL geometric constraints of the present invention;

[0062] Figure 3 This is a schematic diagram illustrating the visualization verification of reprojection error in this invention. Detailed Implementation

[0063] The specific embodiments of this application are described in detail below with reference to the accompanying drawings.

[0064] This application discloses a specific embodiment of an automatic hand-eye calibration method utilizing robot body features, the method flow is as follows: Figure 1As shown, the main steps include: selecting the linear geometric features of the robot body and establishing a 3D model; acquiring images and joint angle data under multiple configurations; extracting the linear projection features of the links in the images; establishing PnL (Perspective-n-Lines) projection constraints; solving the camera extrinsic parameters through parameter identification; and performing reprojection verification. This method can effectively utilize the robot's own geometric features as a 3D reference, requires no external calibration objects, and supports fully automated execution.

[0065] Specifically, an automatic hand-eye calibration method utilizing robot body features, such as... Figure 1 As shown, it includes the following steps:

[0066] S1: Select the straight line features of the robot body geometry and establish a three-dimensional parametric model of the straight line features.

[0067] S11: Selection of linear geometric features of the robot body

[0068] The feature utilized in this invention is the linear projection of the robot's links in a camera image. The selected geometric feature must meet the following conditions: it must have good linearity in three-dimensional space, allowing for accurate modeling by forward kinematics; its image projection must be clearly visible in the camera's field of view for most joint configurations within the robot's working range; the linearity of the image projection must be stable, unaffected by significant changes in joint motion or lighting; and it must have sufficient image length to ensure the accuracy of linear parameter estimation.

[0069] S12: 3D parametric modeling of linear features

[0070] For an n-DOF robot, the selected straight line feature, given the joint angle vector... Under the given conditions, the homogeneous coordinates of the 3D point corresponding to the parameterized coordinates s∈(0,1) on the straight line feature in the reference coordinate system are:

[0071] (1)

[0072] Where T(q) is the transformation matrix (4×4 homogeneous matrix) from the reference coordinate system to the local coordinate system of the selected linear feature, determined by forward kinematics; f(s) is the local coordinate parameterization function of the linear feature, usually in the form [L·s,0, 0, 1]ᵀ, where L is the nominal length of the linear feature. Uniform sampling along s∈(0,1) can generate a set of three-dimensional reference points along the linear feature.

[0073] S2: Synchronously acquire joint angle and image data.

[0074] The robot is controlled to move sequentially to several different joint configurations. In each configuration, camera images and encoder data of each joint of the robot are acquired simultaneously to obtain N frames of images and the corresponding joint angle matrix Q (N×n). The selected configurations should cover the typical poses in the robot's workspace, ensuring that the selected geometric features are visible in each frame and have sufficient length to provide adequate constraint excitation.

[0075] S3: Image line feature extraction.

[0076] like Figure 2 For each image frame, the projection line parameters of the selected geometric line features onto the image plane are extracted. The lines are represented using normalized homogeneous representation:

[0077] (2)

[0078] Where [u, v]ᵀ represents the image pixel coordinates. Normalization ensures that |au + bv + c| has the physical meaning of the distance from a point to a line pixel, which facilitates subsequent constraint construction and parameter identification.

[0079] The method for extracting line features is unrestricted and can include, but is not limited to, semantic line detection methods based on deep learning, traditional image processing methods based on edge detection and Hough transform, manual annotation methods, or combinations of the above methods, to adapt to engineering scenarios with different levels of automation and accuracy requirements. For the detected candidate lines, further quality evaluation and screening are performed, retaining only line features that meet preset quality conditions for subsequent identification.

[0080] The quality assessment may include one or more of the following indicators:

[0081] (a) The length of the line segment corresponding to the straight line is greater than a preset threshold to ensure that the feature has sufficient geometric stability;

[0082] (b) The ratio of the major axis to the minor axis of the fitted region is greater than a preset threshold to suppress the interference of short, thick, clumpy or nonlinear edges on the straight line fitting;

[0083] (c) The proportion of interior points obtained based on RANSAC or other robust fitting methods is greater than a preset threshold to ensure that the line has high fitting consistency.

[0084] (d) The consistency between the straight line direction and the projection direction of the target link meets the preset conditions to reduce the risk of false detection and mismatch.

[0085] S4: Establish PnL projection constraints.

[0086] Let T be the transformation matrix (extrinsic parameter) of the camera coordinate system relative to the reference coordinate system. CBIts rotation component is R, and its translation component is t.

[0087] S41: 3D point perspective projection

[0088] Projecting the 3D sampling point p(s) in the reference coordinate system onto the image plane yields the corresponding pixel coordinates:

[0089] (3)

[0090] Where K is the known camera intrinsic parameter matrix, and ~ represents the equation in the homogeneous sense.

[0091] S4: Calculate the projection error from point to line

[0092] Calculate the point-to-line distance error between the projected point and the corresponding image line l = [a, b, c]ᵀ:

[0093] (4)

[0094] Given N frames of images and M sampling points per frame, an overdetermined system of equations with N×M constraints can be established.

[0095] (5)

[0096] S5: External Parameter Identification

[0097] S51: External parameterization

[0098] To eliminate the orthogonality constraint of the rotation matrix, the problem is transformed into an unconstrained optimization problem, using rotation vectors (angular axis representation). Minimize the parameterization of the rotation matrix R:

[0099] (6)

[0100] Where Exp SO(3) (·) represents the exponential mapping of SO(3), , Combining the translation vector t, the extrinsic parameters are derived from a six-dimensional vector. Complete expression.

[0101] S52: Parameter Identification

[0102] Based on all the PnL constraints constructed by equation (5), the following parameter identification problem is established:

[0103] (7)

[0104] Where ρ(·) is the cost function, used to robustly process the parameter identification process, suppressing noise and interference from abnormal frames during line extraction. The cost function can take various forms, including but not limited to the squared loss function, Huber robust loss function, Cauchy loss function, etc. The solution method can employ nonlinear least squares methods (such as the Levenberg-Marquardt algorithm, Gauss-Newton method) or other applicable parameter identification methods. After convergence, the optimal parameters are obtained. Restore rotation matrix Finally, the extrinsic parameter matrix is ​​obtained:

[0105] (8)

[0106] S6: Result Verification

[0107] The obtained T CB Substituting the values ​​into the projection equation, the 3D sampling points of the straight line are reprojected back into each frame of the image. The pixel distance between the reprojected points and the labeled straight line is calculated and used as a quantitative verification metric. If the reprojection error is within an acceptable range, the calibration result is considered reliable.

[0108] This embodiment takes a serial robotic arm as an example to illustrate the above steps of the method in detail below.

[0109] (1) Camera installation and intrinsic parameter calibration. The camera is fixedly installed on the outside of the robot's workspace in an eye-to-hand manner. The installation position must ensure that, under the typical working configuration of the robot, the projected length of the selected link in the camera's field of view is not less than 15% of the image width. The camera intrinsic parameter matrix K is pre-calibrated using the standard checkerboard method before data acquisition. The calibration reprojection error should be better than 0.5 pixels.

[0110] (2) Selection of linear features. The forearm link is selected as the calibration feature, which satisfies good linearity and can be accurately modeled by DH positive kinematics. Alternatively, the upper arm link can be selected at the same time to increase constraint redundancy.

[0111] (3) Robot motion sequence planning. Control the robot to move sequentially to N≥15 different joint configurations. Requirements: the image projection direction of the selected link should cover an angle range of at least 120°; the change in joint angle between adjacent configurations should not be less than 5°. Simultaneously record camera images and encoder readings of each joint under each configuration.

[0112] (4) Image line feature extraction and quality screening. For each image frame, a pre-trained lightweight segmentation network is used to automatically segment the forearm link region. Based on the segmentation mask, normalized image line parameters are extracted through weighted RANSAC line fitting. The extraction results of each frame are evaluated for quality: the principal axis length of the mask is not less than 80 pixels, the aspect ratio is not less than 5, and the RANSAC inliquity ratio is not less than 0.4. Only frames that meet all three criteria are accepted; otherwise, the frames are discarded and do not participate in subsequent constraint construction.

[0113] (5) Three-dimensional sampling point modeling. Using DH forward kinematics, M = 20 points are uniformly sampled for the link within the range s∈[0.2, 0.8], and the three-dimensional coordinates p of each point in the robot base coordinate system are calculated. B (s).

[0114] (6) Parameter identification. The optimization problem shown in equation (7) is established based on the PnL constraints of all valid frames. Initial values ​​of extrinsic parameters: if there is prior installation information, it is set accordingly; otherwise, multiple sets of random initial values ​​are used to solve the problem, and the result with the smallest reprojection error is selected. The Huber robust cost function is used for robustness, and the Levenberg-Marquardt algorithm is used for iterative solution. The termination condition is that the L2 norm of the parameter update is less than 1×10⁻. 6 If the number of iterations exceeds 200, the extrinsic parameter matrix T is output after convergence. CB .

[0115] (7) Result verification. T CB Substitute into the projection equation and calculate the mean and standard deviation of the reprojection error of the link sampling points in all valid frames (e.g., Figure 3 (As shown). If the average reprojection error is less than 2 pixels, the calibration result is considered reliable; otherwise, check for systematic deviations, and if necessary, increase the number of acquisition frames or adjust the configuration distribution before recalibrating.

[0116] Another specific embodiment of this application provides an automatic hand-eye calibration system utilizing robot body features, including a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the steps in the method described above.

[0117] Another specific embodiment of this application provides a computer-readable medium storing a computer program, which, when executed by a computer, implements the steps in the method described above.

[0118] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of this application, and these all fall within the protection scope of this application.

Claims

1. A method for automatic hand-eye calibration utilizing robot body features, characterized in that, Includes the following steps: S1: Select the straight line geometric features of the robot body and establish a three-dimensional parametric model of the straight line features; S2: Control the robot to move sequentially to several different joint configurations, and simultaneously collect robot joint angle and linear feature image data during the process; S3: For each frame of the image data, extract the projection line parameters of the selected geometric line features onto the image plane; S4: Project the 3D sampling points in the reference coordinate system onto the image plane to obtain the corresponding pixel coordinates. Calculate the point-to-line distance error between the projected points and the corresponding image lines. With N frames of images and M sampling points per frame, establish an overdetermined system of N×M constraints as the PnL projection constraint. The transformation matrix of the camera coordinate system relative to the reference coordinate system is the unknown parameter to be solved. S5: Minimize the transformation matrix of the camera coordinate system relative to the reference coordinate system, and then identify the unknown parameters based on the PnL projection constraint to obtain the transformation matrix of the camera coordinate system relative to the reference coordinate system.

2. The automatic hand-eye calibration method utilizing robot body features according to claim 1, characterized in that, In step S1, the steps for establishing the three-dimensional parametric model are as follows: For an n-degree-of-freedom robot, the selected straight line feature, given the joint angle vector... Under the given conditions, the homogeneous coordinates of the 3D point corresponding to the parameterized coordinates s∈(0,1) on the straight line feature in the reference coordinate system are: (1) Where T(q) is the transformation matrix from the reference coordinate system to the local coordinate system of the selected straight line feature, determined by the forward kinematics; f(s) is the local coordinate parameterization function of the straight line feature, in the form [L·s, 0, 0, 1]ᵀ, where L is the nominal length of the straight line feature; Uniform sampling is performed on s∈(0,1) to generate a set of three-dimensional reference points along the linear feature.

3. The automatic hand-eye calibration method utilizing robot body features according to claim 1, characterized in that, In step S2, the robot is controlled to move to N≥15 different joint configurations, and the image projection direction of the selected straight line feature covers an angle range of at least 120°; the change in joint angle between adjacent configurations is not less than 5°.

4. The automatic hand-eye calibration method utilizing robot body features according to claim 1, characterized in that, In step S3, the projected line is represented using a normalized homogeneous representation: (2) Where [u, v]ᵀ represents the image pixel coordinates. .

5. The automatic hand-eye calibration method utilizing robot body features according to claim 4, characterized in that, Step S3 also includes the following steps: for the detected candidate lines, perform quality assessment and screening, and retain the line features that meet the preset quality conditions to participate in subsequent identification; The preset quality conditions for the quality assessment include one or more of the following indicators: (a) The length of the line segment corresponding to the straight line is greater than a preset threshold; (b) The ratio of the major axis to the minor axis of the fitted region is greater than the preset threshold; (c) The proportion of interior points obtained based on RANSAC or other robust fitting methods is greater than a preset threshold. (d) The consistency between the straight line direction and the projection direction of the target link meets the preset conditions.

6. The automatic hand-eye calibration method utilizing robot body features according to claim 4, characterized in that, In step S4, let T be the transformation matrix of the camera coordinate system relative to the reference coordinate system. CB Its rotation component is R, and its translation component is t; projecting the three-dimensional sampling point p(s) in the reference coordinate system onto the image plane, the corresponding pixel coordinates are expressed as: (3) Where K is a known camera intrinsic parameter matrix, and ~ represents an equation in a homogeneous sense; Calculate the point-to-line distance error between the projected point and the corresponding image line l = [a, b, c]ᵀ: (4) Given N frames of images and M sampling points per frame, an overdetermined system of equations with N×M constraints can be established as follows: (5) 。 7. The automatic hand-eye calibration method utilizing robot body features according to claim 6, characterized in that, In step S5, a rotation vector is used. Minimize the parameters of the rotation matrix R: (6) Where Exp SO(3) (·) represents the exponential mapping of SO(3), , Combined with the translation vector t, the transformation matrix of the camera coordinate system relative to the reference coordinate system consists of a six-dimensional vector. Complete expression; Based on all the PnL constraints constructed by equation (5), the following parameter identification problem is established: (7) Where ρ(·) is the cost function, and after convergence, the optimal parameters are obtained. Restore rotation matrix Finally, the transformation matrix of the camera coordinate system relative to the reference coordinate system is obtained: (8)。 8. The automatic hand-eye calibration method utilizing robot body features according to claim 7, characterized in that, It also includes step S6: obtaining T CB Substitute the points into the projection equation, reproject the three-dimensional sampling points of the line back into each frame image, and calculate the pixel distance between the reprojected points and the labeled line, which is used as a quantitative verification index. If the reprojection error is within the preset range, the calibration result is considered reliable; otherwise, recalibrate.

9. An automatic hand-eye calibration system utilizing robot body features, characterized in that, It includes a processor and a memory, the memory storing a computer program, which, when executed by the processor, implements the steps of the method as described in any one of claims 1-8.

10. A computer-readable medium, characterized in that, The computer-readable medium stores a computer program that, when executed by a computer, implements the steps of the method as described in any one of claims 1-8.

Images