Tail end structure for mechanical arm crack repair and high-precision calibration method

Abstract

The invention discloses a tail end structure for mechanical arm crack repairing and a high-precision calibration method, and belongs to the technical field of industrial robots and machine vision. Multiple sets of original pose data of hand-eye calibration are obtained, the multiple sets of original pose data are preprocessed, effective relative movement pairs are constructed and screened, and the mechanical arm crack repairing quality is improved. On the basis of obtaining an effective relative motion pair, solving an analysis initial value of hand-eye transformation and taking the analysis initial value as a starting point, nonlinear optimization is carried out to obtain a final high-precision calibration result, a globally reasonable initial solution is obtained and optimized, the optimization process is stable and reliable, the space aggregation error of a calibration plate in a base coordinate system is directly minimized, and the calibration accuracy is improved. The influence of noise on solution is effectively suppressed, the noise influence can be remarkably reduced, and a calibration result with higher precision than that of a single analytical method is obtained.

Description

Technical Field

[0001] This invention belongs to the field of industrial robot and machine vision technology, specifically relating to an end-effector structure and a high-precision calibration method for repairing cracks in robotic arms. Background Technology

[0002] With aging infrastructure, the use of robots for automated crack detection and repair has become an important development direction. Among them, high-precision end-point grouting devices are the core execution unit for achieving reliable repair operations. These devices typically integrate vision sensors to form a perception system that is either external to the hand or internal to the hand, thereby enabling real-time location of cracks and guiding grouting.

[0003] In vision-guided robotic tasks, accurate hand-eye calibration is the cornerstone of ensuring system operational accuracy. In vision-guided robot grasping and operating systems, to seamlessly integrate visual perception into robot control, two core spatial transformation relationships must be accurately established: first, tool center point calibration, which determines the pose of the end effector relative to the robot flange; and second, hand-eye calibration, which establishes the rigid transformation between the camera coordinate system and the robot coordinate system (end effector or base).

[0004] Traditional hand-eye calibration methods rely heavily on analytical algorithms, which, while providing rapid solutions, are extremely sensitive to noise in the collected data. This is especially true when the robot's range of motion is limited or its relative motion is small, leading to ill-conditioned equations and unstable calibration results with limited accuracy. While existing calibration techniques can achieve basic functionality, they still have significant limitations: traditional hand-eye calibration methods are highly sensitive to the geometric configuration of the robot's sampled motion. In practical, narrow or restricted working spaces (such as under bridges or tunnel sidewalls), robots often cannot perform ideal movements over a wide range and at multiple angles, resulting in ill-conditioned calibration equations and unstable, highly discrete solutions. This instability is amplified by subsequent visual positioning and robot kinematic chains, ultimately causing deviations in the grouting trajectory and affecting repair quality. More critically, mainstream methods only optimize the residuals of the motion equations in the intermediate processes, without directly minimizing the overall reprojection consistency of the calibration plate under the robot's base. Therefore, they struggle to meet the requirements for high-precision operations in terms of accuracy and robustness. Summary of the Invention

[0005] The purpose of this invention is to provide an end-effector structure and a high-precision calibration method for repairing cracks in robotic arms, so as to overcome the problems of noise sensitivity and unstable pose calculation in existing technologies.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A calibration method for the end-effector of a robotic arm for crack repair includes the following steps: S1, acquire multiple sets of original pose data for hand-eye calibration; S2, preprocess multiple sets of original pose data, construct and filter effective relative motion pairs; S3, based on obtaining effective relative motion pairs, solve for the analytical initial values ​​of hand-eye transformation; S4, starting with the analytical initial value, performs nonlinear optimization to obtain the final high-precision calibration result.

[0007] Preferably, the acquisition of multiple sets of original pose data for hand-eye calibration specifically includes: using the four-point method to calibrate the center point of the tool, controlling the robotic arm to move to different poses, and simultaneously performing two operations in each pose: reading the pose matrix of the wrist joint coordinate system relative to the robot base coordinate system from the robot controller and calculating the pose matrix of the current calibration board coordinate system relative to the camera coordinate system.

[0008] Preferably, the origin of the current tool coordinate system is aligned with the geometric center of the end flange, and the postures are consistent; a fixed reference point is selected in the workspace of the robotic arm; the robotic arm is controlled to make the gripper tip lightly touch the fixed point in sequence under at least four different postures, and the controller records the homogeneous transformation matrix of the wrist joint relative to the base coordinate system at each touch, forming four or more sets of reference postures.

[0009] Preferably, the preprocessing of multiple sets of original pose data to construct and screen effective relative motion pairs specifically includes: combining multiple sets of original pose data in pairs, calculating the relative motion of the robotic arm wrist joint and the relative motion of the calibration plate according to the AX=XB model, calculating the rotation angle of each motion pair, and retaining only motion pairs with rotation angles greater than a preset threshold.

[0010] Preferably, the step of solving the analytical initial value of the hand-eye transformation based on obtaining an effective relative motion pair specifically includes: using the Danilidis quaternion method to convert the rotation part of the motion pair into a quaternion and construct a system of equations, solving for the initial value of the rotation matrix through singular value decomposition, and then solving for the initial value of the translation vector based on this rotation by constructing a linear least squares equation.

[0011] Preferably, starting with the analytical initial value, nonlinear optimization is performed to obtain the final high-precision calibration result. Specifically, this includes: using Nelder-Mead nonlinear optimization based on geometric consistency for fine solution, directly minimizing the sum of the position variance and attitude angle variance of the calibration board in the robot base coordinate system at all sampling times; using the parameter vector corresponding to the analytical initial solution as the initial value, the Nelder-Mead algorithm is used to iteratively minimize the objective function until convergence, and finally the optimized parameter vector is transformed back into a homogeneous transformation matrix to obtain the high-precision final hand-eye extrinsic parameter matrix.

[0012] This invention provides an end-effector structure for repairing cracks in a robotic arm, comprising a robotic arm, an end-effector structure, an RGB-D depth camera, an asymmetric circular array calibration plate, and a control unit; the end-effector structure includes a flange connector, an adapter flange connector, a crack filling and injection connector, a Gemini2 camera, and a crack filling and injection device. The Gemini2 camera is rigidly mounted on the end flange of the robotic arm via the flange connector and the adapter flange connector, so that the optical center of the Gemini2 camera and the wrist joint of the robotic arm form a fixed eye-on-hand relationship. The crack filling and injection device is rigidly connected to the end of the robotic arm via the crack filling and injection device connector, and is used to perform crack filling and injection operations. The asymmetric circular array calibration plate is fixedly placed in the workspace of the robotic arm to provide a unique corner point sorting and calibration plate coordinate system. The control unit includes a data processing system and a robot controller. The robot controller is connected to the robotic arm, the end effector, and an RGB-D camera to control the movement of the robotic arm and collect pose and visual data. The data processing system is used to calibrate the end effector for repairing cracks in the robotic arm.

[0013] Preferably, the data processing system includes: The tool center point calibration module is used to calibrate the center point of the tool at the end of the robotic arm using the four-point method, so as to obtain the pose of the tool coordinate system relative to the wrist joint coordinate system. The calibration board pose solving module is used to detect calibration board features and solve the calibration board pose matrix in the camera coordinate system based on the image and depth data acquired by the RGB-D camera. The calibration data acquisition and pairing module is used to simultaneously acquire the pose matrix from the base to the wrist joint and the pose matrix from the camera to the calibration board under multiple sets of robotic arm sampling postures, and construct relative motion pairs. The hand-eye analytical solution module is used to construct a hand-eye calibration model based on the relative motion pair and use singular value decomposition to obtain the initial hand-eye transformation matrix of the camera relative to the wrist joint, that is, to solve the analytical initial value of the hand-eye transformation. The hand-eye nonlinear optimization module is used to parameterize the initial hand-eye transformation matrix into a pose vector, construct a joint objective function of position error and attitude error, use a derivative-free optimization algorithm to globally refine the hand-eye transformation, and output the final hand-eye calibration result.

[0014] Preferably, the asymmetric circular array calibration plate is arranged with a dot array of unequal row and column spacing or with a unique origin marker.

[0015] Preferably, the calibration board pose solving module is used to perform grayscale conversion and adaptive threshold binarization on the RGB image to obtain a binary image with enhanced contrast; a circular feature detection algorithm is used to extract candidate center points, and asymmetric geometric constraints are used to uniquely sort the points; sub-pixel corner point refinement processing is performed on the sorted center points; and the pose matrix of the calibration board coordinate system relative to the camera coordinate system is solved by combining the pre-calibrated three-dimensional coordinates of the calibration board and the camera intrinsic parameter matrix using the PnP or IPPE algorithm based on planar targets. The calibration board pose solving module also includes a 3D fine-solution unit based on the depth map; the 3D fine-solution unit is used to: extract the depth data of the calibration board region in the corresponding depth map and generate a local point cloud based on the circumscribed polygon region of the calibration board in the image; use the pose obtained by the PnP or IPPE algorithm as the initial value to perform ICP registration on the local point cloud and the 3D model of the calibration board; when the inlier ratio and mean square error of the ICP registration meet the preset threshold, use the rigid body transformation output by ICP as the fine pose estimation of the calibration board in the camera coordinate system.

[0016] Compared with the prior art, the present invention has the following beneficial technical effects: This invention provides a calibration method for the end-effector of a robotic arm for crack repair. It acquires multiple sets of original pose data from hand-eye calibration, preprocesses these data, constructs and filters effective relative motion pairs, and solves for the analytical initial values ​​of the hand-eye transformation based on these pairs. Starting from these analytical initial values, nonlinear optimization is performed to obtain the final high-precision calibration result. By obtaining a globally reasonable initial solution and optimizing it, the optimization process is stable and reliable, directly minimizing the spatial clustering error of the calibration plate in the base coordinate system, effectively suppressing the influence of noise on the solution, significantly reducing noise impact, and obtaining calibration results with higher precision than a single analytical method.

[0017] Preferably, by filtering out relatively motion pairs with excessively small rotation angles, numerical instability caused by pathological samples is avoided; the calibration accuracy is high and the repeatability is good: in actual measurements, the reprojection position error of the calibration plate in the base coordinate system can be controlled at the sub-millimeter level, and the attitude error is within a small angle range, which meets the requirements of high-precision visual guidance tasks.

[0018] Preferably, by disrupting the pattern symmetry, ambiguity of corner indexes is avoided, and jumps in pose solutions during sampling are eliminated, thus improving data consistency. A two-stage strategy of analytical initialization and nonlinear optimization is adopted: in the analytical stage, a globally reasonable initial solution is obtained through the Danilidis quaternion method; in the nonlinear optimization stage, the Nelder-Mead algorithm does not require explicit gradient calculation, is suitable for objective functions with complex rotation operations and reprojection errors, and the optimization process is stable and reliable. It directly minimizes the spatial clustering error of the calibration plate in the base coordinate system, effectively suppresses the influence of noise on the solution, and can significantly reduce the influence of noise, obtaining calibration results with higher accuracy than the single analytical method. Attached Figure Description

[0019] Figure 1 This is a flowchart of the calibration method in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the end structure for repairing cracks in a robotic arm according to an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the asymmetric circular calibration plate structure used in an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of robot multi-pose sampling during the data acquisition stage in an embodiment of the present invention.

[0023] Figure 5 This is a flowchart of the two-stage error analysis process in an embodiment of the present invention.

[0024] Figure 6 This is a schematic diagram illustrating the calibration effect evaluation in an embodiment of the present invention.

[0025] In the diagram: 1 - Flange connector; 2 - Adapter flange connector; 3 - Crack filler connector; 4 - Gemini2 camera; 5 - Crack filler. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] like Figure 1 As shown, the present invention provides a calibration method for the end-effector structure used for repairing cracks in a robotic arm, comprising the following steps: S1, acquire multiple sets of original pose data for hand-eye calibration.

[0029] In a specific embodiment of the present invention, the tool center point (TCP) is obtained using the four-point calibration method to ensure that the robot's kinematics calculations are referenced to the accurate end-effector point. The robotic arm is controlled to move to different poses, and two operations are performed simultaneously in each pose: reading the pose matrix of the wrist joint coordinate system relative to the robot base coordinate system from the robot controller and calculating the pose matrix of the current calibration board coordinate system relative to the camera coordinate system.

[0030] S2 preprocesses multiple sets of original pose data to construct and filter effective relative motion pairs.

[0031] In a specific embodiment of the present invention, the method for preprocessing multiple sets of original pose data to construct and screen effective relative motion pairs is as follows: Multiple sets of acquired original pose data are combined in pairs, and the relative motion of the robotic arm wrist joint and the relative motion of the calibration plate are calculated according to the AX=XB model. The rotation angle of each motion pair is calculated, and only motion pairs with rotation angles greater than a preset threshold are retained; this operation aims to eliminate invalid data pairs that are approximately collinear or stationary, which would lead to ill-conditioned equations.

[0032] S3, based on obtaining effective relative motion pairs, solve for the analytical initial values ​​of hand-eye transformation.

[0033] Based on obtaining effective relative motion pairs, the analytical initial values ​​of the hand-eye transformation are solved. Specifically, the Danilidis quaternion method is used to first convert the rotation part of the motion pair into quaternions and construct a system of equations. The initial values ​​of the rotation matrix are solved by singular value decomposition (SVD). Based on this rotation, the initial values ​​of the translation vector are solved by constructing linear least squares equations.

[0034] S4, starting with the analytical initial value, performs nonlinear optimization to obtain the final high-precision calibration result.

[0035] Starting with analytical initial values, nonlinear optimization is performed to obtain the final high-precision calibration result. Specifically, this includes: using Nelder-Mead nonlinear optimization based on geometric consistency for fine solution, directly minimizing the sum of the position variance and attitude angle variance of the calibration board in the robot base coordinate system at all sampling times; during optimization, the parameter vector corresponding to the analytical initial solution is used as the initial value, and the Nelder-Mead algorithm is used to iteratively minimize the objective function until convergence. Finally, the optimized parameter vector is transformed back into a homogeneous transformation matrix to obtain the high-precision final hand-eye extrinsic parameter matrix.

[0036] In a specific embodiment of the present invention, the end-effector structure for repairing cracks in a robotic arm is specifically as follows: Figure 2 As shown, it includes a robotic arm, an end effector, an RGB-D depth camera, an asymmetric circular array calibration plate, and a control unit. The end effector includes a flange connector 1, a transition flange connector 2, a gap filler connector 3, a Gemini2 camera 4, and a gap filler 5. The Gemini2 camera 4 is rigidly mounted on the end flange of the robotic arm through the flange connector 1 and the transition flange connector 3, so that the optical center of the Gemini2 camera 4 and the wrist joint of the robotic arm form a fixed eye-on-hand relationship. The crack filling and injection device 5 is rigidly connected to the end of the robotic arm via the crack filling and injection device connector 3, and is used to perform crack filling and injection operations. like Figure 3 As shown, the asymmetric circular array calibration plate is fixedly placed in the workspace of the robotic arm to provide a unique corner point sorting and calibration plate coordinate system; The control unit includes a data processing system and a robot controller. The robot controller is connected to the robotic arm, the end effector, and an RGB-D camera to control the movement of the robotic arm and collect pose and visual data. The data processing system includes: The tool center point calibration module is used to calibrate the center point of the tool at the end of the robotic arm using the four-point method, so as to obtain the pose of the tool coordinate system relative to the wrist joint coordinate system. The calibration board pose solving module is used to detect calibration board features and solve the calibration board pose matrix in the camera coordinate system based on the image and depth data acquired by the RGB-D camera. The calibration data acquisition and pairing module is used to simultaneously acquire the pose matrix from the base to the wrist joint and the pose matrix from the camera to the calibration board under multiple sets of robotic arm sampling postures, and construct relative motion pairs. The hand-eye analytical solution module is used to construct a hand-eye calibration model based on the relative motion pair and use singular value decomposition to obtain the initial hand-eye transformation matrix of the camera relative to the wrist joint, that is, to solve the analytical initial value of the hand-eye transformation. The hand-eye nonlinear optimization module is used to parameterize the initial hand-eye transformation matrix into a pose vector, construct a joint objective function of position error and attitude error, use a derivative-free optimization algorithm to globally refine the hand-eye transformation, and output the final hand-eye calibration result.

[0037] In a specific embodiment of the present invention, the asymmetric circular array calibration plate is arranged with a circular array of unequal row and column spacing or with a unique origin mark; The calibration board pose solving module is specifically used for: converting the RGB image to grayscale and adaptive threshold binarization to obtain a binary image with enhanced contrast; extracting candidate center points using a circular feature detection algorithm and uniquely sorting the points using asymmetric geometric constraints; refining the sorted center points to sub-pixel corner points; and solving the pose matrix of the calibration board coordinate system relative to the camera coordinate system using a planar target-based PnP or IPPE algorithm, combined with the pre-calibrated three-dimensional coordinates of the calibration board and the camera intrinsic parameter matrix.

[0038] The calibration board pose solving module also includes a three-dimensional fine solving unit based on the depth map; The three-dimensional fine-solution unit is used to: extract the depth data of the calibration board region in the corresponding depth map and generate a local point cloud based on the circumscribed polygon region of the calibration board in the image; use the pose obtained by the PnP or IPPE algorithm as the initial value to perform ICP registration on the local point cloud and the three-dimensional model of the calibration board; when the inlier ratio and mean square error of the ICP registration meet the preset threshold, use the rigid body transformation output by ICP as the fine pose estimation of the calibration board in the camera coordinate system.

[0039] The tool center point calibration module is used to perform the following four-point calibration process; Within the robotic arm's workspace, a fixed spatial point is selected as the tool calibration reference point. The robotic arm drives the end effector to sequentially contact the fixed spatial point from at least four directions with significantly different postures. Each contact records the corresponding pose matrix from the base to the wrist joint. Each pose matrix is ​​then decomposed into a rotation matrix. Translation vector For any two samples i and j, construct linear constraint equations and stack all constraints to form an overdetermined linear equation system; use the least squares method to solve the three-dimensional position vector p of the tool center point relative to the wrist joint coordinate system, and write the three-dimensional position vector into the tool coordinate system configuration of the robot controller.

[0040] The calibration data acquisition and pairing module is used to: record the pose matrix from the base to the wrist joint and the pose matrix from the camera to the calibration board obtained through image processing in each sampling posture; construct the relative motion matrix of the wrist joint and the relative motion matrix of the calibration board by pairwise combinations for all sampling postures, calculate the rotation angle of each relative motion, and only when the rotation angle is greater than a preset threshold, pair the corresponding... This effective relative motion pair is used for subsequent hand-eye analytical solutions.

[0041] The hand-eye nonlinear optimization module is used to: in the nonlinear optimization stage, first express the hand-eye transformation matrix to be determined as a six-dimensional parameter vector containing translation and rotation information. Based on this parameter and each set of sampled data, calculate the predicted position and predicted attitude of the calibration board in the robot base coordinate system through a coordinate transformation chain.

[0042] Subsequently, an objective function is constructed to directly measure the consistency of all prediction results in the base coordinate system. This function consists of two parts: a position error term, which is the average distance between all predicted position points and their geometric centers; and an attitude error term, which is the average angular deviation between all predicted attitudes and their average attitude. The two errors are weighted by an adjustable weighting coefficient to form a comprehensive evaluation index.

[0043] Finally, the Nelder-Mead simplex algorithm is used to iteratively optimize the objective function in the six-dimensional parameter space. The algorithm automatically adjusts the parameters to minimize the objective function until convergence. The parameters obtained at this point correspond to the optimal high-precision hand-eye transformation matrix that best ensures geometric consistency across all observations.

[0044] In a specific embodiment of the present invention, the specific equipment installation and operation steps are as follows: Preparations before hand-eye calibration: System installation and initial configuration: In this application, the FAIRINO FR5 six-axis collaborative robotic arm is selected as the core actuator; its end effector is equipped with a crack filling and glue injector via a flange adapter plate, serving as a tool for directly performing crack filling; at the same time, the Orbbec Gemini2 RGB-D camera is selected as the vision sensing unit, and is securely mounted above the end effector gripper via a rigid mechanical bracket, thus forming an integrated motion unit with the end effector of the robotic arm, creating a typical "eye in hand" system configuration.

[0045] TCP tool center point identification: like Figure 4 As shown, the drive collaborative robotic arm moves the end effector in at least four directions with significantly different postures, so that the actual working point of the electric parallel gripper (e.g., the midpoint between the two gripper fingers or the tip of a custom tool) successively touches the same fixed reference point.

[0046] Specifically, in the robot controller, the current tool coordinate system is temporarily set to the "wrist joint coordinate system," meaning the origin of the tool coordinate system coincides with the geometric center of the end flange, and their postures are consistent. A fixed reference point is selected in the robot arm's workspace. The robot arm is controlled to sequentially touch this fixed point with the gripper tip in at least four different postures. Each time a touch occurs, the controller records the homogeneous transformation matrix of the wrist joint relative to the base coordinate system. This forms four or more sets of reference attitudes; the controller internally uses the four-point method and equations: (1) Establish linear constraints and rearrange them as follows: (2) The least squares solution is performed on all point pairs to obtain the optimal estimate of the tool center point p in the wrist joint coordinate system; the obtained TCP translation + rotation parameters are written into the controller tool coordinate system configuration so that the subsequent robot kinematics solution uses this TCP as the end reference point.

[0047] Hand-eye calibration scenario and calibration board preparation: An asymmetric circular array calibration board is used as the hand-eye calibration board. The hand-eye calibration board has different row and column spacing or a unique origin mark to eliminate ambiguity in corner point sorting. The hand-eye calibration board is fixed in the working space of the robotic arm, so that the board surface is parallel to the worktable and located within the visible range of the robot's end effector. Using the camera configuration file, the exposure, gain, white balance, etc. are pre-adjusted to make the circular dots on the calibration board have high contrast and clear edges, thus ensuring the quality of corner point detection.

[0048] Hand-eye calibration process: (1) Collect hand-eye positioning posture data: For each sampled pose i, perform the following operations: control the robotic arm to move to a certain pose so that the calibration plate is completely within the camera's field of view; read the pose matrix of the wrist joint relative to the base coordinate system from the robot controller, denoted as . The image is then added to the wrist joint pose set; the current color image and depth map are acquired simultaneously, and the calibration board detection module is invoked: adaptive thresholding and morphological processing are performed on the color image; circular mesh detection and sub-pixel corner optimization algorithms are used to obtain the pixel coordinates of the calibration board corners; combined with the known three-dimensional geometric coordinates on the calibration board, IPPE+PnP pose calculation is used to obtain the transformation matrix of the calibration board coordinate system relative to the camera coordinate system. The reprojection error is calculated; a local point cloud is constructed in the calibration board area using the depth map, and the pose is finely corrected using the ICP algorithm; when the detection is successful and the reprojection error is below a preset threshold, Store the calibration plate pose set; repeat the above process to collect no fewer than a number of sets (e.g., 10 to 20 sets) of pose samples covering different translations and rotations.

[0049] (2) Data preprocessing and motion pair construction: By combining all the acquired pose samples in pairs, the relative motion of the wrist joint and the relative motion of the calibration plate are constructed: (3) (4) For each relative motion Calculate the rotation angle. If the rotation angle is less than the preset threshold, the relative motion is considered too small, which may cause ill-conditioned equations, and the pair is discarded. Only the motion pairs with rotation angles greater than the preset threshold are retained for subsequent solutions, so as to exclude approximately static or ill-conditioned sample pairs, thereby improving the numerical conditions and enhancing the stability of the solution.

[0050] (3) Analytical initial value solution based on Danilidis quaternion method This step aims to obtain an initial solution that is close to the truth, laying the foundation for subsequent optimization.

[0051] Each , The rotating parts are respectively denoted as , The translated part is denoted as , Under the following model: (5) The rotating part satisfies: (6) Will , Convert each equation to a quaternion and establish a homogeneous linear system of equations in the following form: (7) Where L and R are the left-multiplication matrix and the right-multiplication matrix of the quaternions, respectively. Let K be the quaternion of the camera rotation relative to the wrist joint; stack all samples to form a large matrix K, perform singular value decomposition (SVD) on K, and take the singular vector corresponding to the minimum singular value as... The approximate solution is obtained and normalized to obtain the rotation matrix. In the translation part, construct the following linear equation: (8) All samples are superimposed to form A and b. The translation vector of the camera optical center in the wrist joint coordinate system is solved using the least squares method. ;Will ;and Combining these, we obtain the initial analytical solution for the hand-eye transformation: (9) This matrix serves as the initial solution for nonlinear optimization. Since SVD is sensitive to noise and ill-conditioned data, this invention explicitly positions it as a high-quality initial value, rather than the final calibration result.

[0052] (4) Refined solution of Nelder-Mead nonlinear optimization based on geometric consistency: This step is the core of improving calibration accuracy. The initial solution is optimized by directly minimizing the global geometric error to obtain the final high-precision hand-eye extrinsic parameter matrix.

[0053] like Figure 5 As shown, to further improve the glue alignment accuracy and trajectory consistency of crack filling operations, this invention adds a tool error optimization method and a hand-eye accuracy optimization method to the existing "four-point TCP calibration + two-stage hand-eye calibration (quaternion SVD analytical initial value + Nelder-Mead geometric consistency optimization)" method, and couples the two for the final crack filling accuracy improvement: After the tool center point calibration is completed, the actual glue dispensing point of the crack filling glue injector is used as a constraint, and a "predicted end contact point - actual contact point" residual model is constructed through multi-pose repeated contact point verification sampling. The translation and attitude bias of TCP are parameterized into pose vectors, and derivative-free optimization or least squares is used to refine TCP twice to obtain a tool coordinate system that is closer to the actual glue dispensing point. In the hand-eye calibration stage, in addition to the analytical initial solution, the overall consistency of the position and attitude of the calibration plate in the robot base coordinate system (reprojection pose aggregation error) is further refined globally with the goal of "the overall consistency of the position and attitude of the calibration plate in the robot base coordinate system (reprojection pose aggregation error)". The effective motion is combined with screening and external point elimination to suppress the noise amplification effect. Finally, the optimized hand-eye extrinsic parameter matrix and the optimized TCP parameters are written into the control system. When applying crack filling, the pose of the crack centerline / target point extracted by RGB-D vision is stably transformed to the robot base coordinate system. Error compensation and unified coordinate reference constraints are applied to the trajectory of the end glue outlet. This significantly reduces the accumulation of "visual measurement error - coordinate transformation error - tool installation error", thereby improving the accuracy of crack filling alignment, the consistency of path tracking and the repeatability of the operation.

[0054] This invention constructs a geometrically consistent objective function based on analytical initial values ​​and uses the Nelder–Mead derivative-free optimization algorithm for a refined solution.

[0055] For each set of sampled data i, calculate the predicted pose of the calibration plate in the robot base coordinate system based on the current parameter p: Convert to a six-dimensional pose vector p (translation + rotation vector); for each set of sampled data i, use the current parameter p to calculate the predicted pose of the calibration plate in the robot base coordinate system: (10) Extract the predicted position and predicted attitude from them; construct the comprehensive objective function: (11) Let λ be the angle deviation between the i-th predicted posture and the average posture, and let λ be the weighting coefficient of the equilibrium position and posture error. The Nelder–Mead simplex derivativeless optimization algorithm is adopted, with the initial value of p as the parameter p0 corresponding to the analytical solution, and E(p) is iteratively minimized until the convergence condition is met. The optimized parameter p is transformed back into a homogeneous transformation matrix to obtain the final hand-eye extrinsic parameter matrix.

[0056] The objective function directly measures the consistency of the reprojected poses of all calibration plates in the robot base coordinate system: the position error term reflects the degree of clustering of spatial positions, and the attitude error term reflects the consistency of attitude orientation.

[0057] Results Verification and Application: Using the final hand-eye extrinsic parameters, the pose of the calibration board was predicted at all sampling times. The position and attitude variances were statistically analyzed to verify the calibration accuracy; after passing the evaluation, Write it into the robot control system or host computer configuration file as the basis for coordinate transformation in subsequent vision-guided tasks.

[0058] This invention implements all steps from four-point TCP calibration to eye-to-hand hand-eye calibration within the same software framework, facilitating engineering integration and maintenance. An asymmetric circular calibration plate enhances robustness: by disrupting pattern symmetry, it avoids ambiguity in corner indexes, eliminates jumps in pose solutions during sampling, and improves data consistency. A two-stage strategy of analytical initialization followed by nonlinear optimization is employed: the analytical stage uses the Danilidis quaternion method to obtain a globally reasonable initial solution, while the nonlinear optimization stage utilizes the Nelder-Mead algorithm, which eliminates the need for explicit gradient calculation and is suitable for targets with complex rotation operations and reprojection errors. The function is stable and reliable in the optimization process. It directly minimizes the spatial clustering error of the calibration plate in the base coordinate system, effectively suppresses the influence of noise on the solution, and can significantly reduce the influence of noise, thus obtaining a calibration result with higher accuracy than the single analytical method. A motion amplitude screening mechanism is introduced: by filtering relative motion pairs with too small rotation angles, the condition number of equation (3) is improved, and numerical instability caused by ill-conditioned samples is avoided. The calibration accuracy is high and the repeatability is good: in actual measurement, the reprojection position error of the calibration plate in the base coordinate system can be controlled at the sub-millimeter level, and the attitude error is within a small angle range, which meets the requirements of high-precision visual guidance tasks.

[0059] like Figure 6 As shown, using the final hand-eye extrinsic parameters, the predicted pose of the calibration plate in the base coordinate system at all sampling times i is statistically analyzed: the mean and standard deviation of all predicted position points are calculated to obtain the mean and maximum position error; with the average pose as the reference, the angle deviation of each predicted pose is calculated to obtain the mean and maximum pose error.

[0060] In practical applications, the average position error can be controlled to the sub-millimeter level, and the attitude error can be kept within a small angular range, fully meeting the requirements of high-precision vision-guided grasping, assembly, or measurement tasks.

[0061] The final hand-eye extrinsic parameter matrix is ​​stored in the configuration file. In subsequent vision-guided tasks, the pose of the target object detected by the camera in the camera coordinate system can be transformed to the robot's base coordinate system through this matrix and the robot's forward kinematics chain, providing a unified and stable spatial reference for path planning and grasping execution.

[0062] The present invention proposes an RGB-D dual-constraint hand-eye calibration method based on asymmetric circular array guidance, which can achieve high-precision, robust, and fully automatic solution of hand-eye extrinsic parameters on a collaborative robot platform. The TCP tool center point is calibrated using a four-point method to ensure the accuracy of the end-effector coordinate system. An asymmetric circular array calibration board is used to completely eliminate the corner point sorting ambiguity problem caused by traditional symmetrical calibration boards under large viewing angles and strong perspective distortion conditions, ensuring that the calibration board pose estimation remains stable and consistent throughout multi-pose sampling. Combined with an RGB-D camera, IPPE+PnP, and optional ICP fine registration, high-precision 6D pose calculation of the calibration board in the camera coordinate system is achieved, significantly reducing the impact of visual noise on subsequent solutions. In the hand-eye solution stage, an SVD analytical algorithm based on the Danilidis quaternion method is first used to quickly obtain a globally reasonable initial solution. Then, a geometric error function is constructed with the goal of ensuring consistency between the reprojected position and pose of the calibration board in the robot base coordinate system. Nelder-Mead derivative-free optimization is used for global refinement, effectively overcoming the shortcomings of traditional analytical methods that are sensitive to noise and ill-conditioned motion data and cannot guarantee global optimality. The system boasts a highly automated calibration process and a high degree of software and hardware integration. It can be quickly deployed and reused on-site. The calibration results outperform traditional methods in terms of positional accuracy, posture stability, and repeatability. It can provide a unified and reliable spatial coordinate benchmark for various robot applications such as vision-guided grasping, assembly, and measurement, and has strong engineering practical value and promising prospects for promotion.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that various modifications, substitutions, or equivalent improvements can be made to its specific structural form, algorithm flow, parameter configuration, etc., without departing from the spirit and essence of the present invention, and all such modifications or modifications should fall within the protection scope defined by the appended claims.

Claims

1. A calibration method for the end-effector structure used in repairing cracks in a robotic arm, characterized in that, Includes the following steps: S1, acquire multiple sets of original pose data for hand-eye calibration; S2, preprocess multiple sets of original pose data, construct and filter effective relative motion pairs; S3, based on obtaining effective relative motion pairs, solve for the analytical initial values ​​of hand-eye transformation; S4, starting with the analytical initial value, performs nonlinear optimization to obtain the final high-precision calibration result.

2. The calibration method for the end-effector structure used for repairing cracks in a robotic arm according to claim 1, characterized in that, The acquisition of multiple sets of original pose data for hand-eye calibration specifically includes: using the four-point method to calibrate the center point of the tool, controlling the robotic arm to move to different poses, and simultaneously performing two operations in each pose: reading the pose matrix of the wrist joint coordinate system relative to the robot base coordinate system from the robot controller and calculating the pose matrix of the current calibration board coordinate system relative to the camera coordinate system.

3. The calibration method for the end-effector structure used for repairing cracks in a robotic arm according to claim 2, characterized in that, Align the origin of the current tool coordinate system with the geometric center of the end flange, ensuring consistent orientation; select a fixed reference point in the workspace of the robotic arm; control the robotic arm in at least four different orientations, allowing the gripper tip to lightly touch the fixed point in sequence, with the controller recording the homogeneous transformation matrix of the wrist joint relative to the base coordinate system each time a touch occurs, forming four or more sets of reference orientations.

4. The calibration method for the end-effector structure used for repairing cracks in a robotic arm according to claim 1, characterized in that, The preprocessing of multiple sets of original pose data to construct and screen effective relative motion pairs specifically includes: combining multiple sets of original pose data in pairs, calculating the relative motion of the robotic arm wrist joint and the relative motion of the calibration plate according to the AX=XB model, calculating the rotation angle of each motion pair, and retaining only motion pairs with rotation angles greater than a preset threshold.

5. The calibration method for the end-effector structure used for repairing cracks in a robotic arm according to claim 1, characterized in that, The method of obtaining effective relative motion pairs and solving for the analytical initial values ​​of hand-eye transformation specifically includes: using the Danilidis quaternion method to convert the rotation part of the motion pair into quaternions and construct a system of equations; solving for the initial value of the rotation matrix through singular value decomposition; and then solving for the initial value of the translation vector based on this rotation by constructing a linear least squares equation.

6. The calibration method for the end-effector structure used for repairing cracks in a robotic arm according to claim 1, characterized in that, Starting with analytical initial values, nonlinear optimization is performed to obtain the final high-precision calibration result. Specifically, this includes: using Nelder-Mead nonlinear optimization based on geometric consistency for fine solution, directly minimizing the sum of the position variance and attitude angle variance of the calibration board in the robot base coordinate system at all sampling times; using the parameter vector corresponding to the analytical initial solution as the initial value, the Nelder-Mead algorithm is used to iteratively minimize the objective function until convergence, and finally the optimized parameter vector is transformed back into a homogeneous transformation matrix to obtain the high-precision final hand-eye extrinsic parameter matrix.

7. The end structure for repairing cracks in a robotic arm according to claim 1, characterized in that, It includes a robotic arm, an end effector, an RGB-D depth camera, an asymmetric circular array calibration plate, and a control unit; the end effector includes a flange connector, an adapter flange connector, a crack filler connector, a Gemini2 camera, and a crack filler. The Gemini2 camera is rigidly mounted on the end flange of the robotic arm via the flange connector and the adapter flange connector, so that the optical center of the Gemini2 camera and the wrist joint of the robotic arm form a fixed eye-on-hand relationship. The crack filling and injection device is rigidly connected to the end of the robotic arm via the crack filling and injection device connector, and is used to perform crack filling and injection operations; The asymmetric circular array calibration plate is fixedly placed in the workspace of the robotic arm to provide a unique corner point sorting and calibration plate coordinate system. The control unit includes a data processing system and a robot controller. The robot controller is connected to the robotic arm, the end effector, and an RGB-D camera to control the movement of the robotic arm and collect pose and visual data. The data processing system is used to calibrate the end effector for repairing cracks in the robotic arm.

8. The end structure for repairing cracks in a robotic arm according to claim 7, characterized in that, The data processing system includes: The tool center point calibration module is used to calibrate the center point of the tool at the end of the robotic arm using the four-point method, so as to obtain the pose of the tool coordinate system relative to the wrist joint coordinate system. The calibration board pose solving module is used to detect calibration board features and solve the calibration board pose matrix in the camera coordinate system based on the image and depth data acquired by the RGB-D camera. The calibration data acquisition and pairing module is used to simultaneously acquire the pose matrix from the base to the wrist joint and the pose matrix from the camera to the calibration board under multiple sets of robotic arm sampling postures, and construct relative motion pairs. The hand-eye analytical solution module is used to construct a hand-eye calibration model based on the relative motion pair and use singular value decomposition to obtain the initial hand-eye transformation matrix of the camera relative to the wrist joint, that is, to solve the analytical initial value of the hand-eye transformation. The hand-eye nonlinear optimization module is used to parameterize the initial hand-eye transformation matrix into a pose vector, construct a joint objective function of position error and attitude error, use a derivative-free optimization algorithm to globally refine the hand-eye transformation, and output the final hand-eye calibration result.

9. The end structure for repairing cracks in a robotic arm according to claim 8, characterized in that, The asymmetric circular array calibration plate is arranged with an array of dots with unequal row and column spacing or with a unique origin marker.

10. The end structure for repairing cracks in a robotic arm according to claim 8, characterized in that, The calibration board pose solving module is used to perform grayscale conversion and adaptive threshold binarization on the RGB image to obtain a binary image with enhanced contrast; a circular feature detection algorithm is used to extract candidate center points, and asymmetric geometric constraints are used to uniquely sort the points; sub-pixel corner point refinement processing is performed on the sorted center points; combined with the pre-calibrated calibration board 3D coordinates and camera intrinsic parameter matrix, the pose matrix of the calibration board coordinate system relative to the camera coordinate system is solved using the PnP or IPPE algorithm based on planar targets; The calibration board pose solving module also includes a 3D fine-solution unit based on the depth map; the 3D fine-solution unit is used to: extract the depth data of the calibration board region in the corresponding depth map and generate a local point cloud based on the circumscribed polygon region of the calibration board in the image; use the pose obtained by the PnP or IPPE algorithm as the initial value to perform ICP registration on the local point cloud and the 3D model of the calibration board; when the inlier ratio and mean square error of the ICP registration meet the preset threshold, use the rigid body transformation output by ICP as the fine pose estimation of the calibration board in the camera coordinate system.

Images