An eye-hand calibration optimization method and system suitable for a robot welding scene

Abstract

This invention discloses a hand-eye calibration optimization method and system adapted for robotic welding scenarios, relating to the field of robot vision calibration. It decomposes hand-eye calibration into independent position and attitude measurements: First, the end effector physically touches the center of the calibration block to obtain the reference point position; then, based on point cloud processing technology, the camera attitude is actively adjusted so that its Z-axis is collinear with the calibration block normal and its X-axis is collinear with the calibration block feature direction. Under this standard attitude, the point cloud center and the calibration block center are aligned to obtain the working point pose; finally, displacement and attitude parameters are calculated, and tool coordinate system parameters are directly output for use by the robot controller. This invention replaces visual translation calculations with physical interaction and numerical optimization for rotation calculations with active attitude adjustment, significantly reducing the technical threshold and application complexity of hand-eye calibration in welding scenarios, avoiding complex matrix transformations, and improving the intuitiveness and reliability of the calibration process.

Description

Technical Field

[0001] This invention relates to the field of hand-eye calibration technology, and in particular to a hand-eye calibration optimization method and system adapted to robotic welding scenarios. Background Technology

[0002] Hand-eye calibration is a core component of a robot system. It is used to determine the spatial pose relationship between the robot's end effector and the vision sensor. It is a key prerequisite for precise operation guided by robot vision and is widely used in industrial scenarios such as welding.

[0003] Current mainstream hand-eye calibration methods (such as the Tsai-Lenz method and Zhang Zhengyou planar calibration method) usually require the collection of multiple sets of robot end-effector pose and visual observation data, solving nonlinear equations, and complex mathematical calculations and numerical optimizations. The calibration result is the transformation matrix T between the camera coordinate system and the robot end-effector coordinate system, and additional programming is required to convert it into target point coordinates that can be used by the robot.

[0004] Therefore, in the specific field of robotic welding, the specifications of the workpieces to be welded vary and there is no fixed reference for their placement. It is necessary to rely on hand-eye calibration data to complete the conversion between the weld coordinates and the robot's base coordinate system to guide the welding torch operation. However, the use of hand-eye calibration in actual welding processes also has the following problems: (1) Factors such as plate tilting during welding make it difficult to clearly collect the weld trajectory; (2) If the robot end posture is adjusted to optimize the shooting angle, due to the lack of a real-time collision detection mechanism, it is very easy for the welding torch head to collide with the board, damaging the equipment and reducing production efficiency. (3) The transformation matrix obtained by hand-eye calibration cannot be directly input into the robot controller. That is, it is necessary to solve the matrix X in AX=XB. The matrix X is the relationship between the camera and the flange. However, the robot controller needs the form of tool-flange or camera-base coordinate system. Therefore, it is necessary to develop an additional coordinate transformation function for secondary transformation, and the application of the calibration results is complicated. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the technical problem to be solved by the present invention is to propose a hand-eye calibration optimization method and system adapted to robot welding scenarios, so as to safely, efficiently and easily convert high-precision calibration data into precise robot movements.

[0006] To achieve this objective, the present invention adopts the following technical solution: This invention provides a hand-eye calibration optimization method adapted to robotic welding scenarios, comprising the following steps: S00: Place the calibration block on the same horizontal plane as the robot, and maneuver the tip of the robot's end effector to align with and touch the center point of the calibration block in a vertical orientation. Record the TCP tool center position parameters of the robot at this moment. and attitude quaternions Thus, the coordinates of the calibration block and the TCP tool center are obtained. , ); S10: The vision camera is aligned with the visual axis of the calibration block, including: Adjust the orientation of the vision camera to an approximately vertical position, move the vision camera so that the calibration block is at the center of its field of view; capture an image of the calibration block and acquire its point cloud data, and adjust the Z-axis of the vision camera to be collinear with the Z-axis of the calibration block; Continue moving the vision camera and adjusting its posture accordingly to capture images of the calibration block again and acquire its point cloud data. Then, adjust the X-axis of the vision camera to be aligned with the X-axis direction of the calibration block. When the vision camera is aligned with the Z-axis and X-axis of the calibration block, the camera coordinate system and the calibration block coordinate system coincide in orientation, and the robot flange center position parameters are read from the robot controller. and attitude quaternions Thus, the center coordinates of the flange are obtained ( , ); S20: Combining the coordinates of the calibration block ( , ) and the center coordinates of the flange ( , Calculate the displacement parameters of the tool coordinate system relative to the robot flange coordinate system. and attitude parameters The displacement parameters With the attitude parameters By combining these parameters, the final parameters U of the visual camera tool coordinate system are obtained. , ( ), to describe the complete pose of the camera coordinate system relative to the robot flange coordinate system.

[0007] A hand-eye calibration optimization system adapted to robotic welding scenarios is used to execute a hand-eye calibration optimization method adapted to robotic welding scenarios as described above, including: A robot unit includes a robot body, an end effector, and a controller; The vision sensing unit includes a vision camera fixed to the end effector, and the vision camera integrates an auxiliary structure light source. The calibration block, placed in the robot's workspace, has a regular geometric shape and a surface marked with labels to enhance visual feature recognition; The control and processing unit is configured to execute the following modules: The communication and control module is used to control the movement of the robot unit and the vision sensing unit, and to acquire robot TCP coordinates and camera image data; The adaptive image acquisition module is used to automatically adjust camera parameters and light source intensity based on real-time image feedback, and to perform quality assessment and re-acquisition control on the acquired point cloud data. The point cloud processing module is used to perform noise reduction, contour extraction, plane fitting, and feature calculation on the point cloud data acquired by the visual camera. The safe motion planning module is used to construct a simplified bounding box model based on the environmental point cloud and perform collision detection and obstacle avoidance planning for the attitude adjustment path. The pose calculation and alignment module is used to calculate the rotation quaternion required to align the camera axis with the calibration block axis through a two-stage method of virtual pre-alignment and visual servo fine-tuning, based on the output results of the point cloud processing module, and to calculate the final parameters of the camera tool coordinate system. The parameter integration module is used to automatically write the calculated tool coordinate system parameters into the controller via the robot controller API, and perform dynamic verification and compensation during work intervals.

[0008] The beneficial effects of this invention are as follows: (1) This invention decouples the traditional hand-eye calibration process based on complex transformation matrix solutions into an intuitive physical touch and visual axis alignment operation. A reliable translation reference is obtained through end effector touch, and the robot actively adjusts the camera's posture to align its coordinate system with the calibration block's coordinate system on three axes, thereby obtaining the rotation reference. This achieves translation reference decoupling and rotation reference decoupling. This process does not require operators to have in-depth knowledge of matrix transformations and numerical optimization; only simple alignment commands are needed to automatically calculate accurate camera tool coordinates, significantly reducing the difficulty of use for on-site workers in the welding field. (2) In welding scenarios, traditional methods require point cloud data to undergo a series of coordinate transformations before it can be used for actual welding trajectory planning. This invention uses a calibration block to establish the relationship between the camera coordinate system and the robot body coordinate system, that is, to establish the camera-tool coordinate system. Specifically, the robot's gun tip is used to vertically pierce the center of the calibration block to obtain the position of the calibration block center in the robot body coordinate system. Then, by taking a picture, the camera is aligned with the center of the calibration block in a vertical posture, thereby establishing the relationship between the camera and the robot body coordinate system, and establishing a rotation-free relationship between the camera coordinate system and the calibration block. This allows the point cloud data collected under this standard posture to directly reflect the true position and orientation of the workpiece without the need for complex coordinate transformation calculations. The final generated displacement parameters and posture quaternions can be directly written into the robot controller as tool coordinate system parameters, achieving the effect of plug-and-play calibration results without the need for complex secondary transformation of point cloud data. (3) To address the severe interference from arc light, spatter, reflection, and fumes present at the welding site, this invention introduces an adaptive image acquisition optimization and point cloud quality assessment mechanism. By adjusting the camera exposure, gain, and structured light intensity in real time, and using a quantile truncation algorithm to remove outliers, high-quality point cloud data can still be obtained under complex lighting and surface conditions. In particular, the minimum area rectangle and convex hull algorithm is used for the extraction of the calibration block edge contour. Even with local occlusion or missing point clouds, the normal direction and feature edge direction of the calibration block can still be fitted with high accuracy, enhancing the anti-interference capability of the calibration process. (4) This invention deeply integrates visual perception with robot motion planning. Before performing posture adjustment, a simplified bounding box model of the environment is constructed based on point cloud and real-time collision detection and obstacle avoidance path planning are performed, which effectively prevents the risk of collision between the welding torch and the fixture and workpiece. In addition, a two-stage execution strategy of virtual pre-alignment simulation verification combined with visual servo closed-loop fine-tuning is adopted, which not only ensures the safety of the motion process, but also takes into account the calibration efficiency and final accuracy. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating a hand-eye calibration optimization method adapted to robot welding scenarios provided in a specific embodiment of the present invention. Figure 2 This is a schematic diagram of the process of adjusting the Z-axis of the vision camera to be collinear with the Z-axis of the calibration block, provided in a specific embodiment of the present invention. Figure 3 This is a schematic diagram of the process of adjusting the X-axis of the vision camera to be consistent with the X-axis direction of the calibration block, provided in a specific embodiment of the present invention. Detailed Implementation

[0010] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0011] To address three problems encountered in the use of hand-eye calibration during actual welding processes—namely, difficulties in clearly capturing weld trajectories due to plate tilt, lack of real-time collision detection during posture adjustment, and the need for secondary transformation of the transformation matrix (resulting in complex applications of calibration results)—this invention provides a hand-eye calibration optimization method and system adapted to robotic welding scenarios. Suitable for industrial welding and other scenarios emphasizing operability and rapid deployment, it significantly reduces the technical threshold and application complexity of hand-eye calibration. The core inventive concept is to replace visual calculation of translation with physical interaction and to replace numerical optimization of rotation with active posture adjustment. Specifically, firstly, the end effector (gun tip) touches the center of the calibration block to obtain the reference point coordinates (i.e., TCP tool center coordinates). Next, the camera posture is adjusted so that its Z-axis and X-axis are collinear with the Z-axis and X-axis of the calibration block (posture coincidence), and the working point coordinates (i.e., flange center coordinates) are read from the robot controller under this posture. Finally, the difference between the reference point coordinates and the working point coordinates is used to calculate the camera's displacement and posture parameters relative to the flange, which are combined into tool coordinate system parameters for direct use by the robot controller's coordinate system.

[0012] Example 1: A hand-eye calibration optimization method adapted to robot welding scenarios. This method uses physical touch to replace visual calculation of translation, and active axis alignment to replace mathematical optimization of rotation. It decomposes the hand-eye calibration problem into independent position offset measurements (displacement parameters) and attitude offset measurements (attitude parameters), ultimately directly outputting the tool coordinate system parameters required by the robot controller. Therefore, this method includes the following steps in its implementation: S00: Place the calibration block on the same horizontal plane as the robot, and maneuver the tip of the robot's end effector to align with and touch the center point of the calibration block in a vertical orientation. Record the TCP tool center position parameters of the robot at this moment. and attitude quaternions Thus, the coordinates of the calibration block and the TCP tool center are obtained. , At this point, the position parameters in the TCP tool's center coordinates are obtained. Attitude quaternions in TCP tool center coordinates This step primarily establishes a reference point through physical contact, providing a zero-position reference for the subsequent translation relationship between the camera coordinate system and the tool coordinate system. The gun tip is the end effector during actual operation, and contacting the center point directly obtains the three-dimensional coordinates of that point in the robot's base coordinate system. These coordinates do not rely on any visual data and are accurate and reliable. Preferably, the calibration block is a solid with a regular geometric shape, its top surface being a flat rectangle or circle, and its surface is provided with markings or patterns to enhance visual feature recognition, facilitating identification and positioning.

[0013] After establishing the reference point, the robot flange coordinates can be obtained, and the camera attitude adjustment can begin. First, ensure that the camera's Z-axis is perpendicular to the workpiece plane. This step is the foundation for all subsequent alignments—if the optical axis is not perpendicular, subsequent X-axis alignment and center alignment will introduce perspective errors. In step S10: adjust the orientation of the vision camera to an approximately vertical state, move the vision camera so that the calibration block is at the center of its field of view; capture an image of the calibration block and obtain its point cloud data, and adjust the Z-axis of the vision camera to be collinear with the Z-axis of the calibration block; specifically, the camera's Z-axis is the direction of the lens optical axis, and the Z-axis of the calibration block usually refers to the normal direction of its top surface. The top surface plane of the calibration block can be fitted using point cloud data to calculate its normal vector; adjust the camera's orientation (through the robot's end effector's six degrees of freedom movement) so that the camera's Z-axis is parallel (collinear) with this normal vector. This step mainly eliminates the tilt between the camera and the workpiece plane, ensuring that the camera's optical axis is perpendicular to the surface during subsequent measurements, thereby obtaining the clearest image and point cloud with the least geometric distortion. At the same time, it lays the first rotational constraint for defining the pose of the camera tool coordinate system.

[0014] Preferably, the step of adjusting the Z-axis of the vision camera to be collinear with the Z-axis of the calibration block involves automatically and accurately solving the normal direction of the calibration block plane using point cloud processing technology, and using quaternions to drive the robot to actively align the camera's Z-axis, including: S10a1: The point cloud data is denoised using a quantile truncation algorithm to remove point cloud noise. Compared with mean filtering or radius filtering, quantile truncation (such as taking the 5th to 95th percentile of the point cloud coordinates) does not rely on noise distribution assumptions and can adaptively remove extreme outliers while retaining the main structure. This is to address situations where, during welding site calibration, the calibration block surface may be contaminated with spatter, oil, or reflective points, or arc light or dust in the environment may cause outlier noise points in the point cloud that severely affect the distortion of the normal direction.

[0015] S10a2: Extract the contours of the denoised point cloud data and calculate the two-dimensional outer contour points of the smallest convex polygon that can wrap all valid point clouds. By calculating the smallest convex polygon (convex hull), a boundary that tightly wraps all valid point clouds can be obtained. These boundary points exclude internal redundant information, and the convex hull is robust to a small number of missing point clouds.

[0016] S10a3: Based on spatial coordinate mapping rules, establish a one-to-one mapping relationship between the two-dimensional outer contour points and the corresponding three-dimensional outer contour point cloud; the two-dimensional contour points obtained from the point cloud projection (e.g., projection on the image plane) need to be back-calculated back to their true three-dimensional coordinates before they can be used for subsequent plane fitting. Establishing a one-to-one mapping relationship ensures that each two-dimensional contour point can accurately correspond to its three-dimensional spatial position, avoiding coordinate misalignment.

[0017] S10a4: Fit a contour plane based on the three-dimensional outer contour point cloud, and calculate the normal vector of the contour plane; the Z-axis of the calibration block is defined as the normal direction of the top surface. Fitting the plane using the contour point cloud (rather than the entire point cloud) can minimize the influence of internal unevenness and obtain the most robust normal vector. Preferably, least squares or principal component analysis (PCA) is used for plane fitting.

[0018] S10a5: Rotate the normal vector to coincide with the Z-axis direction of the preset world coordinate system, generating the corresponding first rotation quaternion; by calculating the rotation of the current normal to the world Z-axis, a rotation quaternion is obtained. Compared with Euler angles, quaternions can avoid gimbal lock problems and have smooth interpolation, making them suitable for continuous attitude adjustment of the robot's end effector.

[0019] S10a6: Adjust the orientation of the vision camera based on the first rotation quaternion, so that the Z-axis of the vision camera is collinear with the Z-axis of the calibration block; send the quaternion to the robot controller to drive the end effector (with camera) to rotate its orientation, so that the camera's Z-axis coincides with the normal of the calibration block. After this step, the camera's optical axis is perpendicular to the plane of the calibration block, creating a perspective-distortion-free condition for subsequent X-axis alignment and center alignment.

[0020] After the vertical condition is met, adjust the rotation around the Z-axis to make the camera's X-axis parallel to the workpiece's feature edge. The two-step attitude adjustment is decoupled: first adjust pitch and roll (to ensure verticality), then adjust yaw (to ensure direction); that is, further eliminate the yaw angle error around the Z-axis, and further obtain the robot flange center coordinates ( , That is, continue moving the vision camera and adjusting its posture accordingly to take another picture of the calibration block and acquire its point cloud data. Then, adjust the X-axis of the vision camera to be consistent with the X-axis direction of the calibration block. When the vision camera is aligned with the Z-axis and X-axis of the calibration block, the camera coordinate system and the calibration block coordinate system are in the same orientation. Read the robot flange center position parameters from the robot controller. and attitude quaternions Thus, the center coordinates of the flange are obtained ( , The X-axis of the calibration block typically refers to the direction of a preset edge line (e.g., the long side of a rectangle or a scribing line) on its top surface. The direction vector of this edge line is extracted from the point cloud. By rotating the camera around its own Z-axis, the row direction of the camera image coordinate system (or the X-axis of the point cloud) is made parallel to this edge line. Thus, after step S10, the camera's Z-axis is perpendicular to the calibration block plane, and its X-axis is parallel to a feature edge of the calibration block (i.e., the X-axis of the vision camera is adjusted to be consistent with the X-axis direction of the calibration block). The Y-axis is naturally determined by the right-hand rule. This completely defines the camera's attitude relative to the calibration block, ensuring no rotational deviation between the camera coordinate system and the calibration block coordinate system. That is, the three axes of the camera are completely parallel to the three axes of the calibration block. This accurately establishes a rotation-free relationship between the camera coordinate system (tool coordinate system) and the calibration block coordinate system (robot flange coordinate system).

[0021] To ensure that the quality of the input data meets the requirements of subsequent high-precision processing before the actual acquisition of the calibration block point cloud data, step S20, before capturing the image of the calibration block and acquiring the point cloud data, further includes: S101: Adaptive Image Acquisition Optimization Steps: Based on real-time image feedback, the exposure time and gain of the vision camera, as well as the projection intensity of the auxiliary structured light source, are automatically adjusted to ensure that the point cloud data on the calibration block surface meets preset quality indicators. Through real-time image feedback (such as histogram analysis and point cloud density statistics), parameters are automatically adjusted to ensure that the reflection intensity of the calibration block surface falls within the optimal response range of the sensor, thereby obtaining a high signal-to-noise ratio and uniformly dense point cloud. This addresses the challenges posed by complex and variable welding environments, where the calibration block surface may contain reflective surfaces (such as aluminum alloys and stainless steel), dark areas, oil stains, rust, etc., while ambient lighting (arc light, natural light) also changes dynamically. Fixed camera parameters (exposure time, gain) and structured light intensity cannot adapt to all situations, easily leading to overexposure (generating holes) or underexposure (low signal-to-noise ratio) of the point cloud, thus affecting the accuracy of contour extraction and normal calculation.

[0022] Even after parameter optimization in S101, local point cloud defects or excessive noise may still occur due to physical obstructions (such as welding torches blocking part of the field of view), broken laser stripes, or local anomalies on the calibration block surface (such as deep scratches). Directly using such low-quality point clouds for rectangle fitting and angle calculation would result in significant calibration errors. Therefore, S102 is also included: Point Cloud Quality Assessment and Resampling Steps. A pre-trained deep learning model is used to score the quality of the collected calibration block point cloud data. If the score is lower than a preset threshold, a resampling strategy is automatically triggered. The camera position or laser projection angle is adjusted, and point cloud data is re-acquired until the score meets the requirements. This introduction of a pre-trained deep learning model allows for rapid and automated evaluation of the overall quality of the point cloud (completeness, outlier ratio, surface smoothness, etc.). Resampling is automatically triggered when the score is lower than the preset threshold, and the camera position or laser projection angle is adjusted before re-acquiring until the requirements are met. This avoids the subjectivity and inefficiency of manual visual inspection.

[0023] By using steps S101 and S102 as prerequisite steps for S20, and through the design of active perception-quality closed-loop control, point cloud acquisition is transformed from an open-loop operation into closed-loop quality control. That is, dynamically matching camera parameters with surface characteristics to obtain high-quality raw data and achieve adaptive optimization; and using deep learning to automatically ensure that each frame of point cloud used for calibration meets the accuracy requirements. This provides a solid data foundation for subsequent automatic yaw alignment (S10b1~S10b5), thereby significantly improving the reliability, accuracy and automation of the entire hand-eye calibration method.

[0024] The step of adjusting the X-axis of the vision camera to be consistent with the X-axis direction of the calibration block establishes an automatic yaw alignment process based on point cloud contours and a minimum area rectangle. With the camera already perpendicular to the calibration block, by extracting the principal axis direction of the minimum area rectangle, the process automatically calculates and executes rotation around the Z-axis to achieve high-precision alignment between the camera's X-axis and the calibration block's X-axis. Specifically, this includes: S10b1: The quantile truncation algorithm is used to denoise the convex hull point cloud data on the contour plane, removing noisy points. In S10a2, the convex hull contour point cloud of the top surface of the calibration block has been extracted, but this point cloud may still contain edge splashes, reflection noise, or outliers. Quantile truncation (e.g., taking the 2nd to 98th percentile of the coordinates) can adaptively remove extreme outliers, avoiding severe distortion of subsequent rectangle fitting by a single noise point. S10b2: Extract contour features from the denoised convex hull point cloud data and calculate the 2D vertex set of the minimum area rectangle that encloses all valid point clouds. The top surface of the calibration block is usually designed as a rectangle (or has a clear principal direction feature), and its X-axis direction should be parallel to one side of the rectangle. The minimum area rectangle can closely fit the oriented bounding box of the point cloud, and its long side direction is the principal axis direction (X-axis) of the calibration block. Compared with directly fitting a straight line or using PCA, the minimum area rectangle has better robustness to partial point cloud defects (such as a corner being occluded) and has a clear physical meaning. S10b3: Based on coordinate sorting rules, the two-dimensional vertex set is sorted in ascending order of X-coordinate and then in ascending order of Y-coordinate to obtain a standardized feature point set; the four vertices output by the minimum area rectangle algorithm are unordered (for example, they may be given from any counterclockwise starting point). In order to uniformly determine the direction vector of the long side of the rectangle, it is necessary to sort it according to a preset rule (such as first in ascending order of X-coordinate, then in ascending order of Y-coordinate), thereby clarifying the "lower left corner" of the rectangle and its adjacent vertices. This ensures that the extracted X-axis direction has a consistent positive direction, avoiding ambiguity of 180° in subsequent angle calculations; S10b4: Calculate the angle between the X-axis of the calibration block and the X-axis of the vision camera on the contour plane based on the standardized feature point set, and generate the corresponding second rotation quaternion. On the contour plane (i.e., the top surface of the calibration block, which is perpendicular to the camera's Z-axis), the camera's own X-axis direction (defined by the camera's intrinsic parameters, usually corresponding to the horizontal rightward direction of the image) is known. The X-axis direction of the calibration block is given by the standardized long side direction vector of the rectangle. The angle between the two planes is the angle to be rotated. Generating a second quaternion rotating around the camera's Z-axis avoids the singularity caused by using Euler angles (although pure rotation around the Z-axis has no gimbal lock, using quaternions uniformly facilitates subsequent combination with the first quaternion). S10b5: Adjust the orientation of the vision camera based on the second rotation quaternion, making the X-axis of the vision camera collinear with the X-axis of the calibration block; send the second quaternion to the robot controller to drive the end effector to rotate by a specified angle around the current camera Z-axis (i.e., the world vertical direction), making the camera X-axis completely parallel to the calibration block X-axis. At this point, the three axes of the camera coordinate system and the calibration block coordinate system are completely aligned.

[0025] After the posture is completely fixed, only translate the camera so that its field of view center is aligned with the center of the calibration block. Translation and rotation are separated to ensure that the final position measurement is not affected by posture errors; that is, perform S30: maintain the posture of the vision camera unchanged, align the center of the vision camera's point cloud data with the center point of the calibration block, record the robot's coordinate values ​​at this time, and obtain the robot flange center position parameters. Thus, the center coordinates of the flange are obtained ( , That is, the position parameter of the flange center coordinate is The attitude quaternion of the flange center coordinates is The "center of the point cloud data" can be the camera's field of view center (the spatial point corresponding to the intersection of the optical axis and the image plane) or the centroid of all points in the point cloud. In practical applications, the point where the spatial ray corresponding to the camera's field of view intersects with the top surface of the calibration block is more commonly used. The alignment process can be completed through visual servoing or manual fine-tuning: the point cloud center position is displayed in real time, and the robot is moved until the center coincides with the calibration block center (which can be obtained by mapping the touch position in S00) in space. This step involves obtaining the robot's end-effector coordinates when the camera is in the "ideal observation pose" (perpendicular to the workpiece and centered), which also corresponds to the TCP tool coordinates. The two rotation quaternions (the first rotation quaternion and the second rotation quaternion) obtained above are mainly input into the robot to move it to be perpendicular to the calibration block normal and parallel to the calibration block's X-axis.

[0026] After obtaining two key coordinates (the TCP tool center coordinates when the gun tip touches the target and the flange center coordinates when the camera is aligned), the displacement and attitude parameters of the camera tool coordinate system are directly calculated by combining displacement and attitude; S40: Combined with the coordinates of the calibration block ( , ) and the center coordinates of the flange ( , Calculate the displacement parameters of the tool coordinate system relative to the robot flange coordinate system. and attitude parameters ,Right now: ; Therefore, after conversion, we get: ; The vector pointing from the flange center to the TCP tool center is also equal to the vector pointing from the flange center to the camera optical center in the axis-aligned state (because after alignment, there is a definite geometric relationship between the camera optical center and the weld tip-calibration block center).

[0027] The attitude parameters are calculated based on the rotation quaternion used to align the camera axis in step S10. The rotation of the camera coordinate system relative to the flange coordinate system is determined directly through the active axis alignment operation of S10, rather than through matrix solving, that is: ; Therefore, after conversion, we get: ; The displacement parameter With the attitude parameters By combining these parameters, the final parameters U of the visual camera tool coordinate system are obtained. , (), to describe the complete pose of the camera coordinate system relative to the robot flange coordinate system; Specifically, in step S00, the calibration block and the TCP tool center attitude quaternion are obtained. In step S10, the quaternion of the flange center attitude is obtained. Finally, the displacement parameters With the attitude parameters Combination (selecting rotation quaternions or Euler angles depending on the robot type; rotation quaternions are used in this example), wherein the attitude parameters... The displacement parameters can be synthesized through quaternion multiplication or transformed according to the specific rotation order, ultimately yielding the final displacement parameters. With the attitude parameters By combining these parameters, the final parameters U of the visual camera tool coordinate system are obtained. , This describes the complete pose of the camera coordinate system relative to the robot flange coordinate system, where... It is a displacement parameter, which is a translation vector; These are quaternion-represented attitude parameters, essentially a 6-DOF pose description (3D position, 4D attitude, or attitude can be simplified to 3D Euler angles). The output format itself is a tool coordinate system description directly acceptable to the controller, requiring no additional matrix decomposition or coordinate system transformation. It can be directly written into the robot controller, yielding the displacement parameters. and attitude parameters The coordinates can be directly input into the robot and assigned to the tool, achieving the effect of not needing to go through a complex conversion function for secondary transformation.

[0028] After calculating the final parameters of the camera tool coordinate system in step S20, the following steps are also included: S30: Automated parameter injection step: Through the robot controller's application programming interface (API) or fieldbus protocol, the calculated camera tool coordinate system parameters are automatically written into the corresponding tool data storage area in the robot controller, completing the automatic configuration of the calibration results; this eliminates the errors and inefficiencies of manual input, and automatic writing through API or fieldbus avoids human error; from acquisition, processing, calculation to configuration, all processes are automated and closed-loop without human intervention.

[0029] S40: Dynamic verification and compensation of calibration results. During the intervals of robot welding operations, the robot is automatically controlled to return to the calibration pose, and the calibration block is re-observed using a vision camera. The deviation between the current actual tool coordinate system parameters and the calibration values ​​is calculated. If the deviation exceeds a preset threshold, online recalibration or parameter compensation based on the deviation model is automatically triggered, and the tool parameters in the robot controller are updated. In this way, this step makes up for the shortcomings of static calibration in resisting long-term drift, giving the system the ability to self-diagnose and self-correct. It automatically performs periodic verification during the intervals of welding operations to detect deviations in a timely manner. Periodic verification ensures that the hand-eye calibration parameters are always within the effective range, thereby stabilizing the welding quality. Dynamic verification can quickly check for disturbances during the welding process, such as arc light, heat radiation, and spatter, which may affect the rigidity of the camera mount or the relative position of the calibration block (e.g., the thermal expansion of the fixture), thus avoiding cumulative errors.

[0030] Example 2: To address the issue of welding torch tip colliding with workpieces due to the lack of real-time collision detection when adjusting the robot's end-effector posture to optimize the shooting angle, step S10a6 or S10b5 includes a safe motion planning step before adjusting the visual camera posture based on the first or second rotation quaternion. This step deeply integrates visual perception (point cloud) with robot motion planning. The robot no longer blindly executes preset angles but can "see" obstacles and actively avoid them. Specifically, this includes: S1: A simplified bounding box model of the calibration block and its surrounding environment is constructed based on the point cloud data acquired by the vision camera at the initial position. During the attitude adjustment process, the robot's end effector moves along with the camera and welding torch. Objects in the surrounding environment, such as the calibration block, fixtures, and worktable, can all become collision targets. Before S10 or S20, the vision camera has already acquired point cloud data of the calibration block at a certain initial position. Using these point clouds, a simplified geometric model (such as an axis-aligned bounding box (AABB) or an oriented bounding box (OBB)) can be quickly built to represent the spatial occupancy of obstacles. Using a simplified model instead of a precise point cloud is to balance computational speed and collision detection accuracy—the calibration process requires real-time processing, and overly detailed triangular meshes would slow down the detection frequency.

[0031] S2: Perform real-time collision detection based on the current robot pose, target pose, and bounding box model. During the process of the robot moving from the current pose to the target pose (specified by the first or second quaternion), the end effector (including welding torch, camera bracket, etc.) may move along a straight line or a planned trajectory. Therefore, during collision detection, it is necessary to simplify each link of the robot into a cylinder or capsule, and simplify the welding torch and camera into a bounding box. Check whether these geometries intersect with the obstacle bounding box established in S1 at the trajectory interpolation point. If they intersect, it is determined that there is a collision risk.

[0032] S3: If the planned path poses a collision risk, an obstacle-free avoidance path is automatically calculated, and the robot is controlled to perform attitude adjustments along this path. Directly executing attitude adjustment commands that may lead to collisions is dangerous. Therefore, the path planning module needs to be activated to search for a safe path in the configuration space (C-space) or Cartesian space. Using common methods such as bidirectional RRT (Rapid Exploration Random Tree) and potential field methods, the planned path should ensure that the minimum distance between all robot parts and obstacles is greater than a safety threshold (e.g., 5mm) throughout the entire movement. Then, the robot is controlled to smoothly move along this path to the target posture.

[0033] In step S10a6 or S10b5, the step of adjusting the visual camera pose based on the rotation quaternion is specifically executed in two stages: The first stage, virtual pre-alignment (simulation verification combined with coarse adjustment): The generated rotation quaternion is simulated and verified in a simulation environment. After confirming that there are no collisions and that the robot joint limits are not exceeded, the actual coarse adjustment motion is executed. Collision detection and obstacle avoidance planning have been performed in S1~S3 above. Collision detection of real robots is usually based on conservative estimates of simplified bounding boxes, while simulation can use more accurate mesh collisions without damaging the equipment. Execution after verification is equivalent to double insurance; specifically, the actual robot motion may still encounter unexpected events due to model errors, control lag, or unmodeled interference (such as cable pulling). In the simulation environment, a high-fidelity model (including the robot, welding torch, camera, cables, and surrounding equipment) is used to completely simulate the motion trajectory from the current pose to the pose specified by the target quaternion. This can detect dynamic collisions (intersection of an intermediate point with an obstacle during motion), robot joint angle exceeding limits (in some configurations, although the end pose is achievable, the joint is close to the hard limit), sudden velocity changes near singularities, etc.

[0034] Coarse adjustment refers to moving to the target pose at a lower speed (e.g., 30% of the maximum linear speed) and with a larger positioning tolerance (e.g., angular error <1°) according to the quaternion interpolation path. The purpose of coarse adjustment is to quickly approach the ideal alignment state, providing an optimal initial position for subsequent fine adjustment. In other words, coarse adjustment reduces the deviation to within the effective working range of visual servoing.

[0035] The second stage is visual servo fine-tuning (closed-loop fine-tuning): After coarse-tuning, residual errors still exist in the posture, caused by factors such as robot repetitive positioning accuracy, calibration block placement errors, and simulation model deviations. To achieve sub-pixel alignment accuracy, a visual feedback closed loop must be introduced. The visual servo closed loop is activated to monitor the position of the calibration block features in the image in real time, and fine-tunes the robot end effector until the axis alignment accuracy reaches the preset sub-pixel error range. The visual feedback closed loop includes continuous acquisition of calibration block images / point clouds by the camera, real-time feature extraction (such as the slope of the rectangle's sides and the position of the center point), calculation of the current deviation (such as the angle between the camera's X-axis and the calibration block's X-axis), and generation of fine-tuning motion commands (small rotations and translations) until the deviation is less than a preset threshold.

[0036] Through the two stages described above, the first stage ensures safe and reachable motion (without exceeding limits or colliding), while the second stage ensures accuracy (sub-pixel level alignment), thus constructing a dual guarantee of safety to avoid damage to the equipment. Simulation verification can detect dynamic interference that may be missed in real collision detection (such as the welding torch cable swinging and colliding with the fixture during movement). The coarse adjustment stage uses low-speed motion, so even if an accidental collision occurs (such as a temporary object not included in the simulation model), the impact force is small, reducing the risk of damage. Visual servo closed-loop fine adjustment eliminates residual errors and compensates for model inaccuracies.

[0037] Example 3: In step S20, in order to eliminate random errors and improve the accuracy and reliability of hand-eye calibration results through statistical methods, the displacement parameters and attitude parameters can also be obtained by averaging multiple measurements, specifically including: S21: Repeat steps S00 to S10 multiple times to obtain multiple sets of TCP tool center coordinate values ​​and flange center coordinate values, as well as multiple sets of first rotation quaternions and second rotation quaternions; During a single calibration process, random errors may be introduced by factors such as the robot's repeatability positioning accuracy, human or visual alignment errors when the gun tip touches the center of the calibration block, numerical fluctuations in normal calculation and minimum area rectangle extraction in point cloud processing, and minor deformations caused by environmental vibration and temperature changes. These factors can be mitigated by taking multiple independent measurements and averaging them to make the final result closer to the true value. S22: Calculate the average value of multiple sets of displacement parameters as the final displacement parameter; displacement parameters It is a three-dimensional vector. When the random error follows a distribution with a mean of zero, the arithmetic mean of multiple measurements is the minimum variance unbiased estimate. Calculating the average of multiple sets of displacement parameters in this way can yield the optimal position estimate.

[0038] S23: Calculate the average of multiple sets of attitude parameters (Riemann geometric mean) or obtain the final attitude parameters through quaternion spherical interpolation (Slerp); Slerp averaging of quaternions results in an orthogonal normalized rotation matrix, avoiding gimbal lock or angle jumps that may occur with Euler angle averaging. This ensures that the final camera tool coordinate system attitude parameters can be directly input into the robot controller without causing abnormal motion; In summary, steps S21 to S23, through statistical repeated measurements and averaging (for position) and Riemann geometric averaging (for rotation), suppress random errors and improve the accuracy, reliability, and repeatability of the calibration results. This enables the stable output of high-precision calibration results under the complex conditions of industrial welding sites. Furthermore, the averaging of multiple measurements occurs during the adjustment process of aligning the normal direction with the z-axis of the calibration block and the x-axis with the x-axis of the calibration block, thus ultimately yielding accurate camera tool coordinates.

[0039] Example 4: A hand-eye calibration optimization system adapted to robotic welding scenarios, used to execute a hand-eye calibration optimization method adapted to robotic welding scenarios as described above, including: A robot unit includes a robot body, an end effector, and a controller; The vision sensing unit includes a vision camera fixed to the end effector, and the vision camera integrates an auxiliary structure light source. The calibration block, placed in the robot's workspace, has a regular geometric shape and a surface marked with labels to enhance visual feature recognition; The control and processing unit is configured to execute the following modules: The communication and control module is used to control the movement of the robot unit and the vision sensing unit, and to acquire robot TCP coordinates and camera image data; The adaptive image acquisition module is used to automatically adjust camera parameters and light source intensity based on real-time image feedback, and to perform quality assessment and re-acquisition control on the acquired point cloud data. The point cloud processing module is used to perform noise reduction, contour extraction, plane fitting, and feature calculation on the point cloud data acquired by the visual camera. The safe motion planning module is used to construct a simplified bounding box model based on the environmental point cloud and perform collision detection and obstacle avoidance planning for the attitude adjustment path. The pose calculation and alignment module is used to calculate the rotation quaternion required to align the camera axis with the calibration block axis through a two-stage method of virtual pre-alignment and visual servo fine-tuning, based on the output results of the point cloud processing module, and to calculate the final parameters of the camera tool coordinate system. The parameter integration module is used to automatically write the calculated tool coordinate system parameters into the controller via the robot controller API, and perform dynamic verification and compensation during work intervals.

[0040] The visual sensing unit is a 3D structured light camera or laser profilometer with integrated laser-assisted illumination; the calibration block is a regular geometric body with at least one planar feature surface, its surface is coated with a diffuse reflection coating, and its edges are provided with geometric chamfers or marking patterns to enhance feature recognition.

[0041] This invention has been described through preferred embodiments. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. This invention is not limited to the specific embodiments disclosed herein; other embodiments falling within the scope of the claims are also within the protection scope of this invention.

Claims

1. A hand-eye calibration optimization method adapted to robotic welding scenarios, characterized in that, Includes the following steps: S00: Place the calibration block on the same horizontal plane as the robot, and maneuver the tip of the robot's end effector to align with and touch the center point of the calibration block in a vertical orientation. Record the TCP tool center position parameters of the robot at this moment. and attitude quaternions Thus, the coordinates of the calibration block and the TCP tool center are obtained. , ); S10: The vision camera is aligned with the visual axis of the calibration block, including: Adjust the orientation of the vision camera to an approximately vertical position, move the vision camera so that the calibration block is at the center of its field of view; capture an image of the calibration block and acquire its point cloud data, and adjust the Z-axis of the vision camera to be collinear with the Z-axis of the calibration block; Continue moving the vision camera and adjusting its posture accordingly to capture images of the calibration block again and acquire its point cloud data. Then, adjust the X-axis of the vision camera to be aligned with the X-axis direction of the calibration block. When the vision camera is aligned with the Z-axis and X-axis of the calibration block, the camera coordinate system and the calibration block coordinate system coincide in orientation, and the robot flange center position parameters are read from the robot controller. and attitude quaternions Thus, the center coordinates of the flange are obtained ( , ); Before capturing images of the calibration block and acquiring point cloud data, the process also includes: S101: Adaptive image acquisition optimization step: Based on real-time image feedback, automatically adjust the exposure time and gain of the visual camera and the projection intensity of the auxiliary structure light source so that the point cloud data on the surface of the calibration block meets the preset quality indicators. S102: Point cloud quality assessment and resampling steps: The pre-trained deep learning model is used to score the quality of the collected calibration block point cloud data. If the score is lower than the preset threshold, the resampling strategy is automatically triggered. The camera position or laser projection angle is adjusted and the point cloud data is re-acquired until the score meets the requirements. S20: Combining the coordinates of the calibration block ( , ) and the center coordinates of the flange ( , ), calculate the displacement parameters of the tool coordinate system relative to the robot flange coordinate system and attitude parameters The displacement parameters With the attitude parameters By combining these parameters, the final parameters U of the visual camera tool coordinate system are obtained. , ( ), to describe the complete pose of the camera coordinate system relative to the robot flange coordinate system.

2. The hand-eye calibration optimization method adapted to robot welding scenarios according to claim 1, characterized in that: In step S10, the step of adjusting the Z-axis of the vision camera to be collinear with the Z-axis of the calibration block includes: S10a1: The point cloud data is denoised using a quantile truncation algorithm to remove point cloud noise. S10a2: Extract the contours from the denoised point cloud data and calculate the two-dimensional outer contour points of the smallest convex polygon that can enclose the entire effective point cloud. S10a3: Based on the spatial coordinate mapping rules, establish a one-to-one mapping relationship between the two-dimensional outer contour points and the corresponding three-dimensional outer contour point cloud; S10a4: Fit the contour plane based on the three-dimensional outer contour point cloud, and calculate the normal vector of the contour plane; S10a5: Rotate the normal vector to coincide with the Z-axis direction of the preset world coordinate system, and generate the corresponding first rotation quaternion; S10a6: Adjust the orientation of the vision camera based on the first rotation quaternion so that the Z-axis of the vision camera is collinear with the Z-axis of the calibration block.

3. The hand-eye calibration optimization method adapted to robot welding scenarios according to claim 2, characterized in that, In step S10, the step of adjusting the X-axis of the vision camera to be consistent with the X-axis direction of the calibration block includes: S10b1: The quantile truncation algorithm is used to denoise the convex hull point cloud data on the contour plane and remove point cloud noise. S10b2: Extract contour features from the denoised convex hull point cloud data and calculate the two-dimensional vertex set of the minimum area rectangle that encloses all valid point clouds; S10b3: Based on the coordinate sorting rules, the two-dimensional vertex set is sorted in ascending order of X coordinate and ascending order of Y coordinate to obtain a standardized feature point set; S10b4: Calculate the angle between the X-axis of the calibration block and the X-axis of the vision camera on the contour plane based on the standardized feature point set, and generate the corresponding second rotation quaternion; S10b5: Adjust the orientation of the vision camera based on the second rotation quaternion so that the X-axis of the vision camera is collinear with the X-axis of the calibration block.

4. The hand-eye calibration optimization method adapted to robot welding scenarios according to claim 3, characterized in that, In step S20, the displacement parameters and attitude parameters are obtained by averaging multiple measurements, specifically including: S21: Repeat steps S00 to S10 multiple times to obtain multiple sets of TCP tool center coordinate values ​​and flange center coordinate values, as well as multiple sets of first rotation quaternions and second rotation quaternions; S22: Calculate the average value of multiple sets of displacement parameters as the final displacement parameter. ; S23: Calculate the average of multiple sets of attitude parameters or obtain the final attitude parameters through quaternion spherical interpolation. .

5. The hand-eye calibration optimization method adapted to robot welding scenarios according to claim 3, characterized in that, In step S10a6 or S10b5, before adjusting the visual camera pose based on the first rotation quaternion and the second rotation quaternion, a safe motion planning step is also included: S1: Construct a simplified bounding box model of the calibration block and its surrounding environment based on the point cloud data collected by the visual camera at the initial position; S2: Perform real-time collision detection based on the current robot pose, target pose, and the bounding box model; S3: If there is a collision risk in the planned path, automatically calculate a collision-free avoidance path and control the robot to perform attitude adjustments along the path.

6. The hand-eye calibration optimization method adapted to robot welding scenarios according to claim 3, characterized in that, In step S10a6 or S10b5, the step of adjusting the visual camera pose based on the rotation quaternion is specifically executed in two stages: The first stage is virtual pre-alignment: the generated rotation quaternion is simulated and verified in a simulation environment. After confirming that there is no collision and that it does not exceed the robot joint limits, the coarse adjustment motion is actually executed. The second stage is visual servo fine-tuning: After coarse adjustment, visual servo closed-loop is started to monitor the position of the calibration block features in the image in real time, and fine-tune the robot end until the axis alignment accuracy reaches the preset sub-pixel error range.

7. The hand-eye calibration optimization method adapted to robot welding scenarios according to claim 1, characterized in that, After calculating the final parameters U of the camera tool coordinate system in step S20, the following steps are also included: S30: Automated parameter injection step: Through the robot controller's application programming interface (API) or fieldbus protocol, the calculated camera tool coordinate system parameters are automatically written into the corresponding tool data storage area in the robot controller, completing the automatic configuration of the calibration results; S40: Dynamic verification and compensation of calibration results. During the intervals of robot welding operations, the robot is automatically controlled to return to the calibration pose, and the calibration block is re-observed using a vision camera. The deviation between the current actual tool coordinate system parameters and the calibration values ​​is calculated. If the deviation exceeds a preset threshold, online recalibration or parameter compensation based on the deviation model is automatically triggered, and the tool parameters in the robot controller are updated.

8. A hand-eye calibration optimization system adapted to robotic welding scenarios, used to execute a hand-eye calibration optimization method adapted to robotic welding scenarios as described in any one of claims 1 to 7, characterized in that, include: A robot unit includes a robot body, an end effector, and a controller; The vision sensing unit includes a vision camera fixed to the end effector, and the vision camera integrates an auxiliary structure light source. The calibration block, placed in the robot's workspace, has a regular geometric shape and a surface marked with labels to enhance visual feature recognition; The control and processing unit is configured to execute the following modules: The communication and control module is used to control the movement of the robot unit and the vision sensing unit, and to acquire robot TCP coordinates and camera image data; The adaptive image acquisition module is used to automatically adjust camera parameters and light source intensity based on real-time image feedback, and to perform quality assessment and re-acquisition control on the acquired point cloud data. The point cloud processing module is used to perform noise reduction, contour extraction, plane fitting, and feature calculation on the point cloud data acquired by the visual camera. The safe motion planning module is used to construct a simplified bounding box model based on the environmental point cloud and perform collision detection and obstacle avoidance planning for the attitude adjustment path. The pose calculation and alignment module is used to calculate the rotation quaternion required to align the camera axis with the calibration block axis through a two-stage method of virtual pre-alignment and visual servo fine-tuning, based on the output results of the point cloud processing module, and to calculate the final parameters of the camera tool coordinate system. The parameter integration module is used to automatically write the calculated tool coordinate system parameters into the controller via the robot controller API, and perform dynamic verification and compensation during work intervals.

9. A hand-eye calibration optimization system adapted for robotic welding scenarios according to claim 8, characterized in that, The visual sensing unit is a 3D structured light camera or laser profilometer with integrated laser-assisted illumination; the calibration block is a regular geometric body with at least one planar feature surface, its surface is coated with a diffuse reflection coating, and its edges are provided with geometric chamfers or marking patterns to enhance feature recognition.

Images