Humanoid robot hand-eye coordination intelligent calibration control system and method

Abstract

The invention provides a humanoid robot hand-eye coordination intelligent calibration control system and a humanoid robot hand-eye coordination intelligent calibration control method, and the humanoid robot hand-eye coordination intelligent calibration control system integrates high-precision contour extraction, visual feedback closed-loop control, multi-sensor fusion and dynamic path planning. The robot hand-eye calibration method has the beneficial effects that the calibration efficiency and precision can be remarkably improved, the problem of calibration parameter drift caused by system time-varying characteristics and environmental interference is effectively solved through continuous tracking of a target contour and real-time online compensation of pose deviation, and the calibration accuracy is improved. Therefore, the reliability and the absolute positioning precision of the robot in long-term complex operation tasks are guaranteed, and the self-adaptive capability of a robot system is enhanced.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to an intelligent calibration control system and method for hand-eye coordination in humanoid robots. Background Technology

[0002] Hand-eye calibration technology for industrial robots is the core foundation for enabling robots to accurately complete vision-guided tasks such as grasping, assembly, and inspection. Its purpose is to precisely determine the relative pose relationship between the robot's end effector and the vision sensors mounted on it. With the increasing demands for flexibility and precision in intelligent manufacturing, traditional hand-eye calibration methods, such as offline calibration using specific calibration boards and relying on manual intervention, are no longer sufficient to meet the needs of complex and dynamic work scenarios. These methods typically suffer from cumbersome calibration processes, low efficiency, and the inability to adaptively adjust online. In particular, during actual robot operation, initial calibration parameters may drift due to factors such as mechanical vibration, thermal deformation, or external interference. Traditional static calibration methods lack online monitoring and compensation mechanisms, making it difficult to guarantee the absolute positioning accuracy of the robot during long-term operation. This has become the biggest bottleneck restricting the application of high-precision robots. Therefore, developing an online self-calibration technology that can track target features in real time and automatically correct pose deviations based on visual feedback is of great significance for improving the intelligence level and long-term accuracy stability of robot operating systems. Summary of the Invention

[0003] In view of this, the present invention provides an intelligent calibration and control system for hand-eye coordination in humanoid robots. One or more embodiments of this specification also relate to an intelligent calibration and control method for hand-eye coordination in humanoid robots, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the present invention, an intelligent calibration and control system for hand-eye coordination in a humanoid robot is provided, comprising: A high-precision contour extraction and enhancement module is used to acquire and process images to extract the continuous and complete contours of the target object; The contour error calculation and feature matching module is associated with the high-precision contour extraction and enhancement module. It is used to receive contours and match them with pre-stored calibration templates, and calculate pose deviation and matching quality evaluation indicators. The vision-motion closed-loop control module, which is associated with the contour error calculation and feature matching module, is used to generate motion adjustment commands for the robot end effector based on pose deviation and matching quality evaluation indicators. The multi-sensor fusion and pose prediction module, which is associated with the contour error calculation and feature matching module and the vision-motion closed-loop control module, is used to fuse visual data and inertial measurement data to estimate and predict the pose of the robot's end effector. The dynamic path planning and obstacle avoidance module, which is linked to the multi-sensor fusion and pose prediction module and the vision-motion closed-loop control module, is used to adjust the motion path of the robot's end effector online and avoid obstacles based on the predicted pose and environmental perception data. The closed-loop quality assessment and termination judgment module, which is associated with the contour error calculation and feature matching module and the vision-motion closed-loop control module, is used to monitor the convergence of the calibration process and terminate the calibration process when preset conditions are met.

[0005] In some implementations, the high-precision contour extraction and enhancement module includes a semantic segmentation network with an attention mechanism, and also includes an adaptive histogram equalization processing unit and a multi-scale retinal enhancement processing unit for image enhancement.

[0006] In some implementations, the contour error calculation and feature matching module uses a registration method based on distance transformation and iterative nearest point algorithm to calculate the pose deviation, and the matching quality evaluation index includes contour coincidence index and Hausdorff distance.

[0007] In some implementations, the vision-motion closed-loop control module establishes a mapping relationship between the image feature space and the robot joint space through the image Jacobian matrix, and adopts a hybrid control strategy that combines proportional-integral-derivative and fuzzy control, wherein the proportional coefficient is dynamically adjusted based on the contour error calculation and the matching confidence output by the feature matching module.

[0008] In some implementations, the multi-sensor fusion and pose prediction module uses an extended Kalman filter to fuse visual pose data from the contour error calculation and feature matching module with acceleration and angular velocity data from the inertial measurement unit.

[0009] In some implementations, the dynamic path planning and obstacle avoidance module uses a fast random tree algorithm for local path planning and combines it with an artificial potential field method to achieve dynamic obstacle avoidance. The environmental perception data is provided by a depth camera.

[0010] In some implementations, the closed-loop quality assessment and termination judgment module performs convergence judgment based on the uncertainty of the calibration parameters. When the condition number of the calibration equation is lower than the threshold and the contour error norm calculated by the contour error calculation and feature matching module is less than the preset pixel tolerance, the calibration is determined to be converged.

[0011] In some implementations, the formula for calculating the profile error norm includes:

[0012] Where N represents the number of translational degrees of freedom, derived from the system's degree of freedom definition; M represents the number of rotational degrees of freedom, derived from the system's degree of freedom definition; The deviation in the i-th translation direction is obtained by decomposing the translation error vector calculated by the contour error calculation and feature matching module. The angular deviation on the j-th rotation axis is obtained by decomposing the rotation error vector calculated by the contour error calculation and feature matching module. The normalized scaling factor representing the translation error is preset by the closed-loop quality assessment and termination judgment module based on the robot's workspace and accuracy requirements; The normalized scaling factor representing the rotation error is preset by the closed-loop quality assessment and termination judgment module based on the allowable angle deviation of the application scenario; This represents the weighting coefficient for the i-th translation direction, used to adjust the contribution of errors in different translation directions. It is set by the closed-loop quality assessment and termination judgment module according to the current task priority. This represents the weighting coefficient of the j-th rotation axis, used to adjust the contribution of errors of different rotation axes. It is set by the closed-loop quality assessment and termination judgment module according to the current task priority.

[0013] In some implementations, the formula for calculating the weighting coefficient in the translation direction includes:

[0014] Where P represents the total number of image gradient sampling points used to evaluate contour sharpness, which is derived from a preset sampling point distribution scheme around the target contour; This represents the spatial weight coefficient of the k-th sampling point. The closed-loop quality assessment and termination judgment module assigns weights based on the proximity of the sampling point to the edge of the target contour. The closer the sampling point is to the edge, the higher the weight. This represents the image gradient magnitude at the kth sampling point. It is dimensionless and is extracted by the high-precision contour extraction and enhancement module after performing gradient calculation on the preprocessed image. This represents the smoothing factor, a constant greater than zero, used to control the rate at which weights change with sharpness; It represents the standard deviation of pixel values ​​in the target contour region of the current frame image. It is dimensionless and is statistically calculated by the high-precision contour extraction and enhancement module. It is used to characterize the local contrast of the image. This represents the task priority coefficient for the i-th translation direction. It is dimensionless and is preset by the user according to the calibrated task requirements.

[0015] According to a second aspect of the present invention, an intelligent calibration and control method for hand-eye coordination in a humanoid robot is provided. This method is applied to the system of any one of claims 1 to 9, and includes: The robot's end effector moves to the calibrated area along the initial path; The high-precision contour extraction and enhancement module acquires images and extracts target contours. The deviation between the current pose and the ideal pose is calculated through the contour error calculation and feature matching module; The vision-motion closed-loop control module generates adjustment commands to drive the robot's end effector to correct its pose. The multi-sensor fusion and pose prediction module utilizes inertial measurement data to provide motion prediction and compensate for system latency. The dynamic path planning and obstacle avoidance module adjusts the movement trajectory according to environmental changes; The entire process is monitored by a closed-loop quality assessment and termination judgment module, and the process is automatically terminated when the calibration accuracy meets the requirements.

[0016] At least one embodiment of this invention achieves full automation and intelligence in the robot hand-eye calibration process by constructing a system that integrates high-precision contour extraction, visual feedback closed-loop control, multi-sensor fusion, and dynamic path planning. Its beneficial effects are that it can significantly improve the efficiency and accuracy of calibration, and through continuous tracking of the target contour and real-time online compensation for pose deviation, it effectively overcomes the problem of calibration parameter drift caused by system time-varying characteristics and environmental interference, thereby ensuring the reliability and absolute positioning accuracy of the robot in long-term complex tasks and enhancing the adaptive capability of the robot system. Attached Figure Description

[0017] Figure 1 This is a simplified structural diagram of an intelligent calibration control system for hand-eye coordination in a humanoid robot provided by the present invention; Figure 2 This is a flowchart of an intelligent calibration and control method for hand-eye coordination in a humanoid robot, provided by the present invention. Detailed Implementation

[0018] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0019] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the one or more embodiments of this specification. The singular forms “a” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items. The modifications “a” and “a plurality” as used in this disclosure are illustrative and not restrictive, and those skilled in the art will understand that they should be understood as “one or more” unless the context clearly indicates otherwise.

[0020] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0021] See Figure 1 , Figure 1 This diagram illustrates a simplified structural schematic of an intelligent calibration control system for hand-eye coordination in a humanoid robot, according to some embodiments of this specification. Specifically, it includes: a high-precision contour extraction and enhancement module for acquiring and processing images to extract the continuous and complete contour of the target object; a contour error calculation and feature matching module, associated with the high-precision contour extraction and enhancement module, for receiving the contour and matching it with a pre-stored calibration template, calculating pose deviation and matching quality evaluation indicators; a vision-motion closed-loop control module, associated with the contour error calculation and feature matching module, for generating motion adjustment commands for the robot's end effector based on the pose deviation and matching quality evaluation indicators; and a multi-sensor system. The fusion and pose prediction module, linked to the contour error calculation and feature matching module and the vision-motion closed-loop control module, is used to fuse visual data and inertial measurement data to estimate and predict the pose of the robot's end effector. The dynamic path planning and obstacle avoidance module, linked to the multi-sensor fusion and pose prediction module and the vision-motion closed-loop control module, is used to adjust the motion path of the robot's end effector online and avoid obstacles based on the predicted pose and environmental perception data. The closed-loop quality assessment and termination judgment module, linked to the contour error calculation and feature matching module and the vision-motion closed-loop control module, is used to monitor the convergence of the calibration process and terminate the calibration process when preset conditions are met.

[0022] The high-precision contour extraction and enhancement module refers to the functional unit responsible for identifying and enhancing the edge information of target objects from the original image. For example, this module uses a semantic segmentation network with an attention mechanism and image enhancement algorithms to process the original image data captured by the camera to obtain continuous and complete target contour information. The continuous and complete contour of the target object can refer to the set of closed and unbroken boundary lines formed by the target object in the image. For example, after classifying the image pixels through a semantic segmentation network, the edge pixels of the target region can be extracted and connected into a closed curve, which can serve as the geometric feature basis for subsequent pose matching.

[0023] The contour error calculation and feature matching module refers to a processing unit used to quantify the difference between the actual contour and the ideal template and calculate the pose relationship. For example, this module receives extracted contour data and uses algorithms such as Iterative Closest Point (ICP) to register it with the calibration template, outputting the pose deviation and matching confidence evaluation index. The pre-stored calibration template refers to a standard contour model representing the target object in the ideal pose, pre-stored in the system. For example, it can be generated by offline acquisition of multi-view images or 3D model data of the target object to create a reference point cloud or contour sequence, which can serve as a benchmark for real-time matching. Pose deviation refers to the difference in translation and rotation between the current actual position and posture of the robot's end effector and the desired pose. For example, it can be calculated using a feature matching algorithm to obtain translation vectors and rotation matrices (or angle differences represented by Euler angles / quaternions), providing feedback signals for motion control. Matching quality assessment metrics can refer to quantitative parameters used to evaluate the degree of matching between real-time contours and template contours. For example, geometric similarity measures such as contour overlap and Hausdorff distance are used to determine the reliability of the current visual data in order to adjust the control strategy.

[0024] A vision-motion closed-loop control module can refer to a controller that generates robot joint motion commands based on visual feature errors. For example, this module establishes a mapping relationship between image spatial errors and joint spatial motion quantities based on the image Jacobian matrix. It can employ a hybrid strategy of proportional-integral-derivative (PID) and fuzzy control to drive the robot to reduce pose errors. Motion adjustment commands for the robot's end effector can refer to control signals sent to the robot joint actuators to correct the end-effector's pose. For example, the pose deviation calculated by the vision system can be converted into angle or velocity commands for each joint through inverse kinematics calculation to achieve precise closed-loop position control.

[0025] A multi-sensor fusion and pose prediction module refers to a functional unit that integrates visual and inertial measurement data to estimate and predict the motion state of the robot's end effector. For example, an extended Kalman filter (EKF) can be used to fuse visual pose data and IMU angular velocity / accelerometer data to predict motion trends and compensate for system latency. Visual and inertial measurement data refer to information about the target and the robot's own motion from the camera and IMU, respectively. For instance, visual data includes image coordinate sequences of the target contour, while inertial measurement data includes readings from a three-axis accelerometer and a three-axis gyroscope. Data fusion can improve the frequency and robustness of pose estimation. The pose of the robot's end effector refers to the position coordinates and attitude angles of the end tool in three-dimensional space. This can be described, for example, by a state vector output from the fusion module containing three-dimensional translation vectors and rotation matrices (or quaternions), serving as the basis for path planning and control.

[0026] The dynamic path planning and obstacle avoidance module refers to an algorithm unit that generates collision-free motion trajectories in real time based on environmental changes. For example, this module combines the Rapidly-exploring Random Tree (RRT) algorithm and the artificial potential field method, utilizing point cloud data provided by a depth camera, to adjust the path online to ensure safe movement. Predicted pose and environmental perception data refer to the future pose estimates output by the fusion module and the surrounding obstacle information acquired by sensors. For example, combining the short-term pose sequence predicted by EKF with the 3D point cloud of obstacles detected by the depth camera can be used to evaluate path feasibility and replan the path. The motion path of the robot's end effector refers to a series of intermediate pose points traversed by the end tool from its current position to its target position. For example, the collision-free joint angle sequence searched in the configuration space by the path planning module based on A* or RRT* algorithms can guide the robot's smooth and safe movement. Online adjustment and obstacle avoidance can refer to the behavior of dynamically modifying the predetermined path based on real-time sensor information during robot movement. For example, when the depth camera detects a new obstacle, the local planner immediately calculates the detour path and sends the corrected joint trajectory points to achieve continuous and safe operation in dynamic environments.

[0027] The closed-loop quality assessment and termination decision module can refer to a control unit that monitors the convergence status of the calibration process and determines when to stop. For example, this module continuously calculates indicators such as the contour error norm and the condition number of the calibration equations. When the error is below a threshold and the system is stable, a termination command is issued to ensure calibration accuracy and avoid invalid iterations. The convergence status of the calibration process can refer to the state where the calibration parameter sequence tends to stabilize and the error continues to decrease. For example, it can be determined by evaluating whether the norm change rate of the pose deviation is below a preset threshold in multiple consecutive iterations, ensuring that the system reaches the optimal calibration result. Preset conditions can refer to a set of quantitative criteria set for termination decisions, such as logical combinations including contour error norm less than tolerance, parameter uncertainty below a threshold, and the number of consecutive iterations meeting requirements, to achieve automated and reliable process termination.

[0028] The present invention will be further described below through a detailed embodiment: In one embodiment of the intelligent calibration control system for hand-eye coordination of the humanoid robot of the present invention, the humanoid robot needs to accurately grasp a specific tool handle placed on a conveyor belt on a simulated assembly line, in an environment with normal lighting changes and slight vibrations.

[0029] The humanoid robot's hand-eye coordination intelligent calibration and control system includes a high-precision contour extraction and enhancement module, a contour error calculation and feature matching module, a vision-motion closed-loop control module, a multi-sensor fusion and pose prediction module, a dynamic path planning and obstacle avoidance module, and a closed-loop quality assessment and termination judgment module.

[0030] The high-precision contour extraction and enhancement module employs a semantic segmentation network with an integrated attention mechanism to achieve pixel-level recognition of the target object. Specifically, this module acquires scene images at a frame rate of 30 frames per second using a color camera mounted on the robot's wrist. The acquired raw images are first enhanced for contrast by an adaptive histogram equalization unit to mitigate the effects of uneven lighting; subsequently, a multi-scale retinal enhancement unit further sharpens edge details. The preprocessed image is then input into the semantic segmentation network, where the attention mechanism enhances the response to the target tool handle features and suppresses background interference, ultimately outputting a continuous and complete binary contour image containing only the target object. This contour, as a key visual feature, is then passed to the next module.

[0031] The contour error calculation and feature matching module receives the aforementioned contour data via a gigabit Ethernet interface. This module internally stores a calibration template contour of the tool handle in the ideal grasping pose. The module employs an iterative nearest-point algorithm based on distance transformation to register the real-time extracted contour with the template contour. The registration process calculates the pose deviation of the current robot end effector (i.e., the gripper) relative to the ideal grasping pose. This deviation includes deviations in the X, Y, and Z translational directions (in millimeters) and rotational angular deviations around each coordinate axis (in radians). Simultaneously, the module calculates the contour coincidence and Hausdorff distance as matching quality evaluation indicators to quantify the confidence level of the current contour matching.

[0032] The vision-motion closed-loop control module receives pose deviation and matching quality evaluation indicators via the robot control bus. The core function of this module is to establish a mapping relationship between changes in the image feature space and the robot's joint spatial motion based on the image Jacobian matrix. The module employs a hybrid control strategy combining proportional-integral-derivative control and fuzzy logic. Specifically, the proportional control coefficient in this strategy is dynamically adjusted based on the matching confidence level transmitted from the upstream module: when the confidence level is high, a larger proportional coefficient is used to achieve a fast response; when the confidence level is low, the proportional coefficient is reduced, and the module relies more on integral action to smoothly approach the target, avoiding severe jitter caused by visual noise. Finally, the module generates the corresponding joint motion commands and sends them to the robot's joint servo drivers.

[0033] The multi-sensor fusion and pose prediction module simultaneously receives visual pose data from the contour error calculation and feature matching module, as well as triaxial acceleration and triaxial angular velocity data from the inertial measurement unit (IMU) embedded in the end effector. This module employs an extended Kalman filter (EPF) algorithm to fuse these two types of data. The EPF prediction step uses IMU data to predict the short-term pose of the end effector in the next control cycle, while the update step corrects the prediction using visual pose measurements. This process effectively compensates for the inherent time delay in the entire pipeline from image acquisition and processing to control command generation, providing the system with more real-time, smooth, and accurate pose estimation.

[0034] The dynamic path planning and obstacle avoidance module receives predicted poses from the multi-sensor fusion and pose prediction module, as well as point cloud data from a depth camera mounted on the robot's head. This module uses a fast random tree algorithm as its core for local path planning. When the depth camera detects that a tool on the conveyor belt has shifted due to vibration or that other temporary obstacles have entered the pre-grabbing path, the module combines the artificial potential field method to calculate the repulsive force in real time, adjusts the motion trajectory of the end effector online, generates a new collision-free path point sequence, and ensures that this sequence is coordinated with the overall motion adjustment commands generated by the vision-motion closed-loop control module.

[0035] The closed-loop quality assessment and termination judgment module continuously monitors the entire calibration process. It receives the contour error sequence and matching quality index from the contour error calculation and feature matching module, and evaluates the stability of the control output of the vision-motion closed-loop control module. Internally, this module maintains a calibration parameter uncertainty model and calculates the condition number of the calibration equation in real time to determine if the system is stabilizing. When the module determines that the condition number is below a preset threshold, and the contour error norm calculated by combining translation and rotation errors remains below the set pixel tolerance, it indicates that the hand-eye coordination calibration has reached the expected accuracy. The system automatically terminates the closed-loop adjustment process and sends a "grasping ready" signal to the main control system. Afterward, the robot performs the final grasping action.

[0036] The system operates as follows: When the robot's main control system issues a grasping command, the system starts. The robot's end effector moves along the initially planned path to the calibration area roughly above the work area. Subsequently, the system enters a high-frequency vision-motion closed loop: the high-precision contour extraction and enhancement module continuously provides a clear contour; the contour error calculation and feature matching module continuously calculates the pose deviation; the vision-motion closed-loop control module drives the robot to approach the target accordingly; simultaneously, the multi-sensor fusion and pose prediction module provides a forward-looking pose estimate, and the dynamic path planning and obstacle avoidance module ensures safety during the movement process. Test data shows that, under test environments with ±5% brightness fluctuations and slight vibrations, this implementation can shorten the convergence time of the grasping calibration process by approximately 40%, and the final pose accuracy is stabilized within the millimeter level.

[0037] The beneficial effects of one of the embodiments in this specification include at least the following: by constructing a system that integrates high-precision contour extraction, visual feedback closed-loop control, multi-sensor fusion, and dynamic path planning, the entire process of robot hand-eye calibration is automated and intelligent. Its beneficial effects are that it can significantly improve the efficiency and accuracy of calibration, and through continuous tracking of the target contour and real-time online compensation for pose deviation, it effectively overcomes the problem of calibration parameter drift caused by system time-varying characteristics and environmental interference, thereby ensuring the reliability and absolute positioning accuracy of the robot in long-term complex tasks and enhancing the adaptive capability of the robot system.

[0038] In some implementations, the high-precision contour extraction and enhancement module includes a semantic segmentation network with an attention mechanism, and also includes an adaptive histogram equalization processing unit and a multi-scale retinal enhancement processing unit for image enhancement.

[0039] Semantic segmentation networks can refer to convolutional neural networks that classify each pixel in an image into a specific semantic category (such as target or background). For example, architectures like U-Net or DeepLab can be used to process the input image end-to-end, outputting a segmentation mask of the same size as the input to accurately identify pixel regions of the target object. Adaptive histogram equalization processing units can refer to algorithm modules used to enhance local image contrast. For example, this unit divides the image into small regions and performs histogram equalization on each, while limiting contrast gain to avoid noise amplification, thus improving the visibility of contour edges under uneven lighting conditions. Multi-scale retinal enhancement processing units can refer to image enhancement algorithm modules that simulate the processing mechanism of the human retina. For example, this unit calculates the detail and base layers of the image at different scales, adjusts their contrast, and then fuses them to enhance contour edge details while maintaining a natural visual effect.

[0040] As a concrete example: During robot hand-eye calibration, the high-precision contour extraction and enhancement module is activated. The hand-eye camera mounted on the robot's eye acquires images of the workpiece, which may have low contrast in some areas due to uneven ambient lighting. First, the adaptive histogram equalization unit divides the image into multiple small grids and performs histogram equalization with limited contrast on each grid, effectively enhancing the edge information of local dark or bright areas. Subsequently, the enhanced image is fed into a U-Net semantic segmentation network with a ResNet-50 backbone and an embedded Convolutional Block Attention Module (CBAM). The ResNet-50 backbone is responsible for extracting multi-level features, while the CBAM module weights the feature maps channel-wise and spatially, making the network focus more on the workpiece contour region. Finally, the multi-scale retinal enhancement unit post-processes the segmented binary contour image, extracting and fusing contour details at different scales through Gaussian pyramid decomposition, outputting a continuous, smooth, and detailed final contour for subsequent modules to perform high-precision matching.

[0041] By introducing a semantic segmentation network with an attention mechanism and synergistic image enhancement unit, this invention improves the accuracy and robustness of target contour extraction under complex lighting and background interference, laying a reliable visual feature foundation for subsequent high-precision pose matching.

[0042] In some implementations, the contour error calculation and feature matching module uses a registration method based on distance transformation and iterative nearest point algorithm to calculate the pose deviation, and the matching quality evaluation index includes contour coincidence index and Hausdorff distance.

[0043] Registration methods based on distance transform and iterative nearest-point algorithms can refer to a pose calculation technique that combines distance maps with iterative alignment of point clouds. For example, this method first calculates a distance transform map for the template contour image, then finds the nearest point in the distance map for the real-time contour point cloud and solves for the optimal rigid body transformation, effectively handling registration problems with partial occlusion or incomplete contours. Contour overlap indexes can refer to similarity metrics used to quantify the area of ​​overlapping regions between two contours. For example, it can be obtained by calculating the ratio of the intersection area to the union area of ​​the real-time contour and the template contour in image space (i.e., the intersection-union ratio, IoU), which intuitively reflects the geometric consistency of contour matching. Hausdorff distance can refer to the extreme geometric distance used to measure the maximum mismatch between two point sets. For example, it calculates the maximum distance from all points in one contour point set to another, sensitively capturing the local maximum deviation between contours and used to evaluate the worst-case matching.

[0044] As a concrete example: when the contour error calculation and feature matching module receives the real-time contour point cloud of the robot end effector from the high-precision contour extraction and enhancement module, the registration process begins. First, the module performs distance transformation calculations on the pre-stored binary image of the ideal workpiece template contour, generating a distance map where the value of each pixel represents its Euclidean distance to the nearest template contour point. Then, an iterative nearest-neighbor algorithm is used for registration: using the real-time contour point cloud as the source point set, in each iteration, the nearest neighbor point (i.e., the point with the smallest distance value) is found in the distance map for each source point, thus constructing a point-to-point correspondence; then, based on these point pairs, methods such as singular value decomposition (SVD) are used to calculate the optimal rigid body transformation (rotation matrix R and translation vector T) that minimizes the sum of squared distances between corresponding point pairs. This iterative process continues until the transformation parameters converge or the maximum number of iterations is reached. After registration is complete, the module outputs the calculated pose deviation (i.e., the inverse transformation of the optimal transformation). Meanwhile, the module calculates contour overlap indices (such as intersection-over-union ratio, IoU) and bidirectional Hausdorff distance. The former reflects the overall overlap, while the latter is used to ensure that there are no significant local out-of-contour points. Together, they serve as matching quality assessment indicators and are passed to the subsequent control module.

[0045] By employing a robust registration method that combines distance transformation and iterative nearest-point algorithm, and introducing contour overlap and Hausdorff distance as comprehensive evaluation indicators, this invention can achieve high-precision pose deviation calculation without being sensitive to noise and partial occlusion, and provide reliable matching quality feedback for the control loop.

[0046] In some implementations, the vision-motion closed-loop control module establishes a mapping relationship between the image feature space and the robot joint space through the image Jacobian matrix, and adopts a hybrid control strategy that combines proportional-integral-derivative and fuzzy control, wherein the proportional coefficient is dynamically adjusted based on the contour error calculation and the matching confidence output by the feature matching module.

[0047] The image Jacobian matrix can refer to a mathematical matrix describing the linear relationship between changes in the image feature space and changes in the robot joint space motion. For example, this matrix can be obtained by jointly differentiating the camera model and the robot kinematic model, with its elements representing the partial derivatives of the image pixel coordinates with respect to the robot joint angles, used to map image errors into joint motion quantities. The mapping relationship between the image feature space and the robot joint space can refer to a mathematical transformation model from visually observed changes in two-dimensional image features to changes in the robot's joint angles. For example, a linear relationship can be established through the image Jacobian matrix, allowing the displacement of feature points in the image plane to be converted into control commands that drive joint motion. A hybrid control strategy combining proportional-integral-derivative (PID) and fuzzy control can refer to an advanced control method that integrates the precision of classical PID control with the adaptability of fuzzy logic control. For example, using image feature errors as input, the fuzzy logic controller dynamically adjusts the proportional coefficient of the PID controller based on the error magnitude and matching confidence level to balance response speed and overshoot. Hybrid control strategies refer to solutions that combine the advantages of two or more different control algorithms to address complex control problems. For example, they might rely on the robustness of fuzzy logic when system uncertainty is high, while leveraging the precision of PID control as the system approaches its target, thus improving control performance at different stages. The proportional gain, dynamically adjusted based on the contour error calculation and the matching confidence output from the feature matching module, refers to a mechanism where the controller's gain parameter adaptively changes according to the matching quality evaluation index. For instance, when the Hausdorff distance is small and the contour overlap is high, the proportional gain is increased to improve response speed; conversely, it is decreased to prevent oscillation. Matching confidence refers to a probabilistic or qualitative assessment of the reliability of current visual measurement data. For example, it can be calculated by normalizing and fusing contour overlap indices and Hausdorff distance, and is used to characterize the reliability of pose deviation data.

[0048] As a concrete example: when the robot moves along a calibrated trajectory, the vision-motion closed-loop control module begins operation. The module receives pose deviations (such as translation vectors [dx, dy, dz] and rotation vectors [dθx, dθy, dθz]) and matching quality evaluation metrics (such as normalized contour overlap ratio IoU) from the contour error calculation and feature matching module. First, the module uses a pre-calibrated image Jacobian matrix (a 6xN matrix, where N is the number of joints) to convert the pose deviations in the image feature space into joint angle corrections Δθ in the robot joint space. Simultaneously, a hybrid control strategy is activated: the fuzzy logic controller takes the current matching confidence (calculated from IoU and Hausdorff distance) and deviation norm as input, and dynamically adjusts the proportional coefficient Kp in the PID controller online according to a preset fuzzy rule table. For example, when the matching confidence is high and the deviation is large, a larger Kp value is output to accelerate convergence; when approaching the target or with low confidence, a smaller Kp value is output to ensure smoothness. The PID controller then calculates the joint speed or position command to be sent to the robot driver based on the adjusted parameters and the joint angle deviation Δθ, thus completing one closed-loop control cycle.

[0049] By establishing a precise mapping relationship based on the image Jacobian matrix and adopting a hybrid control strategy with adaptive gain adjustment, this invention can dynamically optimize control behavior according to the reliability of visual feedback, thereby achieving fast, smooth, and high-precision robot hand-eye coordination movements while ensuring system stability.

[0050] In some implementations, the multi-sensor fusion and pose prediction module uses an extended Kalman filter to fuse visual pose data from the contour error calculation and feature matching module with acceleration and angular velocity data from the inertial measurement unit.

[0051] The Extended Kalman Filter (EKF) can refer to a recursive filtering algorithm designed for state estimation problems in nonlinear systems. For example, this filter effectively fuses visual pose data and inertial measurement data by linearizing the system model and observation model and then applying standard Kalman filtering steps. Visual pose data refers to the position and attitude information of the robot's end effector in the camera coordinate system, calculated through image contour matching. This includes state vectors containing three-dimensional translation coordinates and rotation quaternions or Euler angles, serving as observations in the fusion algorithm. Acceleration and angular velocity data refer to the raw readings of triaxial linear acceleration and triaxial angular velocity directly measured by IMU (Inertial Measurement Unit) sensors. For example, after temperature compensation and calibration, these data are used to extrapolate short-term motion changes through integration.

[0052] As a concrete example: In the multi-sensor fusion and pose prediction module, the extended Kalman filter is used as the state estimator. Its state vector typically includes the position, velocity, and orientation (represented by quaternions) of the robot's end effector, as well as the gyroscopic bias of the IMU. The system model predicts the evolution of the state vector based on the angular velocity and acceleration data provided by the IMU through kinematic equations. The observation model utilizes visual pose data (such as position and orientation obtained through ICP registration) from the contour error calculation and feature matching module. In each filtering cycle, a prediction step is performed first: using the state from the previous moment and the angular velocity and acceleration of the IMU (subtracting gravitational acceleration and compensating for bias), the state and covariance at the current moment are predicted. Then, an update step is performed: when new visual pose data arrives, the residual between the predicted visual observation and the actual measurement is calculated, and the state estimate and covariance are updated by combining the Kalman gain. Because the visual and IMU data are asynchronous in terms of frequency and latency (visual data has a low frequency and latency; IMU data has a high frequency and is real-time), the filter uses a timestamp-based data fusion strategy. Ultimately, this module not only outputs a fused and optimized current pose estimate, but also makes short-term predictions of the pose in the next few milliseconds based on the current state and the system model, in order to compensate for the delay caused by visual processing and provide forward-looking information for real-time control.

[0053] By employing an extended Kalman filter to deeply fuse visual pose observations and inertial measurement data, this invention can effectively compensate for the delay of a pure vision system and suppress the cumulative drift of inertial navigation, thereby obtaining high-frequency, high-precision real-time pose and short-term prediction of the robot's end effector, providing reliable state estimation for dynamic control and planning.

[0054] In some implementations, the dynamic path planning and obstacle avoidance module uses a fast random tree algorithm for local path planning and combines it with an artificial potential field method to achieve dynamic obstacle avoidance. The environmental perception data is provided by a depth camera.

[0055] Fast random tree algorithm refers to a planning method that rapidly constructs a search tree in configuration space through random sampling to find feasible paths. For example, this algorithm iteratively expands random nodes from the initial pose and connects new nodes to the nearest feasible nodes in the tree, effectively performing local path planning in high-dimensional space. Local path planning refers to the process of real-time trajectory optimization for changes in the robot's local environment, given that the global path is known or exists. For example, based on obstacle information perceived in real time by sensors, a path is replanned within a limited range to bypass newly appearing obstacles. Artificial potential field method refers to a path planning method that treats the robot as a point mass moving in a virtual potential field. For example, an attractive potential field is set for the target point, and a repulsive potential field is set for the obstacle. The robot moves along the negative gradient direction of the potential field, achieving smooth dynamic obstacle avoidance. Dynamic obstacle avoidance refers to the robot's ability to detect and avoid suddenly appearing or moving obstacles in real time during movement. For example, by continuously comparing the planned path with real-time environmental perception data, a local replanning is immediately triggered once a potential collision risk is detected. A depth camera can refer to a visual sensor that can directly acquire scene depth information (i.e., the distance from objects to the camera), such as a camera using structured light, time-of-flight, or binocular stereo vision principles, which can output point cloud data containing the depth value of each pixel. Environmental perception data can refer to the set of information about the physical characteristics of the robot's surrounding environment acquired by sensors, such as 3D point cloud data provided by a depth camera, which includes information such as the geometry, position, and size of obstacles.

[0056] As a concrete example: During the robot's end effector movement along a calibrated trajectory, the dynamic path planning and obstacle avoidance module continuously operates. The module's input includes short-term predicted poses of the robot's end effector from the multi-sensor fusion and pose prediction module, and real-time environmental point cloud data from a depth camera (such as an Intel RealSense D435i). The module first converts the point cloud data into a 3D occupancy grid map. When a new obstacle is detected on the original planned path (such as an operator's hand entering the workspace), the local path planner is activated. This planner, based on the Fast Random Tree (RRT) algorithm, uses the current end effector position as the root node and performs random sampling expansion in the configuration space to find feasible paths that bypass obstacles and reach the next sub-target point. Simultaneously, an artificial potential field method is used as an auxiliary optimization technique: an attractive potential is set for the target point, and a repulsive potential field is constructed for the obstacle point cloud. The planner uses the points on the initial path found by the RRT algorithm as optimization targets, fine-tuning them in the negative gradient direction of the resultant potential field, thereby obtaining a final trajectory that is not only collision-free but also smooth and far from obstacles. The adjusted joint space trajectory is sent to the robot controller in real time, driving the robot's end effector to safely and smoothly avoid obstacles and continue moving toward the target workpiece.

[0057] By combining the efficient search capability of the fast random tree algorithm with the real-time response characteristics of the artificial potential field method, and utilizing the accurate 3D environmental information provided by the depth camera, this invention enables robots to generate and optimize motion paths in real time in complex dynamic environments, effectively avoid sudden obstacles, and ensure the safety and continuity of the operation process.

[0058] In some implementations, the closed-loop quality assessment and termination judgment module performs convergence judgment based on the uncertainty of the calibration parameters. When the condition number of the calibration equation is lower than the threshold and the contour error norm calculated by the contour error calculation and feature matching module is less than the preset pixel tolerance, the calibration is determined to be converged.

[0059] The uncertainty of calibration parameters refers to a quantitative assessment of the potential error range of system parameters (such as the hand-eye matrix) estimated through the calibration process. This can be calculated using the inverse of the covariance matrix or the Fisher information matrix, reflecting the reliability of the parameter estimation. Convergence judgment refers to the logical decision-making process used to assess whether an iterative optimization process has reached or is close to a stable solution. This can be achieved, for example, by monitoring whether the change in the objective function value or parameter vector in consecutive iterations is below a preset threshold. The condition number of the calibration equations measures the sensitivity of the solution to small changes in the input data. This can be obtained by calculating the ratio of the maximum to the minimum singular value of the Jacobian matrix; a high condition number indicates that the problem may be ill-conditioned. The contour error norm is a scalar measure of the overall deviation between the real-time contour and the template contour. This can be obtained by calculating the square root of the weighted sum of squares of the errors at each point, and is used to comprehensively reflect the accuracy of the current pose. Preset pixel tolerance can refer to a pre-set maximum acceptable contour deviation threshold in the image pixel coordinate system. For example, it can be an empirical value set according to the accuracy requirements of the task. When the contour error is lower than this value, the visual alignment is considered to have met the requirements.

[0060] As a concrete example: During robot hand-eye calibration, the closed-loop quality assessment and termination judgment module runs continuously. The module's inputs include the current contour error norm (e.g., a dimensionless scalar value combining translation and rotation errors) from the contour error calculation and feature matching module, and the Jacobian matrix of the calibration equations and the covariance matrix of the parameter estimates calculated internally during the calibration process. The module first calculates the condition number of the calibration equations to assess the numerical stability of the current calibration problem. Simultaneously, it calculates the uncertainty of the calibration parameters (e.g., the standard deviation of the parameter estimates) based on the covariance matrix. The module uses these two metrics, along with the contour error norm, as convergence criteria: it checks whether the condition number remains consistently below a preset stability threshold (indicating numerical stability), monitors whether the contour error norm has decreased and stabilized within a preset pixel tolerance, and whether the parameter uncertainty has also converged to an acceptable low level. Only when all these conditions are simultaneously met, and after several iterations to confirm stability, does the module determine that the calibration process has converged, and then issues a termination command to the system, completing the calibration.

[0061] By comprehensively evaluating the uncertainty of calibration parameters, the condition number of calibration equations, and the real-time profile error norm, this invention can intelligently determine whether the calibration process has fully converged to the optimal solution and automatically terminate the process when the preset accuracy requirement is reached. This ensures calibration accuracy while avoiding unnecessary iterations and improving the overall efficiency of the system.

[0062] In some implementations, the formula for calculating the profile error norm includes:

[0063] Where N represents the number of translational degrees of freedom, derived from the system's degree of freedom definition; M represents the number of rotational degrees of freedom, derived from the system's degree of freedom definition; The deviation in the i-th translation direction is obtained by decomposing the translation error vector calculated by the contour error calculation and feature matching module. The angular deviation on the j-th rotation axis is obtained by decomposing the rotation error vector calculated by the contour error calculation and feature matching module. The normalized scaling factor representing the translation error is preset by the closed-loop quality assessment and termination judgment module based on the robot's workspace and accuracy requirements; The normalized scaling factor representing the rotation error is preset by the closed-loop quality assessment and termination judgment module based on the allowable angle deviation of the application scenario; This represents the weighting coefficient for the i-th translation direction, used to adjust the contribution of errors in different translation directions. It is set by the closed-loop quality assessment and termination judgment module according to the current task priority. This represents the weighting coefficient of the j-th rotation axis, used to adjust the contribution of errors of different rotation axes. It is set by the closed-loop quality assessment and termination judgment module according to the current task priority.

[0064] As a concrete example: During the calibration trajectory movement at the robot's end effector, the closed-loop quality assessment and termination judgment module needs to calculate the contour error norm to determine convergence. The module first receives deviation data from the contour error calculation and feature matching module: the translation deviation vector is... =0.15 mm, =0.08 mm, =0.22 mm (corresponding to three translational degrees of freedom); rotational angle deviation vector is =0.005 radians, =0.003 radians, =0.007 radians (corresponding to the three rotation axes). The system's preset normalized scaling factor is... =1.0 mm, =0.01 radians. The current task priority sets the translation direction weight coefficient. =1.0, =0.8, =1.2; Rotation axis weighting coefficient =1.0, =1.1, =0.9. The calculation process is as follows: First, the translation deviation is dimensionless, and the weighted sum of squares is calculated to be 0.0857, which gives approximately 0.293 by taking the square root. Then, the rotation deviation is dimensionless, and the weighted sum of squares is calculated to be 0.79, which gives approximately 0.889 by taking the square root. The final contour error norm is 0.293 + 0.889 = 1.182. The module compares this result with a preset pixel tolerance (e.g., 1.5) as one of the key criteria for determining whether the calibration has converged.

[0065] By normalizing the scale factors to unify the dimensions of translation and rotation errors with different physical dimensions, and introducing configurable weight coefficients to reflect the relative importance of different degrees of freedom in the current task, this calculation formula can generate a comprehensive, balanced, and physically meaningful integrated error metric, thereby providing a reliable and adaptive quantitative basis for accurately judging the convergence state of the calibration process.

[0066] In some implementations, the formula for calculating the weighting coefficient in the translation direction includes:

[0067] Where P represents the total number of image gradient sampling points used to evaluate contour sharpness, which is derived from a preset sampling point distribution scheme around the target contour; This represents the spatial weight coefficient of the k-th sampling point. The closed-loop quality assessment and termination judgment module assigns weights based on the proximity of the sampling point to the edge of the target contour. The closer the sampling point is to the edge, the higher the weight. This represents the image gradient magnitude at the kth sampling point. It is dimensionless and is extracted by the high-precision contour extraction and enhancement module after performing gradient calculation on the preprocessed image. This represents the smoothing factor, a constant greater than zero, used to control the rate at which weights change with sharpness; It represents the standard deviation of pixel values ​​in the target contour region of the current frame image. It is dimensionless and is statistically calculated by the high-precision contour extraction and enhancement module. It is used to characterize the local contrast of the image. This represents the task priority coefficient for the i-th translation direction, is dimensionless, and is preset by the user according to the calibration task requirements. Contour sharpness is the sum of the spatially weighted gradient magnitudes of multiple sampling points within the target contour's neighborhood, and the normalized ratio of the first level to the local contrast of the image.

[0068] As a concrete example: During the robot's movement along the calibrated trajectory, the closed-loop quality assessment and termination decision module needs to calculate weighting coefficients for different translational directions to accurately evaluate contour errors. The module first obtains the workpiece image of the current frame and its pixel value standard deviation within the target contour region from the high-precision contour extraction and enhancement module. The value is 25.6. The preset smoothing factor λ is 1.5, and the task priority coefficient is in the current X-axis translation direction. The value is 1.2. The module selects P=50 sampling points around the edge of the target contour according to a preset sampling point distribution scheme. The calculation process is as follows: For each sampling point k, the module assigns spatial weights based on its Euclidean distance to the contour edge. (The closer the distance, the better) The larger the value, the greater the maximum value (1.0, 0.1), and the gradient magnitude is read from the image gradient map. Then, the sum of the spatially weighted gradient magnitudes of all sampling points is calculated: assuming ∑( · The calculated result is 185.3. Next, the ratio is calculated: 185.3 / (1.5×25.6) = 185.3 / 38.4 ≈ 4.83. The negative exponent of this ratio is calculated: exp(-4.83) ≈ 0.008. The basic weight based on contour sharpness is obtained: 1 - 0.008 = 0.992. Finally, this basic weight is multiplied by the task priority coefficient to obtain the final weight coefficient in the X-axis translation direction. =0.992×1.2≈1.190. This coefficient will be used in the subsequent calculation of the contour error norm, so that the error contribution is appropriately amplified in directions with high contour sharpness.

[0069] By comprehensively considering the contour sharpness of local image regions and the user-preset task priorities, this weighting coefficient calculation method can dynamically adjust the importance of different translation directions in the overall error assessment, giving higher weights to directions with significant image features and high reliability. This guides the system to focus more on the accuracy optimization of key degrees of freedom, improving the effectiveness and task relevance of the calibration results.

[0070] Corresponding to the above system embodiments, this specification also provides embodiments of an intelligent calibration and control method for hand-eye coordination in humanoid robots. Figure 2 A flowchart illustrating an intelligent calibration and control method for hand-eye coordination in a humanoid robot, provided in some embodiments of this specification, is shown. Figure 2 As shown, the specific steps include: Step 201: The robot end effector moves to the calibration area along the initial path.

[0071] Step 202: Acquire images and extract target contours using the high-precision contour extraction and enhancement module.

[0072] Step 203: Calculate the deviation between the current pose and the ideal pose using the contour error calculation and feature matching module.

[0073] Step 204: Generate adjustment instructions through the vision-motion closed-loop control module to drive the robot end effector to correct its pose.

[0074] Step 205: Motion prediction is provided using inertial measurement data through the multi-sensor fusion and pose prediction module to compensate for system delay.

[0075] Step 206: Adjust the movement trajectory according to environmental changes through the dynamic path planning and obstacle avoidance module.

[0076] Step 207: Monitor the entire process through the closed-loop quality assessment and termination judgment module, and automatically terminate the process when the calibration accuracy meets the requirements.

[0077] The above is a schematic scheme of an intelligent calibration and control method for hand-eye coordination of a humanoid robot according to this embodiment. It should be noted that the technical solution of this intelligent calibration and control method for hand-eye coordination of a humanoid robot belongs to the same concept as the technical solution of the intelligent calibration and control system for hand-eye coordination of a humanoid robot described above. Details not described in detail in the technical solution of the intelligent calibration and control method for hand-eye coordination of a humanoid robot can be found in the description of the technical solution of the intelligent calibration and control system for hand-eye coordination of a humanoid robot described above.

[0078] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0079] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this invention. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A humanoid robot hand-eye coordination intelligent calibration control system, characterized in that, include: A high-precision contour extraction and enhancement module is used to acquire and process images to extract the continuous and complete contours of the target object; The contour error calculation and feature matching module is associated with the high-precision contour extraction and enhancement module. It is used to receive the contour and match the contour with the pre-stored calibration template, and calculate the pose deviation and matching quality evaluation index. The vision-motion closed-loop control module, associated with the contour error calculation and feature matching module, is used to generate motion adjustment commands for the robot end effector based on the pose deviation and the matching quality evaluation index. The multi-sensor fusion and pose prediction module, which is associated with the contour error calculation and feature matching module and the vision-motion closed-loop control module, is used to fuse visual data and inertial measurement data to estimate and predict the pose of the robot end effector. The dynamic path planning and obstacle avoidance module, which is associated with the multi-sensor fusion and pose prediction module and the vision-motion closed-loop control module, is used to adjust the motion path of the robot end effector online and avoid obstacles based on the predicted pose and environmental perception data. The closed-loop quality assessment and termination judgment module, which is associated with the contour error calculation and feature matching module and the vision-motion closed-loop control module, is used to monitor the convergence of the calibration process and terminate the calibration process when preset conditions are met.

2. The system according to claim 1, characterized in that, The high-precision contour extraction and enhancement module includes a semantic segmentation network that incorporates an attention mechanism, and also includes an adaptive histogram equalization processing unit and a multi-scale retinal enhancement processing unit for image enhancement.

3. The system according to claim 1, characterized in that, The contour error calculation and feature matching module uses a registration method based on distance transformation and iterative nearest point algorithm to calculate the pose deviation. The matching quality evaluation index includes contour overlap index and Hausdorff distance.

4. The system according to claim 1, characterized in that, The vision-motion closed-loop control module establishes a mapping relationship between the image feature space and the robot joint space through the image Jacobian matrix, and adopts a hybrid control strategy that combines proportional-integral-derivative and fuzzy control. The proportional coefficient is dynamically adjusted based on the contour error calculation and the matching confidence output by the feature matching module.

5. The system according to claim 1, characterized in that, The multi-sensor fusion and pose prediction module uses an extended Kalman filter to fuse visual pose data from the contour error calculation and feature matching module with acceleration and angular velocity data from the inertial measurement unit.

6. The system according to claim 1, characterized in that, The dynamic path planning and obstacle avoidance module uses a fast random tree algorithm for local path planning and combines it with an artificial potential field method to achieve dynamic obstacle avoidance. The environmental perception data is provided by a depth camera.

7. The system according to claim 1, characterized in that, The closed-loop quality assessment and termination judgment module performs convergence judgment based on the uncertainty of the calibration parameters. When the condition number of the calibration equation is lower than the threshold and the contour error norm calculated by the contour error calculation and feature matching module is less than the preset pixel tolerance, the calibration is determined to be converged.

8. The system according to claim 7, characterized in that, The formula for calculating the contour error norm includes: Where N represents the number of translational degrees of freedom, derived from the system's degree of freedom definition; M represents the number of rotational degrees of freedom, derived from the system's degree of freedom definition; The deviation in the i-th translation direction is obtained by decomposing the translation error vector calculated by the contour error calculation and feature matching module. The angular deviation on the j-th rotation axis is obtained by decomposing the rotation error vector calculated by the contour error calculation and feature matching module. The normalized scaling factor representing the translation error is preset by the closed-loop quality assessment and termination judgment module based on the robot's workspace and accuracy requirements; The normalized scaling factor representing the rotation error is preset by the closed-loop quality assessment and termination judgment module according to the allowable angle deviation of the application scenario; The weighting coefficient for the i-th translation direction is used to adjust the contribution of errors in different translation directions and is set by the closed-loop quality assessment and termination judgment module according to the current task priority. The weighting coefficient for the j-th rotation axis is used to adjust the contribution of errors of different rotation axes and is set by the closed-loop quality assessment and termination judgment module according to the current task priority.

9. The system according to claim 8, characterized in that, The formula for calculating the weighting coefficient in the translation direction includes: Where P represents the total number of image gradient sampling points used to evaluate contour sharpness, which is derived from a preset sampling point distribution scheme around the target contour; The spatial weight coefficient of the k-th sampling point is assigned by the closed-loop quality assessment and termination judgment module based on the proximity of the sampling point to the edge of the target contour; the closer to the edge, the higher the weight. This represents the image gradient magnitude at the kth sampling point. It is dimensionless and is extracted by the high-precision contour extraction and enhancement module after performing gradient calculation on the preprocessed image. This represents the smoothing factor, a constant greater than zero, used to control the rate at which weights change with sharpness; It represents the standard deviation of pixel values ​​in the target contour region of the current frame image. It is dimensionless and is statistically calculated by the high-precision contour extraction and enhancement module. It is used to characterize the local contrast of the image. This represents the task priority coefficient for the i-th translation direction. It is dimensionless and is preset by the user according to the calibrated task requirements.

10. A method for intelligent calibration and control of hand-eye coordination in a humanoid robot, characterized in that, The method is applied to the system according to any one of claims 1 to 9, and the method comprises: The robot's end effector moves to the calibrated area along the initial path; The high-precision contour extraction and enhancement module acquires images and extracts target contours. The deviation between the current pose and the ideal pose is calculated by the contour error calculation and feature matching module. The vision-motion closed-loop control module generates adjustment commands to drive the robot's end effector to correct its pose. The multi-sensor fusion and pose prediction module uses inertial measurement data to provide motion prediction and compensate for system delay. The dynamic path planning and obstacle avoidance module adjusts the movement trajectory according to environmental changes. The entire process is monitored by the closed-loop quality assessment and termination judgment module, and the process is automatically terminated when the calibration accuracy meets the requirements.

Images