Robot double-arm hand-eye dynamic adaptive calibration method and device

Abstract

The present invention discloses a method and device for dynamic adaptive calibration of the hand and eye of a robot dual arm. The method comprises: fixing a calibration plate to the ends of the left and right robotic arms respectively, controlling the robotic arms to perform N translation and rotation movements within the camera field of view through an automated program, and collecting an image of the calibration plate after each movement; extracting the position of the corner points of the calibration plate based on the collected image of the calibration plate, and calculating the position and orientation of the calibration plate in the camera coordinate system; obtaining the position and orientation of the calibration plate corresponding to each image of the calibration plate in the robotic arm coordinate system, and solving the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system. The technical solution of the present application realizes high-precision robot dual-arm hand-eye calibration by real-time correction of the position and orientation through dynamic environment perception and vibration compensation, combined with closed-loop reprojection error optimization and spiral / grid path planning, and reduces the failure rate of environmental interference calibration and improves efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and in particular to a method and device for dynamic adaptive calibration of robot arms, hands, and eyes. Background Art

[0002] Robot hand-eye calibration technology is the core foundation of robotic vision guidance and autonomous operation. Its goal is to achieve precise motion control based on visual information by establishing a precise mapping between the camera coordinate system and the robotic arm coordinate system. Traditional hand-eye calibration methods are typically based on a static calibration plate. By controlling the robotic arm to perform a fixed number of translation and rotation movements, the position changes of the calibration plate's corner points in the image are used, combined with calibration functions from open-source libraries such as OpenCV, to calculate the rotation matrix and translation vector. However, these methods have significant limitations.

[0003] Environmental sensitivity issues: Traditional methods assume that the calibration environment is uniformly illuminated and unobstructed. In actual applications, sudden changes in illumination or occlusion by foreign objects can easily lead to failure in corner point detection on the calibration plate, thereby affecting calibration accuracy.

[0004] Error dependence of robotic arm motion: Calibration accuracy is highly dependent on the accuracy of the robotic arm's theoretical motion trajectory. However, slight vibrations or trajectory deviations in actual motion will introduce systematic errors. Especially in the dual-arm collaborative scenario, the error accumulation effect is more obvious.

[0005] Insufficient adaptability of static calibration: Existing methods lack dynamic environmental perception and real-time adjustment capabilities, and are unable to cope with environmental interference or robot arm posture deviation during the calibration process. Multiple calibrations are required to correct parameters, which is inefficient.

[0006] How to properly solve the above problems has become a topic that needs to be solved urgently in the industry. Summary of the Invention

[0007] The present invention provides a method and device for dynamic adaptive calibration of a robot's dual-arm hand-eye system, which is used to correct the posture in real time through dynamic environment perception and vibration compensation, combined with closed-loop reprojection error optimization and spiral / grid path planning to achieve high-precision robot dual-arm hand-eye calibration.

[0008] According to a first aspect of the present invention, a method for dynamic adaptive calibration of a robot's dual-arm hand-eye system is provided, the method comprising:

[0009] The calibration plate is fixed to the end of the left and right robotic arms respectively. The robotic arms are controlled by an automated program to perform N translation and rotation movements within the camera's field of view, and an image of the calibration plate is captured after each movement.

[0010] Based on the collected calibration plate image, extract the calibration plate corner point positions and calculate the calibration plate position in the camera coordinate system;

[0011] The position and orientation of the calibration plate corresponding to each calibration plate image in the manipulator coordinate system are obtained, and the rotation matrix and translation vector of the camera coordinate system and the manipulator coordinate system are calculated.

[0012] In one embodiment, it further includes:

[0013] The system monitors environmental parameters in real time through light sensors and depth cameras. If a sudden change in light intensity or interference from obstructions is detected, the robot's movement path is dynamically adjusted.

[0014] Generate an obstacle avoidance path based on the position data of the obstruction obtained by the depth camera;

[0015] Adjust the angle of the calibration plate at the end of the robotic arm based on the light sensor data to optimize the lighting conditions for image acquisition;

[0016] A new target pose is generated by the manipulator inverse kinematics algorithm, and the movement path is executed.

[0017] In one embodiment, it further includes:

[0018] Calculate the reprojection error in real time. If the reprojection error exceeds a preset threshold, suspend the calibration and re-move the robotic arm.

[0019] After each positioning, call OpenCV's projectPoints function to calculate the reprojection error;

[0020] If the reprojection error exceeds the preset threshold, the robotic arm is triggered to return to the last referenced pose;

[0021] The robot arm's motion trajectory is replanned based on the error distribution to ensure that the calibration plate is in the high-precision imaging area in subsequent steps.

[0022] In one embodiment, it further includes:

[0023] The inertial measurement unit installed at the end of the robotic arm collects the vibration acceleration and angular velocity data of the robotic arm in real time during its movement;

[0024] Calculating the instantaneous position offset of the calibration plate at the moment of image acquisition based on the vibration acceleration and angular velocity data;

[0025] Compensating the instantaneous pose offset to the pose calculation result of the calibration plate in the camera coordinate system to generate corrected pose data;

[0026] The corrected pose data is input into the calibrateHandEye function to optimize the rotation matrix and translation vector.

[0027] In one embodiment, extracting the corner points of the calibration plate using the calibrateCamera function of the OpenCV library includes:

[0028] Use OpenCV's findChessboardCorners function to detect the corner points of the calibration board;

[0029] Improve corner coordinate accuracy through sub-pixel optimization algorithm (cornerSubPix function);

[0030] The corner point data of the calibration plates of the left and right robotic arms are distorted and corrected respectively to generate standardized input data.

[0031] In one embodiment, the robot arm motion trajectory planning includes:

[0032] Use spiral or grid-like paths to cover multiple areas of the camera's field of view;

[0033] After each movement, verify whether the calibration plate is fully imaged. If no complete corner points are detected, re-execute the current movement pose.

[0034] According to a second aspect of the present invention, a robot dual-arm hand-eye dynamic adaptive calibration device is provided, comprising:

[0035] The acquisition module is used to fix the calibration plate to the ends of the left and right robotic arms respectively, and control the robotic arms to perform N translation and rotation movements within the camera's field of view through an automated program, and to capture the calibration plate image after each movement;

[0036] A calculation module is used to extract the corner points of the calibration plate based on the collected calibration plate image and calculate the position of the calibration plate in the camera coordinate system;

[0037] The conversion module is used to obtain the position and posture of the calibration plate corresponding to each calibration plate image in the robotic arm coordinate system, and solve the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system.

[0038] In one embodiment, the acquisition module, the calculation module and the conversion module are controlled to execute any one of the above-mentioned robot dual-arm hand-eye dynamic adaptive calibration methods.

[0039] According to a third aspect of the present invention, there is provided an electronic device, the electronic device comprising: a communication interface, a processor, and a memory;

[0040] Wherein, the memory is used to store program instructions, and when the program instructions are executed by the processor that is communicatively connected to the memory through the communication interface, any of the above-mentioned robot dual-arm hand-eye dynamic adaptive calibration methods is implemented.

[0041] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a computer (for example, a processor in a computer), any of the above-mentioned robot dual-arm hand-eye dynamic adaptive calibration methods is implemented.

[0042] In summary, the present invention provides a method and device for dynamic adaptive calibration of the hand and eye of a robot dual arm. The method includes: fixing a calibration plate at the ends of the left and right robotic arms respectively, controlling the robotic arms to perform N translation and rotation movements within the field of view of a camera through an automated program, and collecting an image of the calibration plate after each movement; based on the collected image of the calibration plate, extracting the position of the corner points of the calibration plate, and calculating the position and orientation of the calibration plate in the camera coordinate system; obtaining the position and orientation of the calibration plate corresponding to each image of the calibration plate in the robotic arm coordinate system, and solving the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system. The technical solution of the present application significantly improves the accuracy, efficiency and robustness of the robot's dual-arm hand-eye calibration through multi-sensor fusion and dynamic closed-loop optimization mechanism: based on the inertial measurement unit (IMU), the robot arm vibration data is collected in real time, and combined with the calibration plate instantaneous posture offset compensation technology, the systematic error caused by micro-vibration is eliminated, and the calibration accuracy is improved to the sub-pixel level; at the same time, the illumination sensor and depth camera data are integrated to dynamically perceive the sudden change of environmental illumination or occlusion interference, and through obstacle avoidance path planning and adaptive adjustment of the robot arm angle, it ensures that the calibration plate is always in the optimal imaging area, reducing the calibration failure rate caused by environmental interference; further introduces the real-time monitoring of reprojection errors and the threshold trigger mechanism, optimizes the spiral / grid motion trajectory, and combines the calibration plate imaging integrity verification to form a "perception-adjustment-verification" closed-loop process, which significantly improves the calibration success rate; in addition, it is compatible with the OpenCV standard library and multiple types of robot arm hardware, and can adapt to high-precision scenarios such as industrial automation and medical robots without complex modifications, and has wide engineering applicability.

[0043] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0044] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A flowchart of a method for dynamic adaptive calibration of a robot's dual-arm hand-eye provided by an embodiment of the present invention;

[0047] Figure 2 An image captured by the left robotic arm camera provided in an embodiment of the present invention;

[0048] Figure 3 An image captured by the right robotic arm camera provided in an embodiment of the present invention;

[0049] Figure 4 A structural diagram of a robot dual-arm hand-eye dynamic adaptive calibration device provided by an embodiment of the present invention;

[0050] Figure 5 A structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0052] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0053] like Figure 1 As shown, the present invention provides a robot dual-arm hand-eye dynamic adaptive calibration method, the robot dual-arm hand-eye dynamic adaptive calibration method includes:

[0054] In step S11, the calibration plate is fixed to the ends of the left and right robotic arms respectively, and the robotic arms are controlled by an automated program to perform N translation and rotation movements within the camera field of view, and an image of the calibration plate is captured after each movement;

[0055] In step S12, based on the collected calibration plate image, the corner points of the calibration plate are extracted, and the position of the calibration plate in the camera coordinate system is calculated;

[0056] In step S13, the position and posture of the calibration plate corresponding to each calibration plate image in the manipulator coordinate system are obtained, and the rotation matrix and translation vector of the camera coordinate system and the manipulator coordinate system are calculated.

[0057] In one embodiment, by fusing multimodal sensors (light sensor, depth camera, inertial measurement unit) to perceive the environment and robotic arm status in real time, combined with dynamic path adjustment, vibration compensation and closed-loop error optimization mechanisms, the environmental sensitivity and accuracy bottlenecks of traditional calibration methods are broken through, providing a reliable technical foundation for high-precision robot operations in complex scenarios.

[0058] In a preferred embodiment, the calibration plate is calibrated using a 6×9 checkerboard and is fixed to the end effectors of the left and right robotic arms, as shown in the attached figure. Figure 2 and attached Figure 3 As shown. To ensure that the plane of the calibration plate is perpendicular to the flange at the end of the robotic arm, it is necessary to fine-tune it through the posture adjustment function of the robotic arm to avoid subsequent posture solution errors caused by tilted installation. The calibration plate can be fixed with a magnetic adsorption device to ensure that no displacement or shaking occurs during the movement of the robotic arm. The robotic arm is controlled by an automated program to perform translation and rotation movements within the field of view of the camera. The number of movements N usually needs to be greater than 10 times. In a preferred embodiment, the number of movements is set to 15 times, and the movement trajectory adopts a spiral or grid path. For example, the spiral path starts from the center of the field of view, and the end of the robotic arm expands outward along the spiral line. The step length of each translation is 10 cm and the step length of each rotation is 15 degrees. After each movement, the robotic arm pauses for 0.5 seconds to stabilize the posture, and then triggers the camera to capture the image of the calibration plate. For example, suppose that in a certain calibration, the left robotic arm performs a spiral path movement. After the fifth movement, the calibration plate is located in the upper right area of the field of view. At this point, the camera successfully captured an image containing the complete checkerboard corner points. However, during the eighth movement, the robotic arm joint slightly jammed, causing the calibration plate to partially move out of the field of view. This immediately triggered path replanning, and the robotic arm returned to its previous position and re-executed the movement to ensure data validity.

[0059] After the camera captures the calibration plate image, image preprocessing is performed. A Gaussian filter is used to remove environmental noise (such as camera sensor noise or noise caused by uneven lighting). The Canny edge detection algorithm is then used to enhance the edge contours of the calibration plate. OpenCV's findChessboardCorners function is called to detect the calibration plate's corners. If detection is successful, the cornerSubPix function is further applied for sub-pixel optimization, improving the corner coordinate accuracy to within 0.5 pixels. After corner detection, image distortion correction is required. Based on the pre-calibrated camera intrinsic parameters (focal length fx, fy, optical center cx, cy) and distortion coefficients (radial distortion k1, k2, tangential distortion p1, p2), the corner data of the left and right robotic arms are corrected separately to generate standardized inputs. For example, the coordinates of a corner point before correction are (300.5, 250.2), and after correction, they are (299.8, 249.7), eliminating systematic errors caused by lens distortion. Finally, the calibrateCamera function is called to calculate the calibration plate's pose in the camera coordinate system (rotation matrix R_cam and translation vector T_cam). Simultaneously, the robot control system obtains the calibration plate's theoretical pose in the robot base coordinate system (R_robot, T_robot). For example, during one calibration, the calibration plate at the end of the right robot arm was slightly tilted due to installation error. After sub-pixel optimization and distortion correction, the pose error was reduced from an initial 1.2 mm to 0.3 mm.

[0060] During the calibration process, the light sensor and depth camera monitor environmental changes in real time. The light sensor collects ambient light intensity data 10 times per second. If a sudden change in light intensity is detected (e.g., a drop from 500 Lux to 200 Lux), the robot automatically adjusts the angle of the calibration plate at the end of the robotic arm. For example, the robot arm is controlled to rotate 30 degrees around the Z axis so that the calibration plate faces the auxiliary light source to avoid dark images due to backlight. The depth camera uses time-of-flight (ToF) technology to generate a scene depth map and detect obstructions in its field of view in real time. When an obstruction (such as a temporarily placed tool or a person's arm) is detected, a path planning algorithm is invoked to generate an obstacle avoidance path. For example, during one calibration test, the depth map indicated an obstacle 20 cm to the left. The robot automatically shifted 15 cm to the right, bypassed the obstacle, and then resumed its original trajectory. For example, during a calibration test in an industrial workshop, unexpected equipment reflections caused the calibration plate to overexpose. When the light sensor detects that the light intensity exceeds 1000 Lux, the end of the robotic arm automatically pitches -15 degrees to reduce the reflective area, and image acquisition returns to normal.

[0061] During motion, the robotic arm may generate minute vibrations due to motor startup and shutdown or gear backlash. To eliminate the effects of vibration on the calibration pose, an inertial measurement unit (IMU) is installed at the end of the robotic arm to collect vibration acceleration and angular velocity data in real time. The instantaneous displacement offset (e.g., 0.2 mm offset on the X-axis) and angular offset (e.g., 0.5 degrees offset around the Z-axis) caused by vibration are calculated by integration. These offsets are converted into calibration plate pose corrections, and the pose data in the camera coordinate system is updated. For example, the original pose is (X=100.0 mm, Y=50.0 mm, θ=30°), and the corrected pose data is (X=100.2 mm, Y=50.1 mm, θ=30.5°). This corrected pose data is input into the calibrateHandEye function, which recalculates the rotation matrix and translation vector of the camera and robotic arm coordinate systems, and iteratively optimizes the calibration parameters.

[0062] After each calibration pose, call OpenCV's projectPoints function to calculate the reprojection error. Project the 3D corner points of the calibration plate onto the image plane to obtain theoretical pixel coordinates, which are then compared with the actual detected coordinates. If the average error exceeds 1.5 pixels, calibration is paused and the following actions are triggered. Robotic arm retraction: Controls the robotic arm to return to the last calibration pose to avoid error accumulation; Error analysis: Adjusts the motion trajectory step size or path density based on the error distribution (e.g., if the error is concentrated at the edge of the field of view); Path replanning: For example, reduce the spiral path step size from 10 cm to 5 cm and increase the number of sampling points in the center of the field of view.

[0063] After each movement of the robotic arm, the camera image is used to verify that the calibration plate is fully imaged. If a complete corner point is not detected (for example, if the calibration plate partially moves out of view or a corner point is obscured), the following actions are immediately performed: Re-execute the current pose: The robotic arm returns to its current position and re-captures the image; Path fine-tuning: If two consecutive captures fail, the motion trajectory is automatically adjusted (for example, by reducing the rotation angle or translation step size).

[0064] The technical solution in this embodiment can achieve high-precision and high-robustness hand-eye calibration through the following key technologies: automated motion control: spiral / grid paths cover the camera's field of view, combined with integrity verification to ensure data comprehensiveness; multi-sensor collaboration: light sensors, depth cameras and IMUs monitor the environment and robotic arm status in real time; dynamic compensation mechanism: vibration compensation, obstacle avoidance path planning and closed-loop error optimization form a complete adaptive process; deep algorithm integration: seamless connection between the OpenCV function library and the robotic arm control system to improve calibration efficiency.

[0065] The technical solution in this embodiment significantly improves the accuracy, efficiency and robustness of the robot's dual-arm hand-eye calibration through multi-sensor fusion and dynamic closed-loop optimization mechanism: based on the inertial measurement unit (IMU), the robot arm vibration data is collected in real time, and combined with the calibration plate instantaneous posture offset compensation technology, the systematic error caused by micro-vibration is eliminated, and the calibration accuracy is improved to the sub-pixel level; at the same time, the illumination sensor and depth camera data are integrated to dynamically perceive the sudden change of environmental illumination or occlusion interference, and through obstacle avoidance path planning and adaptive adjustment of the robot arm angle, it ensures that the calibration plate is always in the optimal imaging area, reducing the calibration failure rate caused by environmental interference; further introduces real-time monitoring of reprojection errors and threshold triggering mechanism, optimizes the spiral / grid motion trajectory, and combines the calibration plate imaging integrity verification to form a "perception-adjustment-verification" closed-loop process, which significantly improves the calibration success rate; in addition, it is compatible with the OpenCV standard library and multiple types of robot arm hardware, and can adapt to high-precision scenarios such as industrial automation and medical robots without complex modifications, and has wide engineering applicability.

[0066] In one embodiment, Figure 4 FIG1 is a block diagram of a robot dual-arm hand-eye dynamic adaptive calibration device according to an exemplary embodiment. Figure 4 As shown, the robot dual-arm hand-eye dynamic adaptive calibration device includes an acquisition module 41, a calculation module 42 and a conversion module 43.

[0067] The acquisition module 41 is used to fix the calibration plate to the ends of the left and right robotic arms respectively, and control the robotic arms to perform N translation and rotation movements within the camera field of view through an automated program, and to acquire an image of the calibration plate after each movement;

[0068] The calculation module 42 is used to extract the corner points of the calibration plate based on the collected calibration plate image through the calibrateCamera function of the OpenCV library, and calculate the position of the calibration plate in the camera coordinate system;

[0069] The conversion module 43 is used to obtain the position and posture of the calibration plate corresponding to each calibration plate image in the robot coordinate system, call the calibrateHandEye function of the OpenCV library, and solve the rotation matrix and translation vector of the camera coordinate system and the robot coordinate system.

[0070] The acquisition module 41, the calculation module 42 and the conversion module 43 included in the block diagram of the robot dual-arm hand-eye dynamic adaptive calibration device are controlled to execute the robot dual-arm hand-eye dynamic adaptive calibration method described in any of the above embodiments.

[0071] like Figure 5 As shown, the present invention provides an electronic device 500, which includes: a communication interface, a processor 501, and a memory 502;

[0072] In which, the memory 502 is used to store program instructions. When the program instructions are executed by the processor 501 that is communicatively connected to the memory 502 through the communication interface, the calibration plate is fixed to the ends of the left and right robotic arms respectively, and the robotic arms are controlled by an automated program to perform N translation and rotation movements within the camera field of view, and the calibration plate image is collected after each movement; based on the collected calibration plate image, the position of the calibration plate corner points is extracted, and the position and orientation of the calibration plate in the camera coordinate system is calculated; the position and orientation of the calibration plate corresponding to each calibration plate image in the robotic arm coordinate system is obtained, and the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system are solved.

[0073] The present invention provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, a calibration plate is fixed to the ends of a left and a right robotic arm, respectively. The robotic arm is controlled by an automated program to perform N translation and rotation movements within the field of view of a camera, and an image of the calibration plate is captured after each movement. Based on the captured image of the calibration plate, the position of the corner points of the calibration plate is extracted, and the position and orientation of the calibration plate in the camera coordinate system is calculated. The position and orientation of the calibration plate corresponding to each image of the calibration plate in the robotic arm coordinate system is obtained, and the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system are solved.

[0074] It should be understood that the specific features, operations and details described herein above with respect to the method of the present invention may also be similarly applied to the apparatus and system of the present invention, or vice versa. In addition, each step of the method of the present invention described above may be performed by the corresponding components or units of the apparatus or system of the present invention.

[0075] It should be understood that the various modules / units of the apparatus of the present invention may be implemented in whole or in part by software, hardware, firmware, or a combination thereof. Each module / unit may be embedded in a processor of a computer device in the form of hardware or firmware or may be independent of the processor, or may be stored in a memory of a computer device in the form of software for the processor to call to execute the operations of each module / unit. Each module / unit may be implemented as an independent component or module, or two or more modules / units may be implemented as a single component or module.

[0076] In one embodiment, a computer device is provided, comprising a memory and a processor. The memory stores computer instructions executable by the processor, which, when executed by the processor, instruct the processor to perform the steps of the method according to an embodiment of the present invention. The computer device can be broadly defined as a server, a terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device can include a processor, memory, a network interface, a communication interface, etc. connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device can include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. can be stored in or on the non-volatile storage medium. The internal memory can provide an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect to and communicate with external devices via a network. When the computer program is executed by the processor, the steps of the method according to the present invention are performed.

[0077] The present invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the method of an embodiment of the present invention to be performed. In one embodiment, the computer program is distributed on a plurality of computer devices or processors coupled to a network so that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, can be performed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations can be performed by one or more computer devices or processors, and one or more other method steps / operations can be performed by one or more other computer devices or processors. One or more computer devices or processors can perform a single method step / operation, or perform two or more method steps / operations.

[0078] Those skilled in the art will appreciate that the method steps of the present invention can be performed by instructing related hardware, such as a computer device or processor, through a computer program. The computer program can be stored in a non-transitory computer-readable storage medium, and when the computer program is executed, the steps of the present invention are performed. Depending on the circumstances, any reference herein to memory, storage, database, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0079] The various technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification as long as such combination does not conflict.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot dual-arm hand-eye dynamic adaptive calibration method, characterized in that: include: The calibration plate is fixed to the end of the left and right robotic arms respectively. The robotic arms are controlled by an automated program to perform N translation and rotation movements within the camera's field of view, and an image of the calibration plate is captured after each movement. Based on the collected calibration plate image, extract the calibration plate corner point positions and calculate the calibration plate position in the camera coordinate system; Obtain the position and posture of the calibration plate corresponding to each calibration plate image in the robotic arm coordinate system, and solve the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system; Also includes: The inertial measurement unit installed at the end of the robotic arm collects the vibration acceleration and angular velocity data of the robotic arm in real time during its movement; Calculating the instantaneous position offset of the calibration plate at the moment of image acquisition based on the vibration acceleration and angular velocity data; Compensating the instantaneous pose offset to the pose calculation result of the calibration plate in the camera coordinate system to generate corrected pose data; The rotation matrix and translation vector are optimized according to the corrected pose number.

2. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: Also includes: The system monitors environmental parameters in real time through light sensors and depth cameras. If a sudden change in light intensity or interference from obstructions is detected, the robot's movement path is dynamically adjusted. Generate an obstacle avoidance path based on the position data of the obstruction obtained by the depth camera; Adjust the angle of the calibration plate at the end of the robotic arm based on the light sensor data to optimize the lighting conditions for image acquisition; A new target pose is generated by the manipulator inverse kinematics algorithm, and the movement path is executed.

3. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: Also includes: Calculate the reprojection error in real time. If the reprojection error exceeds a preset threshold, suspend the calibration and re-move the robotic arm. After each positioning, call OpenCV's projectPoints function to calculate the reprojection error; If the reprojection error exceeds the preset threshold, the robotic arm is triggered to return to the last referenced pose; The robot arm's motion trajectory is replanned based on the error distribution to ensure that the calibration plate is in the high-precision imaging area in subsequent steps.

4. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: The step of extracting the corner points of the calibration plate includes: Use OpenCV's findChessboardCorners function to detect the corner points of the calibration board; Improve corner coordinate accuracy through sub-pixel optimization algorithm (cornerSubPix function); The corner point data of the calibration plates of the left and right robotic arms are distorted and corrected respectively to generate standardized input data.

5. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: Also includes: Use spiral or grid-like paths to cover multiple areas of the camera's field of view; After each movement, verify whether the calibration plate is fully imaged. If no complete corner points are detected, re-execute the current movement pose.

6. A robot dual-arm hand-eye dynamic adaptive calibration device, characterized in that: include: The acquisition module is used to fix the calibration plate to the ends of the left and right robotic arms respectively, and control the robotic arms to perform N translation and rotation movements within the camera's field of view through an automated program, and to capture the calibration plate image after each movement; A calculation module is used to extract the corner points of the calibration plate based on the collected calibration plate image and calculate the position of the calibration plate in the camera coordinate system; A conversion module is used to obtain the position and posture of the calibration plate corresponding to each calibration plate image in the manipulator coordinate system, and to calculate the rotation matrix and translation vector of the camera coordinate system and the manipulator coordinate system; The device is also used to collect vibration acceleration and angular velocity data of the robotic arm in real time during its movement through an inertial measurement unit installed at the end of the robotic arm; calculate the instantaneous posture offset of the calibration plate at the moment of image acquisition based on the vibration acceleration and angular velocity data; compensate the instantaneous posture offset to the posture calculation result of the calibration plate in the camera coordinate system to generate corrected posture data; and optimize the rotation matrix and translation vector according to the corrected posture data.

7. The robot dual-arm hand-eye dynamic adaptive calibration device according to claim 6, characterized in that: The acquisition module, the calculation module and the conversion module are controlled to execute the robot dual-arm hand-eye dynamic adaptive calibration method according to any one of claims 1 to 5.

8. An electronic device, characterized in that: include: Communication interface, processor, memory; Wherein, the memory is used to store program instructions, and when the program instructions are executed by the processor that is communicatively connected to the memory through the communication interface, the electronic device implements the robot dual-arm hand-eye dynamic adaptive calibration method described in any one of claims 1 to 5.

9. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a computer, the computer implements the robot dual-arm hand-eye dynamic adaptive calibration method according to any one of claims 1 to 5.

Images