Robot double-arm hand-eye dynamic self-adaptive calibration method and device

Abstract

The invention discloses a dynamic self-adaptive calibration method and device for hands and eyes of two arms of a robot. The method comprises the steps that calibration plates are fixed to the tail end of a left mechanical arm and the tail end of a right mechanical arm respectively, the mechanical arms are controlled through an automatic program to execute N times of translation and rotation movement in the view field of a camera, and calibration plate images are collected after each time of movement; based on the collected calibration plate image, the angular point position of the calibration plate is extracted, and the pose of the calibration plate in a camera coordinate system is calculated; and the pose of the calibration plate corresponding to each calibration plate image in the mechanical arm coordinate system is obtained, and the rotation matrix and the translation vector of the camera coordinate system and the mechanical arm coordinate system are solved. According to the technical scheme, the pose is corrected in real time through dynamic environment perception and vibration compensation, closed-loop re-projection error optimization and spiral / grid path planning are combined, high-precision robot double-arm hand-eye calibration is achieved, the environment interference calibration failure rate is reduced, and efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and in particular to a robot double-arm hand-eye dynamic adaptive calibration method and device. Background Art

[0002] Robot hand-eye calibration technology is the core foundation of robot vision guidance and autonomous operation. Its goal is to achieve precise motion control based on visual information by establishing an accurate mapping relationship between the camera coordinate system and the robot arm coordinate system. Traditional hand-eye calibration methods are usually based on static calibration plates. By controlling the robot arm to perform a fixed number of translation and rotation movements, the position changes of the corner points of the calibration plate in the image are used, combined with calibration functions of open source libraries such as OpenCV to solve the rotation matrix and translation vector. However, this type of method has significant limitations.

[0003] Environmental sensitivity issues: Traditional methods assume that the calibration environment has uniform lighting and no obstructions. In actual applications, sudden changes in lighting or obstructions from foreign objects can easily lead to failure in corner point detection on the calibration plate, thus affecting the calibration accuracy.

[0004] Error dependence of robot motion: The calibration accuracy is highly dependent on the theoretical motion trajectory accuracy of the robot, while tiny vibrations or trajectory deviations in actual motion will introduce systematic errors. Especially in the dual-arm collaboration scenario, the error accumulation effect is more obvious.

[0005] Static calibration lacks adaptability: Existing methods lack dynamic environment perception and real-time adjustment capabilities, and are unable to cope with environmental interference or robot arm posture deviation during the calibration process. They need to be repeatedly calibrated to correct parameters, which is inefficient.

[0006] How to properly solve the above problems has become an urgent issue to be solved in the industry. Summary of the invention

[0007] The present invention provides a robot dual-arm hand-eye dynamic adaptive calibration method and device, which are used to correct the posture in real time through dynamic environment perception and vibration compensation, combined with closed-loop reprojection error optimization and spiral / grid path planning, to achieve high-precision robot dual-arm hand-eye calibration.

[0008] According to a first aspect of the present invention, a robot dual-arm hand-eye dynamic adaptive calibration method is provided, and the robot dual-arm hand-eye dynamic adaptive calibration method comprises: The calibration plate is fixed to the ends of the left and right robotic arms respectively, and the robotic arms are controlled by an automated program to perform N translation and rotation movements within the camera field of view, and the calibration plate image is collected after each movement; Based on the collected calibration plate image, extract the corner point positions of the calibration plate, and calculate the position and posture of the calibration plate in the camera coordinate system; The position and posture of the calibration plate corresponding to each calibration plate image in the robot arm coordinate system are obtained, and the rotation matrix and translation vector of the camera coordinate system and the robot arm coordinate system are solved.

[0009] In one embodiment, it further includes: The environmental parameters are monitored in real time through the light sensor and depth camera. If a sudden change in light intensity or interference from an obstruction is detected, the movement path of the robotic arm is dynamically adjusted. Generate an obstacle avoidance path based on the position data of the obstruction obtained by the depth camera; According to the light sensor data, adjust the angle of the calibration plate at the end of the robot arm to optimize the lighting conditions for image acquisition; A new target posture is generated by the robot arm inverse kinematics algorithm, and the movement path is executed.

[0010] In one embodiment, it further includes: Calculate the reprojection error in real time, and if the reprojection error exceeds a preset threshold, suspend the calibration and re-move the robotic arm; After each positioning, call OpenCV's projectPoints function to calculate the reprojection error; If the reprojection error exceeds the preset threshold, the robotic arm is triggered to return to the last calibration posture; The robot arm motion trajectory is replanned based on the error distribution to ensure that the calibration plate is in the high-precision imaging area in the subsequent steps.

[0011] In one embodiment, it further includes: The inertial measurement unit installed at the end of the robotic arm collects the vibration acceleration and angular velocity data of the robotic arm in real time during its movement. Based on the vibration acceleration and angular velocity data, calculating the instantaneous position offset of the calibration plate at the moment of image acquisition; Compensating the instantaneous posture offset to the posture calculation result of the calibration plate in the camera coordinate system to generate corrected posture data; The corrected pose data is input into the calibrateHandEye function to optimize the rotation matrix and translation vector.

[0012] In one embodiment, extracting the corner point positions of the calibration plate through the calibrateCamera function of the OpenCV library includes: Use OpenCV's findChessboardCorners function to detect the corners of the calibration board; Improve the accuracy of corner coordinates through sub-pixel optimization algorithm (cornerSubPix function); The corner point data of the calibration plates of the left and right robotic arms are distorted and corrected respectively to generate standardized input data.

[0013] In one embodiment, the robot arm motion trajectory planning includes: Use spiral or grid-like paths to cover multiple areas of the camera's field of view; After each movement, verify whether the calibration plate is fully imaged. If no complete corner points are detected, re-execute the current movement posture.

[0014] According to a second aspect of the present invention, a robot dual-arm hand-eye dynamic adaptive calibration device is provided, comprising: The acquisition module is used to fix the calibration plate at the ends of the left and right robotic arms respectively, control the robotic arms to perform N translation and rotation movements within the camera field of view through an automated program, and acquire the calibration plate image after each movement; A calculation module, used to extract the corner points of the calibration plate based on the acquired calibration plate image, and calculate the position and posture of the calibration plate in the camera coordinate system; The conversion module is used to obtain the position and posture of the calibration plate corresponding to each calibration plate image in the robot arm coordinate system, and solve the rotation matrix and translation vector of the camera coordinate system and the robot arm coordinate system.

[0015] In one embodiment, the acquisition module, the calculation module and the conversion module are controlled to execute any one of the above-mentioned robot dual-arm hand-eye dynamic adaptive calibration methods.

[0016] According to a third aspect of the present invention, there is provided an electronic device, the electronic device comprising: a communication interface, a processor, and a memory; Wherein, the memory is used to store program instructions, and when the program instructions are executed by the processor that is communicatively connected to the memory through the communication interface, any of the above-mentioned robot dual-arm hand-eye dynamic adaptive calibration methods is implemented.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a computer (for example, a processor in the computer), any of the above-mentioned robot dual-arm hand-eye dynamic adaptive calibration methods is implemented.

[0018] In summary, the present invention provides a robot dual-arm hand-eye dynamic adaptive calibration method and device, the method comprising: fixing the calibration plate at the ends of the left and right robotic arms respectively, controlling the robotic arms to perform N translation and rotation movements within the field of view of the camera through an automated program, and collecting the calibration plate image after each movement; based on the collected calibration plate image, extracting the position of the calibration plate corner points, and calculating the position and posture of the calibration plate in the camera coordinate system; obtaining the position and posture of the calibration plate corresponding to each calibration plate image in the robotic arm coordinate system, and solving the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system. The technical solution of the present application significantly improves the accuracy, efficiency and robustness of the robot's dual-arm hand-eye calibration through multi-sensor fusion and dynamic closed-loop optimization mechanism: based on the inertial measurement unit IMU, the robot arm vibration data is collected in real time, combined with the calibration plate instantaneous posture offset compensation technology, the systematic error caused by micro-vibration is eliminated, and the calibration accuracy is improved to the sub-pixel level; at the same time, the light sensor and the depth camera data are integrated to dynamically perceive the sudden change of environmental light or occlusion interference, and the obstacle avoidance path planning and the robot arm angle adaptive adjustment are used to ensure that the calibration plate is always in the optimal imaging area, thereby reducing the calibration failure rate caused by environmental interference; further introduce the real-time monitoring of reprojection errors and the threshold trigger mechanism, optimize the spiral / grid motion trajectory, and combine the calibration plate imaging integrity verification to form a "perception-adjustment-verification" closed-loop process, which significantly improves the calibration success rate; in addition, it is compatible with the OpenCV standard library and multiple types of robot arm hardware, and can adapt to high-precision scenarios such as industrial automation and medical robots without complex modifications, and has a wide range of engineering applicability.

[0019] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0020] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flowchart of a robot dual-arm hand-eye dynamic adaptive calibration method provided by an embodiment of the present invention; Figure 2An image captured by a left robotic arm camera provided by an embodiment of the present invention; Figure 3 An image taken by a camera of the right mechanical arm provided by an embodiment of the present invention; Figure 4 A structural diagram of a robot dual-arm hand-eye dynamic adaptive calibration device provided by an embodiment of the present invention; Figure 5 A structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0024] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0025] like Figure 1 As shown, the present invention provides a robot dual-arm hand-eye dynamic adaptive calibration method, the robot dual-arm hand-eye dynamic adaptive calibration method comprises: In step S11, the calibration plate is fixed to the ends of the left and right robotic arms respectively, and the robotic arms are controlled by an automated program to perform N translation and rotation movements within the camera field of view, and an image of the calibration plate is captured after each movement; In step S12, based on the acquired calibration plate image, the corner point positions of the calibration plate are extracted, and the position and posture of the calibration plate in the camera coordinate system are calculated; In step S13, the position and posture of the calibration plate corresponding to each calibration plate image in the robot arm coordinate system is obtained, and the rotation matrix and translation vector of the camera coordinate system and the robot arm coordinate system are calculated.

[0026] In one embodiment, by fusing multimodal sensors (light sensor, depth camera, inertial measurement unit) to perceive the environment and robotic arm status in real time, combined with dynamic path adjustment, vibration compensation and closed-loop error optimization mechanism, the environmental sensitivity and accuracy bottlenecks of traditional calibration methods are broken through, providing a reliable technical foundation for high-precision robot operations in complex scenarios.

[0027] In a preferred embodiment, the calibration plate is calibrated with a 6×9 checkerboard and is fixed to the end effectors of the left and right robotic arms, as shown in the attached figure. Figure 2 and attached Figure 3 As shown. To ensure that the plane of the calibration plate is perpendicular to the flange at the end of the robot arm, it is necessary to fine-tune it through the posture adjustment function of the robot arm to avoid subsequent posture solution errors caused by tilted installation. The calibration plate can be fixed by a magnetic adsorption device to ensure that no displacement or shaking occurs during the movement of the robot arm. The robot arm is controlled by an automated program to perform translation and rotation movements within the field of view of the camera. The number of movements N usually needs to be greater than 10 times. In a preferred embodiment, the number of movements is set to 15 times, and the movement trajectory adopts a spiral or grid path. For example, the spiral path starts from the center of the field of view, and the end of the robot arm expands outward along the spiral line. The translation step length is 10 cm each time and the rotation step length is 15 degrees. After each movement, the robot arm pauses for 0.5 seconds to stabilize the posture, and then triggers the camera to collect the calibration plate image. For example, suppose that in a certain calibration, the left robot arm performs a spiral path movement. After the fifth movement, the calibration plate is located in the upper right area of ​​the field of view. At this point, the camera successfully captured an image containing the complete chessboard corner points. However, during the eighth movement, the robotic arm joint was slightly stuck, and part of the calibration plate moved out of the field of view, immediately triggering path re-planning. The robotic arm returned to the previous position and re-executed the movement to ensure data validity.

[0028] After the camera captures the image of the calibration plate, it performs image preprocessing. The environmental noise (such as noise caused by camera sensor noise or uneven lighting) is eliminated by Gaussian filtering, and then the Canny edge detection algorithm is used to enhance the edge contour of the calibration plate. The findChessboardCorners function of OpenCV is called to detect the corners of the calibration plate. If the detection is successful, the cornerSubPix function is further applied for sub-pixel optimization to improve the accuracy of the corner coordinates to within 0.5 pixels. After completing the corner detection, the image distortion needs to be corrected. According to the pre-calibrated camera intrinsic parameters (focal length fx, fy, optical center cx, cy) and distortion coefficients (radial distortion k1, k2, tangential distortion p1, p2), the corner data of the left and right manipulators are corrected separately to generate standardized input. For example, the coordinates of a corner point before correction are (300.5, 250.2), and after correction are (299.8, 249.7), eliminating the systematic error caused by lens distortion. Finally, the calibrateCamera function is called to calculate the position and pose of the calibration plate in the camera coordinate system (rotation matrix R_cam and translation vector T_cam). At the same time, the theoretical position and pose of the calibration plate in the robot base coordinate system (R_robot, T_robot) is obtained through the robot control system. For example, in one calibration, the calibration plate at the end of the right robot arm was slightly tilted due to installation error. After sub-pixel optimization and distortion correction, the position and pose solution error was reduced from the initial 1.2 mm to 0.3 mm.

[0029] During the calibration process, the light sensor and depth camera monitor environmental changes in real time. The light sensor collects ambient light intensity data at a frequency of 10 times per second. If a sudden change in light is detected (such as a sudden drop from 500 Lux to 200 Lux), the angle adjustment of the calibration plate at the end of the robotic arm is automatically triggered. For example, the robotic arm is controlled to rotate 30 degrees around the Z axis so that the plane of the calibration plate faces the auxiliary light source to avoid the image being too dark due to backlight. The depth camera generates a scene depth map through time-of-flight (ToF) technology and detects obstructions in the field of view in real time. When an obstruction (such as a temporarily placed tool or a person's arm) is detected, the path planning algorithm is called to generate an obstacle avoidance path. For example, in a calibration, the depth map shows that there is an obstacle 20 cm to the left, and the robotic arm automatically translates 15 cm to the right, bypasses the obstacle, and continues to execute the original motion trajectory. For example, in an industrial workshop calibration test, sudden equipment reflections caused the calibration plate to be overexposed. When the light sensor detects that the light intensity exceeds 1000 Lux, the end of the robotic arm automatically pitches -15 degrees to reduce the reflective area, and image acquisition returns to normal.

[0030] During the movement, the robot arm may produce slight vibrations due to motor start and stop or gear clearance. In order to eliminate the influence of vibration on the calibration posture, an inertial measurement unit (IMU) is installed at the end of the robot arm to collect vibration acceleration and angular velocity data in real time. The instantaneous displacement offset (such as 0.2 mm offset on the X axis) and the angle offset (such as 0.5 degrees offset around the Z axis) caused by vibration are calculated by integration. After converting the offset into the calibration plate posture correction, the posture data in the camera coordinate system is updated. For example, the original posture is (X=100.0mm, Y=50.0mm, θ=30°), and the corrected posture is (X=100.2mm, Y=50.1mm, θ=30.5°). The corrected posture data is input into the calibrateHandEye function, the rotation matrix and translation vector of the camera and robot arm coordinate system are recalculated, and the calibration parameters are iteratively optimized.

[0031] After each calibration pose, call OpenCV's projectPoints function to calculate the reprojection error. Project the 3D corner points of the calibration plate onto the image plane to obtain the theoretical pixel coordinates, which are compared with the actual detection coordinates. If the average error exceeds 1.5 pixels, suspend calibration and trigger the following operations. Robotic arm retraction: control the robotic arm to return to the last calibration pose to avoid error accumulation; error analysis: adjust the motion trajectory step size or path density according to the error distribution (such as the error is concentrated at the edge of the field of view); path replanning: for example, reduce the spiral path step size from 10 cm to 5 cm and increase the sampling points in the center area of ​​the field of view.

[0032] After each movement of the robot arm, the camera image is used to verify whether the calibration plate is fully imaged. If no complete corner point is detected (such as the calibration plate is partially out of view or the corner point is blocked), the following operations are performed immediately: Re-execute the current posture: the robot arm returns to the current position and re-captures the image; Path fine-tuning: if two consecutive acquisitions fail, automatically adjust the motion trajectory (such as reducing the rotation angle or translation step length).

[0033] The technical solution in this embodiment can achieve high-precision and high-robustness hand-eye calibration through the following key technologies: automated motion control: spiral / grid paths cover the camera's field of view, combined with integrity verification to ensure data comprehensiveness; multi-sensor collaboration: light sensors, depth cameras and IMUs monitor the environment and robotic arm status in real time; dynamic compensation mechanism: vibration compensation, obstacle avoidance path planning and closed-loop error optimization form a complete adaptive process; deep algorithm integration: seamless connection between the OpenCV function library and the robotic arm control system to improve calibration efficiency.

[0034] The technical solution in this embodiment significantly improves the accuracy, efficiency and robustness of the robot's dual-arm hand-eye calibration through multi-sensor fusion and dynamic closed-loop optimization mechanism: based on the inertial measurement unit IMU, the robot arm vibration data is collected in real time, combined with the calibration plate instantaneous posture offset compensation technology, the systematic error caused by micro-vibration is eliminated, and the calibration accuracy is improved to the sub-pixel level; at the same time, the light sensor and the depth camera data are integrated to dynamically perceive the sudden change of environmental light or occlusion interference, and the obstacle avoidance path planning and the robot arm angle adaptive adjustment are used to ensure that the calibration plate is always in the optimal imaging area, thereby reducing the calibration failure rate caused by environmental interference; further introduce the real-time monitoring of reprojection errors and the threshold trigger mechanism, optimize the spiral / grid motion trajectory, and combine the calibration plate imaging integrity verification to form a "perception-adjustment-verification" closed-loop process, and the calibration success rate is significantly improved; in addition, it is compatible with the OpenCV standard library and multiple types of robot arm hardware, and can adapt to high-precision scenarios such as industrial automation and medical robots without complex modifications, and has a wide range of engineering applicability.

[0035] In one embodiment, Figure 4 FIG. 1 is a block diagram of a robot dual-arm hand-eye dynamic adaptive calibration device according to an exemplary embodiment. Figure 4 As shown, the robot dual-arm hand-eye dynamic adaptive calibration device includes a collection module 41, a calculation module 42 and a conversion module 43.

[0036] The acquisition module 41 is used to fix the calibration plate to the ends of the left and right mechanical arms respectively, control the mechanical arms to perform N translation and rotation movements within the camera field of view through an automated program, and acquire the calibration plate image after each movement; The calculation module 42 is used to extract the corner point positions of the calibration plate based on the collected calibration plate image through the calibrateCamera function of the OpenCV library, and calculate the position and posture of the calibration plate in the camera coordinate system; The conversion module 43 is used to obtain the position and posture of the calibration plate corresponding to each calibration plate image in the robot arm coordinate system, call the calibrateHandEye function of the OpenCV library, and solve the rotation matrix and translation vector of the camera coordinate system and the robot arm coordinate system.

[0037] The acquisition module 41, the calculation module 42 and the conversion module 43 included in the block diagram of the robot dual-arm hand-eye dynamic adaptive calibration device are controlled to execute the robot dual-arm hand-eye dynamic adaptive calibration method described in any of the above embodiments.

[0038] like Figure 5 As shown, the present invention provides an electronic device 500, the electronic device comprising: a communication interface, a processor 501, and a memory 502; Among them, the memory 502 is used to store program instructions. When the program instructions are executed by the processor 501 that is communicatively connected to the memory 502 through the communication interface, the calibration plate is fixed to the ends of the left and right robotic arms respectively, and the robotic arms are controlled by an automated program to perform N translation and rotation movements within the field of view of the camera, and the calibration plate image is collected after each movement; based on the collected calibration plate image, the position of the calibration plate corner points is extracted, and the position and posture of the calibration plate in the camera coordinate system is calculated; the position and posture of the calibration plate corresponding to each calibration plate image in the robotic arm coordinate system is obtained, and the rotation matrix and translation vector of the camera coordinate system and the robotic arm coordinate system are solved.

[0039] The present invention provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, a calibration plate is fixed to the ends of a left and a right robotic arm respectively, and the robotic arm is controlled by an automated program to perform N translation and rotation movements within a camera field of view, and an image of the calibration plate is collected after each movement; based on the collected image of the calibration plate, the position of the corner points of the calibration plate is extracted, and the position and posture of the calibration plate in the camera coordinate system is calculated; the position and posture of the calibration plate corresponding to each image of the calibration plate in the robotic arm coordinate system is obtained, and the rotation matrix and the translation vector of the camera coordinate system and the robotic arm coordinate system are solved.

[0040] It should be understood that the specific features, operations and details described hereinabove about the method of the present invention may also be similarly applied to the device and system of the present invention, or, vice versa. In addition, each step of the method of the present invention described above may be performed by the corresponding parts or units of the device or system of the present invention.

[0041] It should be understood that each module / unit of the device of the present invention can be implemented in whole or in part by software, hardware, firmware or a combination thereof. Each module / unit can be embedded in the processor of the computer device in the form of hardware or firmware or independent of the processor, or can be stored in the memory of the computer device in the form of software for the processor to call to perform the operation of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0042] In one embodiment, a computer device is provided, which includes a memory and a processor, and the memory stores computer instructions executable by the processor, and the computer instructions instruct the processor to execute each step of the method of the embodiment of the present invention when executed by the processor. The computer device can be a server, a terminal, or any other electronic device with necessary computing and / or processing capabilities in a broad sense. In one embodiment, the computer device may include a processor, a memory, a network interface, a communication interface, etc. connected through a system bus. The processor of the computer device can be used to provide necessary computing, processing and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. may be stored in or on the non-volatile storage medium. The internal memory can provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device can be used to connect and communicate with external devices through a network. The steps of the method of the present invention are executed by the processor.

[0043] The present invention may be implemented as a computer-readable storage medium having a computer program stored thereon, which causes the steps of the method of an embodiment of the present invention to be executed when executed by a processor. In one embodiment, the computer program is distributed on a plurality of computer devices or processors coupled to a network so that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be performed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be performed by one or more computer devices or processors, and one or more other method steps / operations may be performed by one or more other computer devices or processors. One or more computer devices or processors may perform a single method step / operation, or perform two or more method steps / operations.

[0044] It will be appreciated by those skilled in the art that the method steps of the present invention may be performed by instructing related hardware such as a computer device or a processor through a computer program, and the computer program may be stored in a non-temporary computer-readable storage medium, which causes the steps of the present invention to be performed when the computer program is executed. Depending on the circumstances, any reference to memory, storage, database, or other media herein may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0045] The various technical features described above can be combined arbitrarily. Although all possible combinations of these technical features are not described, any combination of these technical features should be considered to be covered by this specification as long as there is no contradiction in such combination.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot dual-arm hand-eye dynamic adaptive calibration method, characterized in that: include: The calibration plate is fixed to the ends of the left and right robotic arms respectively, and the robotic arms are controlled by an automated program to perform N translation and rotation movements within the camera field of view, and the calibration plate image is collected after each movement; Based on the collected calibration plate image, extract the corner point positions of the calibration plate, and calculate the position and posture of the calibration plate in the camera coordinate system; The position and posture of the calibration plate corresponding to each calibration plate image in the robot arm coordinate system are obtained, and the rotation matrix and translation vector of the camera coordinate system and the robot arm coordinate system are solved.

2. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: Also includes: The environmental parameters are monitored in real time through the light sensor and depth camera. If a sudden change in light intensity or interference from an obstruction is detected, the movement path of the robotic arm is dynamically adjusted. Generate an obstacle avoidance path based on the position data of the obstruction obtained by the depth camera; According to the light sensor data, adjust the angle of the calibration plate at the end of the robot arm to optimize the lighting conditions for image acquisition; A new target posture is generated by the robot arm inverse kinematics algorithm, and the movement path is executed.

3. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: Also includes: Calculate the reprojection error in real time, and if the reprojection error exceeds a preset threshold, suspend the calibration and re-move the robotic arm; After each positioning, call OpenCV's projectPoints function to calculate the reprojection error; If the reprojection error exceeds the preset threshold, the robotic arm is triggered to return to the last calibration posture; The robot arm motion trajectory is replanned based on the error distribution to ensure that the calibration plate is in the high-precision imaging area in the subsequent steps.

4. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: Also includes: The inertial measurement unit installed at the end of the robotic arm collects the vibration acceleration and angular velocity data of the robotic arm in real time during its movement. Based on the vibration acceleration and angular velocity data, calculating the instantaneous position offset of the calibration plate at the moment of image acquisition; Compensating the instantaneous posture offset to the posture calculation result of the calibration plate in the camera coordinate system to generate corrected posture data; The corrected pose data is input into the calibrateHandEye function to optimize the rotation matrix and translation vector.

5. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: The step of extracting the corner point positions of the calibration plate comprises: Use OpenCV's findChessboardCorners function to detect the corners of the calibration board; Improve the accuracy of corner coordinates through sub-pixel optimization algorithm (cornerSubPix function); The corner point data of the calibration plates of the left and right robotic arms are distorted and corrected respectively to generate standardized input data.

6. The robot dual-arm hand-eye dynamic adaptive calibration method according to claim 1, characterized in that: Also includes: Use spiral or grid-like paths to cover multiple areas of the camera's field of view; After each movement, verify whether the calibration plate is fully imaged. If no complete corner points are detected, re-execute the current movement posture.

7. A robot dual-arm hand-eye dynamic adaptive calibration device, characterized in that: include: The acquisition module is used to fix the calibration plate at the ends of the left and right robotic arms respectively, control the robotic arms to perform N translation and rotation movements within the camera field of view through an automated program, and acquire the calibration plate image after each movement; A calculation module, used to extract the corner points of the calibration plate based on the acquired calibration plate image, and calculate the position and posture of the calibration plate in the camera coordinate system; The conversion module is used to obtain the position and posture of the calibration plate corresponding to each calibration plate image in the robot arm coordinate system, and solve the rotation matrix and translation vector of the camera coordinate system and the robot arm coordinate system.

8. The robot dual-arm hand-eye dynamic adaptive calibration device according to claim 7, characterized in that: The acquisition module, the calculation module and the conversion module are controlled to execute the robot dual-arm hand-eye dynamic adaptive calibration method described in any one of claims 1-6.

9. An electronic device, characterized in that: include: Communication interface, processor, memory; Wherein, the memory is used to store program instructions, and when the program instructions are executed by the processor that is communicatively connected to the memory through the communication interface, the electronic device implements the robot dual-arm hand-eye dynamic adaptive calibration method described in any one of claims 1 to 6.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a computer, the computer implements the robot dual-arm hand-eye dynamic adaptive calibration method as described in any one of claims 1 to 6.

Images