Hand-eye calibration method and computer device

Abstract

The application discloses a hand-eye calibration method and computer equipment, and belongs to the technical field of computers. The method comprises the following steps: taking a first pose in a base coordinate system of a mechanical arm as a reference to perform n times of pose adjustment on a mechanical arm end; determining a first hand-eye matrix based on n groups of first calibration data, wherein each group of first calibration data comprises a pose of the mechanical arm end in the base coordinate system after the n times of pose adjustment and a pose of a calibration object in a camera coordinate system of the mechanical arm; determining a second pose of the calibration object in the base coordinate system based on the first hand-eye matrix; performing m times of pose adjustment on the mechanical arm end; and determining a second hand-eye matrix based on m groups of second calibration data, wherein each group of second calibration data comprises the second pose, a pose of the mechanical arm end in the base coordinate system after each time of pose adjustment and a pose of the calibration object in the camera coordinate system. The application can improve the accuracy and efficiency of determining the hand-eye matrix by automatically obtaining the first hand-eye matrix and determining the second hand-eye matrix.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a hand-eye calibration method and computer device. Background Technology

[0002] With the continuous development of technology, robots such as six-axis robotic arms have been widely used in manufacturing scenarios such as welding, handling, painting, and assembly. Before leaving the factory, robotic arms need to undergo hand-eye calibration to ensure that they can smoothly and accurately complete operations such as grasping, welding, and assembly along a planned path. In related technologies, technicians manually guide the robotic arm to multiple specific poses and record the data one by one, and then complete the calibration of the robotic arm based on the recorded data. However, this calibration method is not only time-consuming and labor-intensive but also prone to introducing human error, resulting in low calibration efficiency. Summary of the Invention

[0003] This application provides a hand-eye calibration method and a computer device, which can improve calibration efficiency. The technical solution is as follows: Firstly, a hand-eye alignment method is provided, the method comprising: Using the first pose in the base coordinate system of the robotic arm as a reference, the pose of the robotic arm end effector is adjusted n times. After the pose adjustment is completed, the first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the robotic arm end effector in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm. n is an integer greater than or equal to 2. The second pose of the calibration object in the base coordinate system is determined based on the first hand-eye matrix. The robotic arm end effector is subjected to m pose adjustments. After the pose adjustments are completed, a second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data includes the second pose, the pose of the robotic arm end effector in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system. m is an integer greater than or equal to 2.

[0004] In this application, the first pose in the base coordinate system of the robotic arm is used as a reference, and the pose of the robotic arm end effector is adjusted n times. After the pose adjustment is completed, a first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the robotic arm end effector in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm; n is an integer greater than or equal to 2. Based on the first hand-eye matrix, the second pose of the calibration object in the base coordinate system is determined, and the pose of the robotic arm end effector is adjusted m times. After the pose adjustment is completed, a second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data includes the second pose, the pose of the robotic arm end effector in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system; m is an integer greater than or equal to 2. By obtaining the first hand-eye matrix through n pose adjustments in the first stage, the basic transformation relationship between the robotic arm end effector and the camera coordinate system can be quickly established. Secondly, the initial matrix is ​​used to calculate the second pose of the calibration object in the base coordinate system, which is equivalent to introducing a global reference point. This means that the second-stage calibration no longer depends on the unknown pose of the calibration object, but uses this reference point as a known constraint. Based on this, a second m-fold pose adjustment is performed. Since the target pose is known and the entire calibration process can be completed automatically, the residual random and systematic errors in the first-stage calibration can be effectively suppressed, thereby efficiently obtaining a more accurate second-hand eye matrix.

[0005] Optionally, the step of adjusting the position of the robotic arm's end effector n times, based on the first pose in the robotic arm's base coordinate system, includes: Based on the first pose, the robotic arm end effector is adjusted n times according to multiple preset pose offsets.

[0006] Optionally, determining the first hand-eye matrix based on n sets of first calibration data includes: A first hand-eye constraint equation is determined based on every two sets of first calibration data from the n sets of first calibration data. The first hand-eye matrix is ​​determined based on the identified multiple first hand-eye constraint equations.

[0007] Optionally, determining the second pose of the calibration object in the base coordinate system based on the first hand-eye matrix includes: Adjust the end effector of the robotic arm to the first position; Obtain the third pose of the calibration object in the camera coordinate system; Based on the first pose, the third pose, and the first hand-eye matrix, the second pose of the calibration object in the base coordinate system is determined.

[0008] Optionally, determining the second pose of the calibration object in the base coordinate system based on the first pose, the third pose, and the first hand-eye matrix includes: Based on the third pose and the first hand-eye matrix, the fourth pose of the calibration object in the end-effector coordinate system of the robotic arm is determined; Based on the first pose and the fourth pose, the second pose of the calibrated object in the base coordinate system is determined.

[0009] Optionally, the step of performing m pose adjustments on the end effector of the robotic arm includes: Based on the first posture of the robotic arm in the base coordinate system and the preset posture rotation matrix, determine m second postures; The position corresponding to each second pose is determined based on the translation vector in the first hand-eye matrix, the second pose, and each of the m second poses; The end effector of the robotic arm is sequentially adjusted to m fifth poses, each of the m fifth poses including a second pose and a corresponding position.

[0010] Optionally, determining the second hand-eye matrix based on m sets of second calibration data includes: A second hand-eye constraint equation is determined based on each of the m sets of second calibration data. The second hand-eye matrix is ​​determined based on the identified m second hand-eye constraint equations.

[0011] Optionally, the calibration object includes a calibration plate and two calibration blocks mounted on the calibration plate, wherein the two calibration blocks are at different distances from the calibration plate.

[0012] Optionally, the method further includes: The Z-axis of the object coordinate system of the calibration object is determined based on the plane of the calibration plate; The origin of the object coordinate system is determined based on the center of the calibration block that is furthest from the calibration plate among the two calibration blocks. The X-axis of the object's coordinate system is determined based on the projection of the line connecting the centers of the two calibration blocks onto the plane of the calibration plate. The Y-axis of the object coordinate system is determined based on the Z-axis and X-axis of the object coordinate system.

[0013] Secondly, a hand-eye calibration device is provided, the device comprising: The first calibration module is used to perform n pose adjustments on the end effector of the robotic arm, based on the first pose in the base coordinate system of the robotic arm. After the pose adjustment is completed, a first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm. n is an integer greater than or equal to 2. The first determining module is used to determine the second pose of the calibration object in the base coordinate system based on the first hand-eye matrix. The second calibration module is used to perform m pose adjustments on the end effector of the robotic arm; after the pose adjustment is completed, a second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data in the m sets includes the second pose, the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system; where m is an integer greater than or equal to 2.

[0014] Thirdly, a computer device is provided, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the hand-eye calibration method described in the first aspect.

[0015] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the hand-eye calibration method described in the first aspect.

[0016] Fifthly, a computer program product is provided that, when the computer program product is run on a computer device, causes the computer device to execute the hand-eye calibration method described in the first aspect.

[0017] It is understood that the beneficial effects of the second, third, fourth, and fifth aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the structure of a calibration system provided in an embodiment of this application; Figure 2 This is a schematic diagram of a calibration object provided in an embodiment of this application; Figure 3 This is a flowchart of a hand-eye calibration method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a hand-eye calibration device provided in an embodiment of this application; Figure 5This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and technologies are set forth for illustrative purposes and not for limiting purposes, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details.

[0020] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0021] It should be understood that "one or more" as used in this application refers to one, two, or more, and "multiple" as used in this application refers to two or more. In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0022] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0023] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0024] The application scenarios involved in the embodiments of this application are described below.

[0025] With the continuous development of technology, robots such as six-axis robotic arms have been widely used in industrial manufacturing scenarios such as welding, handling, spraying, and assembly. Before leaving the factory, six-axis robotic arms need to undergo hand-eye calibration to establish the transformation relationship between the camera coordinate system and the coordinate system of the robotic arm's end effector, providing core technical support for precision grasping, positioning, and assembly tasks in industrial automation.

[0026] In related technologies, technicians manually guide the robotic arm to multiple specific poses and record the data one by one, then calibrate the robotic arm based on the recorded data. However, this calibration method is not only time-consuming and labor-intensive but also prone to introducing human error, resulting in low calibration efficiency and poor repeatability.

[0027] Therefore, this application provides a hand-eye calibration method applied to a robotic arm. In this method, the first pose of the robotic arm in its base coordinate system is used as a reference, and the end effector of the robotic arm is adjusted n times. After the pose adjustment is completed, a first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm; n is an integer greater than or equal to 2. Based on the first hand-eye matrix, the second pose of the calibration object in the base coordinate system is determined, and the end effector of the robotic arm is adjusted m times. After the pose adjustment is completed, a second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data includes the second pose, the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system; m is an integer greater than or equal to 2. The first-stage n-fold pose adjustment yields the first-hand-eye matrix, enabling the rapid establishment of the fundamental transformation relationship between the robotic arm's end effector and the camera coordinate system. Next, this initial matrix is ​​used to calculate the second pose of the calibration object in the base coordinate system, effectively introducing a global reference point. This allows the second-stage calibration to no longer rely on the unknown pose of the calibration object, but rather use this reference point as a known constraint. Based on this, a second m-fold pose adjustment is performed. Since the target pose is known and the entire calibration process can be completed automatically, residual random and systematic errors from the first-stage calibration can be effectively suppressed, resulting in a more accurate second-hand-eye matrix.

[0028] The robotic arm in the embodiments of this application will be described below.

[0029] Figure 1 This is a schematic diagram of the structure of a calibration system provided in an embodiment of this application. See also... Figure 1 The calibration system may include a robotic arm and a control module (not shown in the figure), wherein the robotic arm may include a base, a robotic arm end effector, and a camera.

[0030] For example, Figure 1The following description uses a six-axis robotic arm as an example. In practical applications, the robotic arm can be a five-axis, six-axis, or seven-axis robotic arm, etc., and this application does not limit this.

[0031] The robotic arm can communicate with the control module via wired or wireless connection.

[0032] The base is bolted to the worktable and rigidly connected to the joints of the robotic arm, providing stable support and power distribution for the entire machine. The base has a base coordinate system, which is the world coordinate system on the base of the robotic arm and serves as the reference for all movements. For example, this base coordinate system can also be called the base Cartesian base coordinate system.

[0033] The end effector of the robotic arm is mechanically coupled to a camera via a flange and communicates with the control module via an internal bus to achieve precise trajectory tracking and tool operation. The end effector has an end-effector coordinate system (tool), which is fixed on the end effector flange and moves with the robotic arm.

[0034] The camera is mounted on the end effector of a robotic arm using a specialized clamp to acquire visual information about target objects for identification and localization. The camera has a camera coordinate system, which is fixed to the camera and describes the position of the observed object. For example, the camera can transmit image data to the control module via a universal serial bus (USB) or a network cable.

[0035] The control module is used to execute the calibration algorithm and control the robotic arm to perform calibration. For example, the control module can be integrated into the robotic arm, forming a separate device, or it can be separate from the robotic arm, meaning it can be a module within another device or a separate module, etc. This application does not limit this. For example, the control module can be connected to the base and each joint motor via a servo driver harness to perform kinematic calculations, process visual feedback, and output control commands.

[0036] In six-axis robotic arm guidance, the purpose of hand-eye calibration is to solve the transformation relationship between the three-dimensional (3D) camera coordinate system and the robotic arm end effector coordinate system (i.e., the hand-eye matrix), thereby obtaining the coordinates or pose of the object (Obj) in the base coordinate system. .like Figure 1 As shown, Figure 1 The following transformation relationships exist between the coordinate systems in a six-axis robotic arm: ; in, The pose of the object in the base coordinate system; The transformation relationship from the end-effector coordinate system to the base coordinate system can be read from the robotic arm; This represents the transformation relationship from the camera coordinate system to the end coordinate system. This represents the pose of the object in the camera coordinate system.

[0037] As can be seen from the above transformation relationship, establishing an accurate transformation relationship between the camera coordinate system and the end coordinate system is the core technical support for tasks such as precision gripping, positioning and assembly in industrial automation.

[0038] The following explains the approach to establishing the hand-eye matrix in the embodiments of this application.

[0039] The establishment of the hand-eye matrix in this embodiment may include the following two-stage process: Phase 1: Coarse calibration phase. After the technician manually teaches an initial photographic pose, the robotic arm performs multi-pose movements with a small offset. It pauses at each pose and triggers the photographing of a fixed calibration object. At the same time, it records the pose of the robotic arm and the center coordinates of the calibration object identified by the camera, obtaining n sets of first calibration data. Any set of the first calibration data can include the pose of the robotic arm end in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm. Based on the n sets of first calibration data, the hand-eye transformation matrix E (i.e., the first hand-eye matrix below) is determined by the hand-eye calibration algorithm.

[0040] For example, such as Figure 2 As shown, the calibration object may include a calibration plate and two calibration blocks mounted on the calibration plate, the two calibration blocks being at different distances from the calibration plate.

[0041] The difference in distance between the two calibration blocks and the calibration plate refers to the difference in the distance between the two calibration blocks and the surface of the calibration plate, and the difference in the position of the two calibration blocks within the calibration plate. For example, the two calibration blocks can be circular calibration blocks, square calibration blocks, etc.

[0042] The calibration object design uses a plane and two calibration blocks of different heights to give the calibration object obvious differences, thereby providing complete pose information, reducing the complexity of visual processing, and accurately determining the six-degree-of-freedom pose of the calibration object's coordinate system. This improves the practicality and reliability of the calibration system.

[0043] The second stage: fine calibration stage. The robotic arm returns to its initial position, and the precise coordinates of the calibration object in the base coordinate system are calculated using the hand-eye transformation matrix E (i.e., the second pose described below). Then, based on the significantly different new pose and the translation components of the hand-eye matrix, a new position is determined to realign the tool's center point with the sphere's center. The robotic arm's end effector is then driven to this pose (new pose and new position) to capture an image, determining m sets of second calibration data. Any set of these m sets of second calibration data can include the second pose, the pose of the robotic arm's end effector in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system. Based on these m sets of second calibration data, a more precise calibration matrix (i.e., the second hand-eye matrix described below) is determined.

[0044] For example, the core idea of ​​this hand-eye calibration algorithm is to determine the coordinate system through the transformation relationship between different coordinate systems. The detailed steps of this hand-eye calibration algorithm may include the following steps (a) to (e): Step (a): For any two sets of first calibration data in the n sets of second calibration data, establish the core equation based on the two sets of calibration data.

[0045] ; .

[0046] Step (b): Eliminate the pose of the calibration object.

[0047] Since the position of the calibration object in space remains unchanged, and its pose in space is unknown, its effect on both sides of the equation is the same, so it can be multiplied by... The inverse matrix eliminates the pose.

[0048] Multiply both sides of the equation by the right Then there is .

[0049] Step (c): Alignment base transformation.

[0050] By transferring the influence of the base coordinate system to one side of the equation, we can separate the transformation terms that only include the camera and the end effector.

[0051] Multiply both sides of the equation by the left side Then there is .

[0052] Step (d): Variable substitution.

[0053] make Then there is ; Where A represents the relative motion of the robotic arm under the base; B represents the relative motion of the calibration object observed by the camera; and X represents the first-hand eye matrix.

[0054] Step (e): Solve for X using singular value decomposition (SVD).

[0055] The hand-eye calibration method provided in the embodiments of this application will be explained in detail below.

[0056] Figure 3 This is a flowchart of a hand-eye calibration method provided in an embodiment of this application. For example, this method can be applied to the above... Figure 1 The calibration system in the embodiments. For example, when the control module is integrated into the robotic arm, the method can be applied to the robotic arm; when the control module is not integrated into the robotic arm, the method can be applied to the control module. See also Figure 3 The method may include the following steps: Step 301: The computer device uses the first pose in the base coordinate system of the robotic arm as a reference to perform n pose adjustments on the end effector of the robotic arm; after the pose adjustment is completed, the first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm; n is an integer greater than or equal to 2.

[0057] For example, the first pose (T) base_flange_0 The first pose is the position of the robotic arm when the calibration object can be clearly observed. The first pose can include the position P0 and orientation R0 of the robotic arm. For example, a technician can manually operate the robotic arm to move it to a position where the calibration object on the worktable can be clearly observed (also known as the initial teaching photograph position), record the position of the robotic arm, and obtain the first pose.

[0058] For example, the robotic arm end effector can be adjusted n times based on a first pose and multiple preset pose offsets. These multiple preset pose offsets can be pre-set and can be set to a relatively small value.

[0059] For example, the setting of these multiple preset pose offsets needs to ensure that the calibration block is always within the camera's field of view. This is because at the start of calibration, only the initial pose is known; the hand-eye matrix is ​​unknown, and the camera's mounting position and angle relative to the robotic arm's end effector are also unknown. If the robotic arm is controlled to make large-scale, significant pose changes at the beginning, the camera may turn away from the calibration object, causing the calibration object to deviate from the camera's field of view and resulting in data acquisition failure. Therefore, by making small, fine adjustments to the pose of the robotic arm's end effector (e.g., rotating it only a few degrees or moving it a few millimeters), it can be ensured that the calibration object always remains in the central area of ​​the camera's image, thus guaranteeing the continuity and stability of data acquisition.

[0060] For example, using the first pose as a reference, the robotic arm can be controlled to move according to a preset fine-tuning angle sequence (ΔR_x, ΔR_y, ΔR_z) and position sequence (ΔX, ΔY, ΔZ). Each time the robotic arm reaches a new pose, it pauses to take a picture. Then, an image processing algorithm is used to identify the pose of the calibration object in the camera coordinate system, and simultaneously records the pose of the current robotic arm end effector in the base coordinate system, thus obtaining a set of first calibration data.

[0061] The first hand-eye matrix is ​​a relatively accurate hand-eye transformation matrix used to indicate the transformation relationship between the camera coordinate system and the end-point coordinate system.

[0062] For example, the calibration device includes a calibration plate and two calibration blocks mounted on the calibration plate at different distances from the calibration plate. Further details regarding this calibration device can be found in [reference needed]. Figure 1 The detailed explanation of the calibration objects in the embodiments will not be repeated here. Of course, it is not limited to this; other forms or types of calibration objects can also be used for calibration, as long as they can meet the constraints of the calibration blocks in the above-mentioned calibration objects.

[0063] The position of the calibration object in the camera coordinate system is determined by the origin of the object coordinate system. That is, the position of the calibration object in the camera coordinate system describes the translation of the origin of the object coordinate system within the camera coordinate system. The orientation of the calibration object in the camera coordinate system is determined by the rotational relationship between the three axes (X / Y / Z) of the object coordinate system and the corresponding axes of the camera coordinate system. That is, the orientation of the calibration object in the camera coordinate system describes the rotation of the object coordinate system relative to the camera coordinate system.

[0064] Since the transformation relationship between the end-effector coordinate system and the camera coordinate system is unknown, multiple adjustments to the end-effector pose and obtaining multiple sets of "end-effector pose + camera observation" data can provide sufficient reference data (i.e., constraint equations) for solving the first hand-eye matrix. Furthermore, since the pose of the calibration object remains unchanged in the base coordinate system during the n adjustments, the pose of the calibration object determined by each of the n sets of first calibration data is the same in the base coordinate system. Therefore, the solution equations can be constructed based on these n sets of first calibration data to determine the first hand-eye matrix.

[0065] By automating the control of multi-posture motion and data acquisition, the efficiency of data acquisition and calibration can be greatly improved; furthermore, by acquiring n sets of first calibration data, sufficient reference data can be provided for determining the first hand-eye matrix.

[0066] In some implementations, the operation of the computer device to determine the first hand-eye matrix based on n sets of first calibration data can be as follows: determining a first hand-eye constraint equation based on every two sets of first calibration data in the n sets of first calibration data; and determining the first hand-eye matrix based on the determined multiple first hand-eye constraint equations.

[0067] For example, the process by which the computer device determines a first hand-eye constraint equation based on every two sets of first calibration data in the n sets of first calibration data can refer to steps (a) to (c) in the hand-eye calibration algorithm in the above embodiment, and a first hand-eye constraint equation can be obtained.

[0068] For example, based on multiple first-hand-eye constraint equations, the first-hand-eye matrix can be determined using SVD. Specifically, all the collected data pairs are substituted into the hand-eye calibration equation XB=AX to construct an overdetermined system of equations. Then, SVD is used to numerically solve this system of equations, and the optimal rotation matrix R is determined in the least squares sense. e The translation vector t is used to minimize the overall residual of all equations, thereby outputting a preliminary hand-eye transformation matrix E (i.e., the first hand-eye matrix).

[0069] For example, the first-hand-eye matrix can define the transformation relationship from the camera coordinate system to the robot arm end effector coordinate system, that is: C flange =R e ·C cam +t. Where R e t is the hand-eye rotation matrix from the camera coordinate system to the end-eye coordinate system, and t is the hand-eye translation vector (i.e., the coordinates of the origin of the camera coordinate system in the end-eye coordinate system).

[0070] By constructing multiple constraint equations from pairwise data and solving them comprehensively, the noise resistance and solution accuracy of the coarse calibration algorithm can be improved, the impact of single measurement errors can be reduced, and the accuracy of determining the first-hand eye matrix can be improved.

[0071] In some implementations, the operation of the computer device to determine the pose of the calibrated object in the camera coordinate system of the robotic arm may include the following steps (1) to (4): Step (1): The computer device determines the Z-axis of the object coordinate system of the calibration object based on the plane of the calibration plate.

[0072] Since the direction of the plane normal vector is relatively stable, the normal vector of the calibration plate plane can be used as the Z-axis reference.

[0073] For example, using a set of planar points in a 3D point cloud, the plane equation of the calibration block base plate is fitted by the least squares or RANSAC algorithm, and its normal vector is taken as the Z-axis direction.

[0074] Specifically, the 3D point set of the calibration block region can be obtained through depth maps or point cloud data. First, the point cloud belonging to the bottom plane is segmented using pass-through filtering or region growing algorithms. A plane fitting algorithm is then applied to these points to construct a covariance matrix, and the eigenvector corresponding to the smallest eigenvalue is calculated as the plane normal vector. The plane equation is fitted, and its normal vector, after normalization, becomes the unit vector along the Z-axis. This vector points upwards on the calibration object, thus determining the Z-axis pose.

[0075] Step (2): The computer device determines the origin of the object's coordinate system based on the center of the calibration block that is furthest from the calibration plate among the two calibration blocks.

[0076] For example, the surface points of two calibration blocks can be extracted from the point cloud, the center coordinates of the calibration blocks can be obtained using the sphere fitting algorithm, and then the vertical distance from the center of each calibration block to the plane obtained in step (1) can be calculated.

[0077] Specifically, the point clouds of the two calibration blocks can be separated from the background (e.g., by the height difference between the calibration blocks and the base plate). For the point cloud of each calibration block, the center coordinates and radius of each calibration block are determined using the least squares sphere fitting algorithm, resulting in two center coordinates. Then, the perpendicular distances from the two center coordinates to the fitted plane are calculated using the point-to-plane distance formula.

[0078] Step (3): The computer device determines the X-axis of the object's coordinate system based on the projection of the line connecting the centers of the two calibration blocks onto the plane of the calibration plate.

[0079] Specifically, the distances from the centers of the two calibration blocks to the plane can be compared. The calibration block with the larger distance is designated as calibration block 1 (high sphere), and its center is used as the origin of the subsequent attitude coordinate system. The vector from the center of calibration block 1 to the center of calibration block 2 is calculated, and this vector is projected onto the bottom plane obtained in step 1. The projection direction is normalized and used as the X-axis direction.

[0080] Step (4): The computer device determines the Y-axis of the object's coordinate system based on the Z-axis and X-axis of the object's coordinate system.

[0081] For example, the Y-axis unit vector can be calculated by performing a cross product of the X-axis unit vector and the Z-axis unit vector using the right-hand rule, thus constructing an orthogonal coordinate system. Specifically, to ensure the orthogonality of the coordinate system and the right-hand rule, the Z-axis unit vector and the X-axis unit vector are multiplied to obtain the Y-axis unit vector. Since the Z-axis and Y-axis unit vectors are normalized and perpendicular to each other, the cross product result is automatically a unit vector and orthogonal to both. This yields a complete right-handed coordinate system (also called the object coordinate system), with its origin at the center of the high sphere, the X-axis pointing towards the low sphere, the Z-axis perpendicular to the base plate and upwards, and the Y-axis determined by the cross product. This coordinate system represents the pose of the calibration block in the camera coordinate system, including position (origin coordinates) and rotation (the directions of the three axes, i.e., pose).

[0082] By using plane fitting and center fitting, the object coordinates of the calibration object can be automatically constructed, providing a unified and stable reference benchmark for calibration.

[0083] Step 302: The computer device determines the second pose of the calibrated object in the base coordinate system based on the first hand-eye matrix.

[0084] The second pose is the pose of the calibration object in the base coordinate system determined by the first hand-eye matrix.

[0085] Since the first hand-eye matrix and the transformation matrix for converting the end-effector coordinate system to the base coordinate system are known, after acquiring the pose of the calibration object in the camera coordinate system, it can be transformed to the base coordinate system to obtain the second pose of the calibration object in that base coordinate system. In this way, the base coordinates of the calibration object can be automatically obtained without external measuring equipment, providing a known reference for subsequently determining the second hand-eye matrix.

[0086] In some implementations, step 302 may involve: the computer device adjusting the end effector of the robotic arm to a first pose; acquiring the third pose of the calibrated object in the camera coordinate system; and determining the second pose of the calibrated object in the base coordinate system based on the first pose, the third pose, and the first hand-eye matrix.

[0087] By adjusting the robotic arm's end effector to the first pose, existing coarse calibration results and known end effector poses can be utilized for unified calculations. Furthermore, real-time camera recognition enables automated data acquisition. Using coordinate system transformation, the third pose can be converted to the base coordinate system, obtaining the fixed position of the calibration object.

[0088] By using the first-hand eye matrix and the initial pose (i.e., the first pose), the system can automatically and accurately determine the fixed base coordinates of the calibration object without external measurement. This improves calibration efficiency.

[0089] In some implementations, the operation of the computer device to determine the second pose of the calibrator in the base coordinate system based on the first pose, the third pose, and the first hand-eye matrix can be: determining the fourth pose of the calibrator in the end-effector coordinate system of the robotic arm based on the third pose and the first hand-eye matrix; and determining the second pose of the calibrator in the base coordinate system based on the first pose and the fourth pose.

[0090] For example, the fourth pose can be converted into the second pose of the calibrated object in the base coordinate system using the principle of forward kinematics.

[0091] The hand-eye matrix describes the transformation from the camera coordinate system to the end-effector coordinate system, directly converting camera coordinates to end-effector coordinates. The first pose describes the transformation from the end-effector coordinate system to the base coordinate system. Multiplying this by the coordinates in the end-effector coordinate system yields the coordinates of the calibration object in the base coordinate system. This step-by-step mapping through two coordinate transformations ensures the accuracy of the calculation results.

[0092] Step 303: The computer device performs m pose adjustments on the end effector of the robotic arm; after the pose adjustment is completed, the second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data in the m sets includes the second pose, the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system; m is an integer greater than or equal to 2.

[0093] The second hand-eye matrix is ​​the determined high-precision hand-eye matrix.

[0094] The second calibration data was detected after adjusting the pose of the robotic arm's end effector.

[0095] Since the coarse calibration results have residual errors, a more accurate optimization equation is constructed through fine calibration (changing the posture and using the known coordinates of the calibration object) to further improve the accuracy of the hand-eye matrix.

[0096] In each of the m sets of second calibration data, the second pose is the same, and the pose of the robotic arm end in the base coordinate system and the pose of the calibration object in the camera coordinate system are both detected real data. Therefore, based on the m sets of second calibration data and the transformation relationship between the coordinate systems, a high-precision hand-eye matrix can be determined.

[0097] In some implementations, the computer device may perform m pose adjustments on the end effector of the robotic arm as follows: based on the first pose of the robotic arm in the base coordinate system and a preset pose rotation matrix, determine m second poses; based on the translation vector in the first hand-eye matrix, the second poses, and each of the m second poses, determine the position corresponding to each second pose; and sequentially adjust the end effector of the robotic arm to m fifth poses, where each of the m fifth poses includes a second pose and a corresponding position.

[0098] The preset attitude rotation matrix can be pre-set. For example, the preset attitude rotation matrix can be set to be relatively large. Specifically, when the robotic arm rotates by a large angle, the end effector pose changes drastically. This transformation can significantly amplify the impact of rotation errors, which is equivalent to providing strong constraints to the solver. This causes the solution, which was originally volatile under small perturbations, to be constrained to a fixed position by data points with large differences. By introducing data with large attitude changes, not only is the ill-conditioned problem of the equation system caused by small-angle data mathematically overcome, significantly improving the stability and noise resistance of the solution, but the coupling error between rotation and translation parameters is also effectively amplified. This allows the nonlinear optimization algorithm to accurately identify and correct the parameters of the hand-eye matrix, achieving high-precision decoupled solutions. Finally, the coverage of calibration data in the workspace is expanded, ensuring the generalization ability and practical application accuracy of the calibration matrix in the entire motion domain of the robotic arm.

[0099] The m second postures are the postures that the end effector of the robotic arm (such as a flange ring) needs to achieve.

[0100] The fifth pose includes the second pose and the position corresponding to the second pose.

[0101] At each of the m second poses, the calibration object is highly likely to be within the camera's field of view in the robotic arm, and highly likely to be located in the central area of ​​that field of view. That is, at this position, the camera can aim at the calibration object and take a picture.

[0102] For example, after the computer device determines the m second poses, it can first control the robotic arm to rotate according to the second poses in sequence. Each time it rotates to a second pose, it determines the position corresponding to the second pose based on the translation vector in the first hand-eye matrix, the second pose, and the second pose. Then, it controls the robotic arm to move to the position corresponding to the second pose. This application embodiment does not limit this.

[0103] By determining the position corresponding to each of the m second poses and sequentially adjusting the end effector of the robotic arm to the m fifth poses, the camera is always aligned with the new pose of the calibration object. This allows for a wide range of pose changes without losing the target, thus ensuring the validity and richness of the calibration data. Furthermore, the calculation and adjustment processes are fully automated, requiring no manual intervention, which improves calibration efficiency and guarantees calibration accuracy.

[0104] The principle behind the m-fold pose adjustment of the robotic arm's end effector is explained below.

[0105] When it is necessary to adjust the robot arm's current first pose in the base coordinate system to a preset pose rotation matrix, in order to keep the pose of the calibration object in the camera coordinate system unchanged, the robot arm's end effector needs to move to a new position. This is to ensure that the camera coordinate system can continue to be aligned with the calibrator.

[0106] Then, based on the translation vector in the first-hand eye matrix... and preset attitude rotation matrix The formula derivation is as follows: In the base coordinate system, the position of the origin of the terminal coordinate system is denoted as . The posture is The vector of any point in space in the final coordinate system. The formula for transforming to the base coordinate system is: ; in, It is the vector of that point in the base coordinate system.

[0107] The virtual tool point is fixed to the origin of the camera coordinate system; therefore, the coordinates of this tool point in the end-effector coordinate system are the hand-eye translation vector. (Coordinates of the camera coordinate system origin in the end coordinate system).

[0108] Therefore, the coordinates of the tool point in the base coordinate system are: ; To align the tool point with the fixed calibration block ,make: ; Solve for the end position That is, to obtain a new position : ; in, The fixed coordinates of the calibration object in the base coordinate system; Preset the attitude rotation matrix (from the end coordinate system to the base coordinate system); It is the hand-eye translation vector in the first hand-eye matrix (the coordinates of the origin of the camera coordinate system in the end coordinate system).

[0109] In this way, the position that the robotic arm end effector needs to move to can be obtained while keeping the pose of the calibrated object in the camera coordinate system unchanged.

[0110] In some implementations, the operation of the computer device to determine the second hand-eye matrix based on m sets of second calibration data can be as follows: determining a second hand-eye constraint equation based on each set of second calibration data in the m sets of second calibration data; and determining the second hand-eye matrix based on the determined m second hand-eye constraint equations.

[0111] For example, the second hand-eye matrix can be determined through nonlinear optimization or hand-eye calibration algorithms.

[0112] Since the base coordinates of the calibration object, the pose of the robotic arm's end effector in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system are known, least-squares optimization constraints can be directly established for each data point without pairwise combinations. Furthermore, the m sets of second calibration data are detected after a preset pose rotation matrix with large rotation angles, which can effectively amplify the coupling error between rotation and translation parameters. Therefore, by using the calibration data from each fine calibration to construct constraints and perform overall optimization, the advantages of multiple data points can be fully utilized, significantly improving the final accuracy of the second hand-eye matrix.

[0113] For example, the computer device, based on m second-hand eye constraint equations, can determine the second-hand eye matrix using the following formula.

[0114] ; Where m is the number of data sets collected; For the first The posture and position of the robotic arm's end effector during alignment; For the first The coordinates of the calibrated object in the camera coordinate system during the first photograph; The parameters of the hand-eye matrix to be optimized are shown.

[0115] For ease of understanding, the complete process of this hand-eye calibration method is illustrated below with reference to steps A through H. This hand-eye calibration method may include a coarse calibration stage and a fine calibration stage.

[0116] coarse calibration stage Step A: Teaching the initial photo-taking position.

[0117] Specifically, the technicians manually moved the robotic arm to a position where the calibration block could be clearly observed and recorded the first pose.

[0118] Step B: The computer device performs multi-pose coarse calibration.

[0119] Using the initial pose as a reference, the robotic arm is automatically controlled to move according to a preset sequence of small-amplitude angles and positions. Upon reaching a new pose, the robotic arm pauses, the camera captures an image of a calibration object and identifies its coordinates in the camera coordinate system, and simultaneously records the current pose of the robotic arm's end effector. Multiple sets of data are collected cyclically.

[0120] Step C: The computer device calculates the first-hand eye matrix.

[0121] Using multiple sets of collected data pairs, a hand-eye calibration equation is constructed, and the first hand-eye matrix (including rotation matrix and translation vector) is determined by linear solvers such as SVD decomposition.

[0122] fine calibration stage Step D: The computer device returns to the first pose and calculates the second pose of the calibration object in the base coordinate system.

[0123] The robotic arm is automatically returned to its first position, and the camera takes another picture to obtain the coordinates of the current calibration block in the camera coordinate system. The first hand-eye matrix is ​​used to transform it to the end-effector coordinate system, and then forward kinematics is used to transform it to the base coordinate system to obtain the fixed coordinates of the calibration block in the base coordinate system.

[0124] Step E: The computer device changes the posture of the robotic arm's end effector.

[0125] Set a rotation matrix that is significantly different from the first pose in the first pose. The robotic arm only rotates while keeping its position unchanged, and adjusts the pose of the end effector of the robotic arm to the pose corresponding to the rotation matrix.

[0126] Step F: The computer device calculates the position corresponding to the new robotic arm's end effector posture.

[0127] Step G: The computer device performs the motion and takes a picture.

[0128] The robotic arm moves to the calculated new pose, at which point the calibration block should be approximately located at the center of the image. The camera takes a picture and records the coordinates of the current calibration block in the camera coordinate system, while also recording the corresponding robot end-effector pose.

[0129] Step H: The computer device repeatedly collects and solves the data precisely.

[0130] In this embodiment, the computer device uses the first pose in the base coordinate system of the robotic arm as a reference to perform n pose adjustments on the end effector of the robotic arm. After the pose adjustments are completed, a first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm; n is an integer greater than or equal to 2. Based on the first hand-eye matrix, the second pose of the calibration object in the base coordinate system is determined, and m pose adjustments are performed on the end effector of the robotic arm. After the pose adjustments are completed, a second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data includes the second pose, the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system; m is an integer greater than or equal to 2. By obtaining the first hand-eye matrix through n pose adjustments in the first stage, the basic transformation relationship between the end effector of the robotic arm and the camera coordinate system can be quickly established. Secondly, the initial matrix is ​​used to calculate the second pose of the calibration object in the base coordinate system, which is equivalent to introducing a global reference point. This means that the second-stage calibration no longer depends on the unknown pose of the calibration object, but uses this reference point as a known constraint. Based on this, a second m-fold pose adjustment is performed. Since the target pose is known and the entire calibration process can be completed automatically, the residual random and systematic errors in the first-stage calibration can be effectively suppressed, thereby efficiently obtaining a more accurate second-hand eye matrix.

[0131] Figure 4 This is a schematic diagram of a hand-eye calibration device provided in an embodiment of this application. The device can be implemented as part or all of a computer device by software, hardware, or a combination of both. This computer device can be described below. Figure 5 The computer equipment shown. See also Figure 4 The device includes: a first calibration module 401, a first determination module 402, and a second calibration module 403.

[0132] The first calibration module 401 is used to perform n pose adjustments on the end effector of the robotic arm based on the first pose in the base coordinate system of the robotic arm; after the pose adjustment is completed, the first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm; n is an integer greater than or equal to 2. The first determining module 402 is used to determine the second pose of the calibration object in the base coordinate system based on the first hand-eye matrix. The second calibration module 403 is used to perform m pose adjustments on the end effector of the robotic arm. After the pose adjustment is completed, the second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data in the m sets includes the second pose, the pose of the end effector of the robotic arm in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system. m is an integer greater than or equal to 2.

[0133] Optionally, the first calibration module 401 is used for: Based on the first pose, the position of the robotic arm end effector is adjusted n times according to multiple preset pose offsets.

[0134] Optionally, the first calibration module 401 is used for: A first hand-eye constraint equation is determined for every two sets of first calibration data from the n sets of first calibration data. The first-hand eye matrix is ​​determined based on the identified multiple first-hand eye constraint equations.

[0135] Optionally, the first determining module 402 is used for: Adjust the end effector of the robotic arm to the first position; Obtain the third pose of the calibration object in the camera coordinate system; Based on the first pose, the third pose, and the first hand-eye matrix, the second pose of the calibration object in the base coordinate system is determined.

[0136] Optionally, the first determining module 402 is used for: Based on the third pose and the first hand-eye matrix, the fourth pose of the calibrated object in the end-effector coordinate system of the robotic arm is determined; Based on the first and fourth poses, determine the second pose of the calibration object in the base coordinate system.

[0137] Optionally, the second calibration module 403 is used for: Based on the robot arm’s current first posture in the base coordinate system and the preset posture rotation matrix, determine m second postures; The position corresponding to each second pose is determined based on the translation vector in the first hand-eye matrix, the second pose, and each of the m second poses; The end effector of the robotic arm is sequentially adjusted to m fifth poses, each of the m fifth poses including a second pose and a corresponding position.

[0138] Optionally, the second calibration module 403 is used for: A second hand-eye constraint equation is determined based on each of the m sets of second calibration data. The second hand-eye matrix is ​​determined based on the identified m second hand-eye constraint equations.

[0139] Optionally, the calibration object includes a calibration plate and two calibration blocks mounted on the calibration plate, the two calibration blocks being at different distances from the calibration plate.

[0140] Optionally, the device further includes: The second determining module is used to determine the Z-axis of the object coordinate system of the calibration object based on the plane of the calibration plate; The third determining module is used to determine the origin of the object's coordinate system based on the center of the calibration block that is furthest from the calibration plate among the two calibration blocks; The fourth determining module is used to determine the X-axis of the object's coordinate system based on the projection of the line connecting the centers of the two calibration blocks onto the plane of the calibration plate; The fifth determining module is used to determine the Y-axis of the object's coordinate system based on the Z-axis and X-axis of the object's coordinate system.

[0141] In this embodiment, the robot arm's end effector undergoes n pose adjustments based on its first pose in the base coordinate system. After the pose adjustments, a first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the robot arm's end effector in the base coordinate system after each pose adjustment and the pose of the calibration object in the robot arm's camera coordinate system; n is an integer greater than or equal to 2. The second pose of the calibration object in the base coordinate system is determined based on the first hand-eye matrix, and the robot arm's end effector undergoes m pose adjustments. After the pose adjustments, a second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data includes the second pose, the pose of the robot arm's end effector in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system; m is an integer greater than or equal to 2. By obtaining the first hand-eye matrix through n pose adjustments in the first stage, the basic transformation relationship between the robot arm's end effector and the camera coordinate system can be quickly established. Secondly, the initial matrix is ​​used to calculate the second pose of the calibration object in the base coordinate system, which is equivalent to introducing a global reference point. This means that the second-stage calibration no longer depends on the unknown pose of the calibration object, but uses this reference point as a known constraint. Based on this, a second m-fold pose adjustment is performed. Since the target pose is known and the entire calibration process can be completed automatically, the residual random and systematic errors in the first-stage calibration can be effectively suppressed, thereby efficiently obtaining a more accurate second-hand eye matrix.

[0142] It should be noted that the hand-eye calibration device provided in the above embodiments is only illustrated by the division of the above functional modules when performing hand-eye calibration. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0143] The functional modules in the above embodiments can be integrated into one processing unit, or each functional module can exist as a separate physical processing unit, or two or more functional modules can be integrated into one processing unit. The processing unit can be implemented in hardware or software. Furthermore, the specific names of the functional modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0144] The hand-eye calibration device and the hand-eye calibration method provided in the above embodiments belong to the same concept. The specific working process and technical effects of the functional modules in the above embodiments can be found in the method embodiments section, and will not be repeated here.

[0145] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 5 As shown, the computer device 5 includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, it implements the steps in the hand-eye calibration method in the above embodiments.

[0146] Computer device 5 may be a robotic arm, a general-purpose computer device, or a special-purpose computer device. Those skilled in the art will understand that... Figure 5 The computer device 5 is merely an example and does not constitute a limitation on the computer device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0147] Processor 50 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0148] In some embodiments, memory 51 may be an internal storage unit of the computer device 5, such as a hard disk or memory of the computer device 5. In other embodiments, memory 51 may be an external storage device of the computer device 5, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device 5. Furthermore, memory 51 may include both internal storage units and external storage devices of the computer device 5. Memory 51 is used to store the operating system, applications, boot loader, data, and other programs. Memory 51 may also be used to temporarily store data that has been output or will be output.

[0149] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0150] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0151] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the various method embodiments described above.

[0152] This application provides a computer program product that, when run on a computer, causes the computer to perform the steps described in the various method embodiments above.

[0153] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above method embodiments of this application can be implemented by a computer program. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a computer device, recording media, computer memory, read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage devices. The computer-readable storage medium mentioned in this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0154] It should be understood that all or part of the steps of the above embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented in whole or in part as a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-described computer-readable storage medium.

[0155] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0156] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this application according to actual needs.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0158] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A hand-eye calibration method, characterized in that, The method includes: Using the first pose in the base coordinate system of the robotic arm as a reference, the pose of the robotic arm end effector is adjusted n times. After the pose adjustment is completed, the first hand-eye matrix is ​​determined based on n sets of first calibration data. Each set of first calibration data includes the pose of the robotic arm end effector in the base coordinate system after each pose adjustment and the pose of the calibration object in the camera coordinate system of the robotic arm. n is an integer greater than or equal to 2. The second pose of the calibration object in the base coordinate system is determined based on the first hand-eye matrix. The robotic arm end effector is subjected to m pose adjustments. After the pose adjustments are completed, a second hand-eye matrix is ​​determined based on m sets of second calibration data. Each set of second calibration data includes the second pose, the pose of the robotic arm end effector in the base coordinate system after each pose adjustment, and the pose of the calibration object in the camera coordinate system. m is an integer greater than or equal to 2.

2. The method as described in claim 1, characterized in that, The process of adjusting the position of the robotic arm's end effector n times, based on the first pose in the robotic arm's base coordinate system, includes: Based on the first pose, the robotic arm end effector is adjusted n times according to multiple preset pose offsets.

3. The method as described in claim 1, characterized in that, The determination of the first hand-eye matrix based on n sets of first calibration data includes: A first hand-eye constraint equation is determined based on every two sets of first calibration data from the n sets of first calibration data. The first hand-eye matrix is ​​determined based on the identified multiple first hand-eye constraint equations.

4. The method as described in claim 1, characterized in that, Determining the second pose of the calibrated object in the base coordinate system based on the first hand-eye matrix includes: Adjust the end effector of the robotic arm to the first position; Obtain the third pose of the calibration object in the camera coordinate system; Based on the first pose, the third pose, and the first hand-eye matrix, the second pose of the calibration object in the base coordinate system is determined.

5. The method as described in claim 4, characterized in that, Determining the second pose of the calibration object in the base coordinate system based on the first pose, the third pose, and the first hand-eye matrix includes: Based on the third pose and the first hand-eye matrix, the fourth pose of the calibration object in the end-effector coordinate system of the robotic arm is determined; Based on the first pose and the fourth pose, the second pose of the calibrated object in the base coordinate system is determined.

6. The method as described in claim 1, characterized in that, The step of adjusting the pose of the robotic arm end effector m times includes: Based on the first posture of the robotic arm in the base coordinate system and the preset posture rotation matrix, determine m second postures; The position corresponding to each second pose is determined based on the translation vector in the first hand-eye matrix, the second pose, and each of the m second poses; The end effector of the robotic arm is sequentially adjusted to m fifth poses, each of the m fifth poses including a second pose and a corresponding position.

7. The method as described in claim 1, characterized in that, The determination of the second hand-eye matrix based on m sets of second calibration data includes: A second hand-eye constraint equation is determined based on each of the m sets of second calibration data. The second hand-eye matrix is ​​determined based on the identified m second hand-eye constraint equations.

8. The method according to any one of claims 1 to 7, characterized in that, The calibration object includes a calibration plate and two calibration blocks mounted on the calibration plate, the two calibration blocks being at different distances from the calibration plate.

9. The method as described in claim 8, characterized in that, The method further includes: The Z-axis of the object coordinate system of the calibration object is determined based on the plane of the calibration plate; The origin of the object coordinate system is determined based on the center of the calibration block that is furthest from the calibration plate among the two calibration blocks. The X-axis of the object's coordinate system is determined based on the projection of the line connecting the centers of the two calibration blocks onto the plane of the calibration plate. The Y-axis of the object coordinate system is determined based on the Z-axis and X-axis of the object coordinate system.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 9.

Images